Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Sunday 8 June 2014

Postgresql Backup To Amazon S3 On OpenShift Origin

To move forward, you need to backup. Backing up your production data is critical and with Postgresql, you can backup WAL (Write Ahead Log) archives and this post gives you steps to accomplish for backing up postgresql WALs to Amazon S3 on your OpenShift Origin using WAL-E.

WAL-E is a great tool that simplifies backup of postgresql by performing continuous archiving of PostgreSQL WAL files and base backups. Enough blabbering, you can reach out technical docs on how WAL works. I'll just mention series of commands and steps necessary for sending WAL archives to AWS S3 bucket.

On the node containing application with postgresql cartridge, run the following commands:

$ yum install python-pip lzop pv
$ rpm -Uvh ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/p_conrad:/branches/Fedora_19/x86_64/daemontools-0.76-3.1.x86_64.rpm
$ pip install wal-e
$ umask u=rwx,g=rx,o=
$ mkdir -p /etc/wal-e.d/env
$ echo "secret-key-content" > /etc/wal-e.d/env/AWS_SECRET_ACCESS_KEY
$ echo "access-key" > /etc/wal-e.d/env/AWS_ACCESS_KEY_ID
$ echo 's3://backup/production/pgsql' > \
  /etc/wal-e.d/env/WALE_S3_PREFIX
$ chmod -R 765 /etc/wal-e.d/


Then, edit the postgresql configuration file so as to turn on wal archiving. You need to find the right container for your postgresql in /var/lib/openshift (Its quite trivial if you know OpenShift basics).

$ vi YOUR_OO_CONTAINER/postgresql/data/postgresql.conf wal_level = archive # hot_standby in 9.0 is also acceptable
archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env wal-e wal-push %p'
archive_timeout = 60


Finally, you need to ensure that you are taking base backups periodically which can be achieved by utilizing cron cartridge. Clone the repo, add the following file and push to the application.

$ vi .openshift/cron/daily/postgres-backup
#!/bin/bash

if [ $OPENSHIFT_POSTGRESQL_DIR ]; then
        /usr/bin/envdir /etc/wal-e.d/env /bin/wal-e backup-push ${OPENSHIFT_POSTGRESQL_DIR}data
fi

$ git add .openshift/cron/daily/postgres-backup
$ git commit -m "Added pg cron script"
$ git push origin master


Make sure you use the OPENSHIFT_POSTGRESQL_DIR env-var or some other env-var that does not have two forward slashes adjacently since WAL-E hates it.

This should help you keep your data backed up regularly and you can enjoy beers.


Read more...

Thursday 5 June 2014

Setting Up JVM Heap Size In JBoss OpenShift Origin

Openshift is an awesome technology and have fell in love with it recently. In this post, I will talk about how we can set JVM Heap Size for your application using Jboss cartridge.

If you look into the content of the standalone.conf located at $OPENSHIFT_JBOSSEAP_DIR/bin, you can see that JVM_HEAP_RATIO is set to 0.5 if it is not already set.

if [ -z "$JVM_HEAP_RATIO" ]; then
        JVM_HEAP_RATIO=0.5


And, later this ratio is used to calculate the max_heap so as to inject the maximum heap size in jboss java process. You can see how gear memory size is used to calculate the value of heap size. This is the very reason why the default installation allocates half of total gear memory size.

max_memory_mb=${OPENSHIFT_GEAR_MEMORY_MB}
max_heap=$( echo "$max_memory_mb * $JVM_HEAP_RATIO" | bc | awk '{print int($1+0.5)}')


OpenShift keeps its number of environment variables inside /var/lib/openshift/OPENSHIFT_GEAR_UUID/.env so what I did was SSH to my OO node and run the command below (you should replace your gear's UUID):

$ echo -n 0.7 > /var/lib/openshift/52e8d31bfa7c355caf000039/.env/JVM_HEAP_RATIO


Alternatively, rhc set-env JVM_HEAP_RATIO=0.7 -a appName should also work but I have not tried it.


Read more...

Sunday 10 November 2013

JPEG To PDF With Imagemagick

ImageMagick is an awesome toolkit with several powerful features for image creation and manipulation. You can use ImageMagick to translate, flip, mirror, rotate, scale, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and Bezier curves. Here, I will show how you can use ImageMagick suite to convert JPEG to PDF quickly.

First make sure imagemagick suite is installed in your system.

Ubuntu/Debian
$ sudo apt-get install imagemagick


CentOS/Fedora
$ sudo yum install imagemagick


Below are some of the examples of using convert which is a part of ImageMagick to convert Jpeg to PDF.

Single Image
$ convert image.jpg image.pdf


Multiple Images
$ convert 1.jpg 2.jpg 3.jpg output.pdf


Resize and Convert
$ convert -resize 80% image.jpg image.pdf


Negate and Convert
$ convert -negate image.jpg image.pdf


You can actually use different available switches to get your output as expected. I usually use PdfTk in conjunction with this technique to work in different scenarios and it really works great. I hope this helps :)


Read more...

Thursday 17 October 2013

How I am Trying To Keep My Eyes Safe On Computer

Lately I've been on computer a lot and with this, the usual problem with most computer users has started to bother me. Going through some of the blogs online for keeping eyes safe while using computer, I came through few suggestions and in this post, I'm writing how I'm trying to keep my eyes safe. Though not tremendously helpful for everybody, I thought I would share this and you could also use my technique.

The problem with computer addicts is not getting their eyes off the computer for much longer period and though I've been trying to remember to keep my eyes off the computer in regular interval, I usually never implement this.

My two principles based on my readings on different websites are:

  • 20-20-20: In the 20 minutes interval, keep your eyes away for 20 seconds (& view other objects which are around 20 feet away)
  • 2 hrs rule: In the 2 hours interval, stay away from computers for at least 2 minutes.

But, you can not really follow the rules so easily and I had to find some other alternative to do so. This is how I am doing it now.

Create two cron jobs for each of the above mentioned methods such that notify-send is triggered in each 20 minutes and each 2 hours informing you to keep yourself safe from computers. So my /etc/crontab looked like this:

*/20 * * * * techgaun export DISPLAY=:0.0 && /usr/bin/notify-send -i /home/techgaun/Samar/scripts/eye_inv.ico "20 - 20 - 20" "Time to take rest. Keep your eye safe :)"
01 */2 * * * techgaun export DISPLAY=:0.0 && /usr/bin/notify-send -i /home/techgaun/Samar/scripts/eye_inv.ico "2 hrs eye rest" "Time to take rest for 2 minutes. Keep your eye safe :)"


You need to replace techgaun with your username and need to give correct path to the ico file if you like to use icon like me. Otherwise, you could just omit the icon in notify-send command. I hope this proves useful for some of you :)


Read more...

Monday 16 September 2013

Two Ways To Print Lines From File Reversely

Ever tried to print lines in files in the reverse order? You will know two simple methods to print lines from file in the reverse order.

Imagine a file somefile.txt with content something like this:
a
b
c
d
e


Method 1:


$ tac somefile.txt
e
d
c
b
a


Method 2:


$ sort -r somefile.txt
e
d
c
b
a


You can achieve the same effect through other techniques as well but I'll stick to these simple ones :)


Read more...

Wednesday 7 August 2013

Compile 32 Bit Binaries On 64 Bit Machine

Well I had this special need if you recall my previous blog post since my friend had 64 bit machine. Sometimes, there might be this necessity to compile 32 bit binaries on your 64 bit machine. This post describes how to do so.

First make sure the necessary x86 libraries are installed. We require 32-bit shared libraries for AMD64 to compile binaries in 32 bit format. The command below installs the i386 version of libc6-dev:

$ sudo apt-get install libc6-dev-i386


Now you can compile your code in 32 bit binary format using the -m32 flag where 32 represents the x86 processor (-m64 would mean x64 processor).

$ gcc -m32 -o test test.c


I hope this helps :)


Read more...

Thursday 27 June 2013

Manual Sun Java Installation In Linux

Be it be multiple installations of java or be it be custom server, you might run into the necessity of manually installing java. This tutorial will provide step by step commands for installing java manually in linux.

Though the process was done on CentOS, it should work for most linux systems with or without slightest modifications. The process below installs Sun Java and configures Sun Java to be the default java to be used. Below are the steps I took to install and configure java in my system:

$ cd /opt/java
$ wget http://download.oracle.com/otn-pub/java/jdk/6u45-b15/jdk-6u45-linux-i586.tar.gz
$ tar xvfz jdk-6u45-linux-i586.tar.gz
$ echo 'export JAVA_HOME=/opt/java/jdk1.6.0_45' > /etc/profile.d/sun-jdk.sh
$ echo 'export PATH=$JAVA_HOME/bin:$PATH' >> /etc/profile.d/sun-jdk.sh
$ alternatives --install /usr/bin/java java /opt/java/jdk1.6.0_45/bin/java 2
$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)


If you wish to reconfigure the default java, you can run alternatives as below & choose the appropriate option:

$ alternatives --config java
I hope this helps :)


Read more...

Friday 21 June 2013

Share Local Directory With Remote Server During RDP Session

Well I have to constantly rdesktop to the remote servers at my workstation and sometimes I have to copy files and folders from my local machine. This post will provide you the steps on how to share files and folders with remote server during rdp session. Normally, your RDP session would start with the following command:

$ rdesktop -g90% your_server


But we wish to do something extra i.e. we need to share our directory with the remote server. The good news is that the rdesktop command supports device redirection using a -r flag which can be repeated.

Your command would look something like below:

$ rdesktop -g90% -r disk:share=/home/samar/scripts myserver


You can then access your share as a drive or media. I hope this helps :)


Read more...