Showing posts with label centos. Show all posts
Showing posts with label centos. Show all posts

Sunday 8 June 2014

Postgresql Backup To Amazon S3 On OpenShift Origin

To move forward, you need to backup. Backing up your production data is critical and with Postgresql, you can backup WAL (Write Ahead Log) archives and this post gives you steps to accomplish for backing up postgresql WALs to Amazon S3 on your OpenShift Origin using WAL-E.

WAL-E is a great tool that simplifies backup of postgresql by performing continuous archiving of PostgreSQL WAL files and base backups. Enough blabbering, you can reach out technical docs on how WAL works. I'll just mention series of commands and steps necessary for sending WAL archives to AWS S3 bucket.

On the node containing application with postgresql cartridge, run the following commands:

$ yum install python-pip lzop pv
$ rpm -Uvh ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/p_conrad:/branches/Fedora_19/x86_64/daemontools-0.76-3.1.x86_64.rpm
$ pip install wal-e
$ umask u=rwx,g=rx,o=
$ mkdir -p /etc/wal-e.d/env
$ echo "secret-key-content" > /etc/wal-e.d/env/AWS_SECRET_ACCESS_KEY
$ echo "access-key" > /etc/wal-e.d/env/AWS_ACCESS_KEY_ID
$ echo 's3://backup/production/pgsql' > \
  /etc/wal-e.d/env/WALE_S3_PREFIX
$ chmod -R 765 /etc/wal-e.d/


Then, edit the postgresql configuration file so as to turn on wal archiving. You need to find the right container for your postgresql in /var/lib/openshift (Its quite trivial if you know OpenShift basics).

$ vi YOUR_OO_CONTAINER/postgresql/data/postgresql.conf wal_level = archive # hot_standby in 9.0 is also acceptable
archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env wal-e wal-push %p'
archive_timeout = 60


Finally, you need to ensure that you are taking base backups periodically which can be achieved by utilizing cron cartridge. Clone the repo, add the following file and push to the application.

$ vi .openshift/cron/daily/postgres-backup
#!/bin/bash

if [ $OPENSHIFT_POSTGRESQL_DIR ]; then
        /usr/bin/envdir /etc/wal-e.d/env /bin/wal-e backup-push ${OPENSHIFT_POSTGRESQL_DIR}data
fi

$ git add .openshift/cron/daily/postgres-backup
$ git commit -m "Added pg cron script"
$ git push origin master


Make sure you use the OPENSHIFT_POSTGRESQL_DIR env-var or some other env-var that does not have two forward slashes adjacently since WAL-E hates it.

This should help you keep your data backed up regularly and you can enjoy beers.


Read more...

Thursday 5 June 2014

Setting Up JVM Heap Size In JBoss OpenShift Origin

Openshift is an awesome technology and have fell in love with it recently. In this post, I will talk about how we can set JVM Heap Size for your application using Jboss cartridge.

If you look into the content of the standalone.conf located at $OPENSHIFT_JBOSSEAP_DIR/bin, you can see that JVM_HEAP_RATIO is set to 0.5 if it is not already set.

if [ -z "$JVM_HEAP_RATIO" ]; then
        JVM_HEAP_RATIO=0.5


And, later this ratio is used to calculate the max_heap so as to inject the maximum heap size in jboss java process. You can see how gear memory size is used to calculate the value of heap size. This is the very reason why the default installation allocates half of total gear memory size.

max_memory_mb=${OPENSHIFT_GEAR_MEMORY_MB}
max_heap=$( echo "$max_memory_mb * $JVM_HEAP_RATIO" | bc | awk '{print int($1+0.5)}')


OpenShift keeps its number of environment variables inside /var/lib/openshift/OPENSHIFT_GEAR_UUID/.env so what I did was SSH to my OO node and run the command below (you should replace your gear's UUID):

$ echo -n 0.7 > /var/lib/openshift/52e8d31bfa7c355caf000039/.env/JVM_HEAP_RATIO


Alternatively, rhc set-env JVM_HEAP_RATIO=0.7 -a appName should also work but I have not tried it.


Read more...

Monday 18 November 2013

Install HTTrack On CentOS

Since I could not find the rpm in the repo, here is the quick How To to install HTTrack website copier on CentOS.

$ yum install zlib-devel
$ wget http://download.httrack.com/cserv.php3?File=httrack.tar.gz -O httrack.tar.gz
$ tar xvfz httrack.tar.gz
$ cd httrack-3.47.27
$ ./configure
$ make && sudo make install


This should do all. If you wish not to install zlib compression support, you can skip the first step and run the configure as ./configure --without-zlib. I hope this helps :)


Read more...

Thursday 17 October 2013

How I am Trying To Keep My Eyes Safe On Computer

Lately I've been on computer a lot and with this, the usual problem with most computer users has started to bother me. Going through some of the blogs online for keeping eyes safe while using computer, I came through few suggestions and in this post, I'm writing how I'm trying to keep my eyes safe. Though not tremendously helpful for everybody, I thought I would share this and you could also use my technique.

The problem with computer addicts is not getting their eyes off the computer for much longer period and though I've been trying to remember to keep my eyes off the computer in regular interval, I usually never implement this.

My two principles based on my readings on different websites are:

  • 20-20-20: In the 20 minutes interval, keep your eyes away for 20 seconds (& view other objects which are around 20 feet away)
  • 2 hrs rule: In the 2 hours interval, stay away from computers for at least 2 minutes.

But, you can not really follow the rules so easily and I had to find some other alternative to do so. This is how I am doing it now.

Create two cron jobs for each of the above mentioned methods such that notify-send is triggered in each 20 minutes and each 2 hours informing you to keep yourself safe from computers. So my /etc/crontab looked like this:

*/20 * * * * techgaun export DISPLAY=:0.0 && /usr/bin/notify-send -i /home/techgaun/Samar/scripts/eye_inv.ico "20 - 20 - 20" "Time to take rest. Keep your eye safe :)"
01 */2 * * * techgaun export DISPLAY=:0.0 && /usr/bin/notify-send -i /home/techgaun/Samar/scripts/eye_inv.ico "2 hrs eye rest" "Time to take rest for 2 minutes. Keep your eye safe :)"


You need to replace techgaun with your username and need to give correct path to the ico file if you like to use icon like me. Otherwise, you could just omit the icon in notify-send command. I hope this proves useful for some of you :)


Read more...

Monday 16 September 2013

Two Ways To Print Lines From File Reversely

Ever tried to print lines in files in the reverse order? You will know two simple methods to print lines from file in the reverse order.

Imagine a file somefile.txt with content something like this:
a
b
c
d
e


Method 1:


$ tac somefile.txt
e
d
c
b
a


Method 2:


$ sort -r somefile.txt
e
d
c
b
a


You can achieve the same effect through other techniques as well but I'll stick to these simple ones :)


Read more...

Thursday 27 June 2013

Manual Sun Java Installation In Linux

Be it be multiple installations of java or be it be custom server, you might run into the necessity of manually installing java. This tutorial will provide step by step commands for installing java manually in linux.

Though the process was done on CentOS, it should work for most linux systems with or without slightest modifications. The process below installs Sun Java and configures Sun Java to be the default java to be used. Below are the steps I took to install and configure java in my system:

$ cd /opt/java
$ wget http://download.oracle.com/otn-pub/java/jdk/6u45-b15/jdk-6u45-linux-i586.tar.gz
$ tar xvfz jdk-6u45-linux-i586.tar.gz
$ echo 'export JAVA_HOME=/opt/java/jdk1.6.0_45' > /etc/profile.d/sun-jdk.sh
$ echo 'export PATH=$JAVA_HOME/bin:$PATH' >> /etc/profile.d/sun-jdk.sh
$ alternatives --install /usr/bin/java java /opt/java/jdk1.6.0_45/bin/java 2
$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)


If you wish to reconfigure the default java, you can run alternatives as below & choose the appropriate option:

$ alternatives --config java
I hope this helps :)


Read more...