KVM and Postgresql Problem:
Slow I/O – with Software Raid 1 and LVM2.

So we had some trouble with a Software Raid 1 and our KVM Virtual Machines, the writing and reading speed was just bad. But this was only  for paralle wirte/read requests. So doing single test like dd or even bonnie didn’t show the problem.

A friend of my helped me out. The problem is the writing scheduler. After i know this I found in a post here, followed by the first link:

This approach is great, but the fatal flaw is that it assumes a single, physical disk, attached to a single physical SCSI controller in a single physical host. How does the elevator algorithm know what to do when the disk is actually a RAID array? Does it? Or, what if that one Linux kernel isn’t the only kernel running on a physical host? Does the elevator mechanism still help in virtual environments?

No, no it doesn’t. Hypervisors have elevators, too. So do disk arrays. Remember that in virtual environments the hypervisor can’t tell what is happening inside the VM[0]. It’s a black box, and all it sees is the stream of I/O requests that eventually get passed to the hypervisor. It doesn’t know if they got reordered, how they got reordered, or why. It doesn’t know how long the request has been outstanding. As a result it probably won’t make the best decision about handling those requests, adding latency and extra work for the array. Worst case, all that I/O ends up looking very random to the disk array.

Fixing this using the io scheduler of the SCSI HDD/Raid 1. Disabling it via on a live system.

echo noop > /sys/block/queue/scheduler

Enabling it into the system by editing /etc/grub.conf with option

elevator=noop

I dislike MySQL. I never did like it, since Oracle bought Sun it starting to become more then just disliking it. I prefer to work with Postgresql. I want to migrate some of my MySQL DB to Postgresql.

(This is an imported feed item. You can read the original item at http://blog.akendo.eu/mysql-postgresql/)

Related to the  previors Post, i hade now to apply from virtualenv to a apache webserver. I used to deployed it with  the WSGI.

For that you need to enable the mod_wsgi in apache2.

To enable it in debian:

1
2
3

apt-getinstall libapache2-mod-wsgi
a2enmod wsgi
services apache2 restart

Now disabling the old mod_python in the configuration files. Add this to the top of your virtualhost file:

1
2
3
4
5
6
7
8
9
10

WSGIDaemonProcess $GROUPNAME python-path=/path/to/django/project/.env/lib/python2.6/site-packages user=apache group=apache processes=2threads=25
WSGIProcessGroup $GROUPNAME
WSGIScriptAlias /"/path/to/django/project/wsgi.py"
Documentroot /path/to/django/project/<Directory /path/to/django/project/><Files wsgi.py>
 Order allow,deny
 Allow from all
 Files>Directory>

This is it. Some details:

Important is to set a  WSGIDaemonProcess, it allows to specifically the resource that the WSGI Process can use. Then the $GROUPENAME(Set it as you like to), so when you have more then one WSGI implementation, that the WSGI no interfering each other. There still some more options possible.

virutalenv

Virutalenv is a python program that allows you to create system separate container with an own version of python/pip. All installed packages(with pip) are store there.

This allow a developer to install all necessarily packages into the userspace. It’s smoother and allow to have separate version of the same project, for example django.

Read here more http://blog.akendo.eu/bulding-virtualenv/

Remote access to a Postgtres Database

In the last days i have worked a lot with PostgreSQL. We have some Django Application which needs some extra SQL love. I have a installation script what insert all the script to the extra SQL into the database. The Problem: ” How to do this on a remote database host, without coping everything to it more often?”

A try with psql command show also support for remote host. The great thing about psql is that will automatic using a SSL connection.

Read here more http://blog.akendo.eu/enable-remote-access-postgresql/