Message-ID: <4F25CCD4.8080009@gmail.com>
Date: Sun, 29 Jan 2012 23:48:52 +0100
From: Ron Arts <ron.arts@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
	rv:9.0) Gecko/20111222 Thunderbird/9.0.1
MIME-Version: 1.0
To: pgsql-performance@postgresql.org
Subject: Having I/O problems in simple virtualized environment
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi list,

I am running PostgreSQL 8.1 (CentOS 5.7) on a VM on a single XCP (Xenserver) host.
This is a HP server with 8GB, Dual Quad Core, and 2 SATA in RAID-1.

The problem is: it's running very slow compared to running it on bare metal, and
the VM is starving for I/O bandwidht, so other processes (slow to a crawl.
This does not happen on bare metal.

I had to replace the server with a bare-metal one, I could not troubleshoot in production.
Also it was hard to emulte the workload for that VM in a test environment, so I
concentrated on PostgreSQLand why it apparently generated so much I/O.

Before I start I should confess having only spotty experience with Xen and PostgreSQL
performance testing.

I setup a test Xen server created a CentOS5.7 VM with out-of-the-box PostgreSQL and ran:
pgbench -i  pgbench ; time pgbench -t 100000 pgbench
This ran for 3:28. Then I replaced the SATA HD with an SSD disk, and reran the test.
It ran for 2:46. This seemed strange as I expected the run to finish much faster.

I reran the first test on the SATA, and looked at CPU and I/O use. The CPU was not used
too much in both the VM (30%) and in dom0 (10%). The I/O use was not much as well,
around 8MB/sec in the VM. (Couldn't use iotop in dom0, because of missing kernel support
in XCP 1.1).

It reran the second test on SSD, and experienced almost the same CPU, and I/O load.

(I now probably need to run the same test on bare metal, but didn't get to that yet,
all this already ruined my weekend.)

Now I came this far, can anybody give me some pointers? Why doesn't pgbench saturate
either the CPU or the I/O? Why does using SSD only change the performance this much?

Thanks,
Ron