Received: from localhost (unknown [200.46.204.183]) by postgresql.org (Postfix) with ESMTP id 4D571650E8A for ; Wed, 2 Jul 2008 07:57:14 -0300 (ADT) Received: from postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 51873-05 for ; Wed, 2 Jul 2008 07:57:06 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from fk-out-0910.google.com (fk-out-0910.google.com [209.85.128.190]) by postgresql.org (Postfix) with ESMTP id 8FD8764FD64 for ; Wed, 2 Jul 2008 07:57:10 -0300 (ADT) Received: by fk-out-0910.google.com with SMTP id e30so320052fke.3 for ; Wed, 02 Jul 2008 03:57:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=aLh0Kmq2DmakDr933FMG/4yj7PPygsNVylp1tTFpnSQ=; b=o3tlJbPSQHdn02yrYPUmfMpTcRqWuTgxZTonVAEioM0xa9Peh0EkNLK9HTXnQxynt3 QYPVNsRLvfkNkBFMUIreuTc8UETEohm9NdWXrrVRKHG7tquHldah6JpBeaghFXOhDyGI aZQStImrU5EMXEaXQxPOs6lKQVrZXD8p8n0VU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=mw4zVcIZnVSjNWCu1g6q4OlW9EFPv5gP/HkPEKvzAiVCIhj+uZQ2cCVFMwK08dBbhM va+teMxr0zpQzqr4A2ve2EnjymsbqaXUOKtvoePwMO5IBIKhTP1CkRDTsMlgNrTYN8RJ tMgHKwMluyt6Uag2eilXgRj5A8nF0abKYjhQI= Received: by 10.82.184.2 with SMTP id h2mr294958buf.11.1214996226128; Wed, 02 Jul 2008 03:57:06 -0700 (PDT) Received: by 10.82.147.4 with HTTP; Wed, 2 Jul 2008 03:57:06 -0700 (PDT) Message-ID: Date: Wed, 2 Jul 2008 06:57:06 -0400 From: "Merlin Moncure" To: "Jeffrey Baker" Subject: Re: Fusion-io ioDrive Cc: "pgsql-performance@postgresql.org" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200807/16 X-Sequence-Number: 30616 On Tue, Jul 1, 2008 at 8:18 PM, Jeffrey Baker wrote: > I recently got my hands on a device called ioDrive from a company > called Fusion-io. The ioDrive is essentially 80GB of flash on a PCI > card. It has its own driver for Linux completely outside of the > normal scsi/sata/sas/fc block device stack, but from the user's > perspective it behaves like a block device. I put the ioDrive in an > ordinary PC with 1GB of memory, a single 2.2GHz AMD CPU, and an > existing Areca RAID with 6 SATA disks and a 128MB cache. I tested the > device with PostgreSQL 8.3.3 on Centos 5.3 x86_64 (Linux 2.6.18). > > The pgbench database was initialized with scale factor 100. Test runs > were performed with 8 parallel connections (-c 8), both read-only (-S) > and read-write. PostgreSQL itself was configured with 256MB of shared > buffers and 32 checkpoint segments. Otherwise the configuration was > all defaults. > > In the following table, the "RAID" configuration has the xlogs on a > RAID 0 of 2 10krpm disks with ext2, and the heap is on a RAID 0 of 4 > 7200rpm disks with ext3. The "Fusion" configuration has everything on > the ioDrive with xfs. I tried the ioDrive with ext2 and ext3 but it > didn't seem to make any difference. > > Service Time Percentile, millis > R/W TPS R-O TPS 50th 80th 90th 95th > RAID 182 673 18 32 42 64 > Fusion 971 4792 8 9 10 11 > > Basically the ioDrive is smoking the RAID. The only real problem with > this benchmark is that the machine became CPU-limited rather quickly. > During the runs with the ioDrive, iowait was pretty well zero, with > user CPU being about 75% and system getting about 20%. > > Now, I will say a couple of other things. The Linux driver for this > piece of hardware is pretty dodgy. Sub-alpha quality actually. But > they seem to be working on it. Also there's no driver for > OpenSolaris, Mac OS X, or Windows right now. In fact there's not even > anything available for Debian or other respectable Linux distros, only > Red Hat and its clones. The other problem is the 80GB model is too > small to hold my entire DB, Although it could be used as a tablespace > for some critical tables. But hey, it's fast. > > I'm going to put this board into my 8-way Xeon to see if it goes any > faster with more CPU available. > > I'd be interested in hearing experiences with other flash storage > devices, SSDs, and that type of thing. So far, this is the fastest > hardware I've seen for the price. Any chance of getting bonnie results? How long are your pgbench runs? Are you sure that you are seeing proper syncs to the device? (this is my largest concern actually) merlin.