Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id BF9781337BA3 for ; Thu, 23 Dec 2010 15:47:14 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 96370-01-8 for ; Thu, 23 Dec 2010 19:47:06 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from proofpoint1.lanl.gov (proofpoint1.lanl.gov [204.121.3.25]) by mail.postgresql.org (Postfix) with ESMTP id 800C01337C03 for ; Thu, 23 Dec 2010 15:47:01 -0400 (AST) Received: from mailrelay2.lanl.gov (mailrelay2.lanl.gov [128.165.4.103]) by proofpoint1 (8.14.3/8.14.3) with ESMTP id oBNJkw7M018273; Thu, 23 Dec 2010 12:46:58 -0700 Received: from nis-mail.lanl.gov (nis-mail.lanl.gov [128.165.4.121]) by mailrelay2.lanl.gov (Postfix) with ESMTP id 322FF15C8D98; Thu, 23 Dec 2010 12:46:58 -0700 (MST) Received: from localhost (localhost.localdomain [127.0.0.1]) by nis-mail.lanl.gov (Postfix) with ESMTP id 2F51CDC00BD; Thu, 23 Dec 2010 12:46:58 -0700 (MST) X-NIE-2-Virus-Scanner: amavisd-new at nis-mail.lanl.gov Received: from [128.165.202.190] (procyon.lanl.gov [128.165.202.190]) by nis-mail.lanl.gov (Postfix) with ESMTP id 1FCA7DC00B2; Thu, 23 Dec 2010 12:46:58 -0700 (MST) Subject: Re: concurrent IO in postgres? From: Przemek Wozniak Reply-To: wozniak@lanl.gov To: Scott Marlowe Cc: pgsql-performance@postgresql.org In-Reply-To: References: <1293125873.3685.91.camel@procyon.lanl.gov> Content-Type: text/plain Organization: LANL Date: Thu, 23 Dec 2010 12:46:57 -0700 Message-Id: <1293133617.3685.97.camel@procyon.lanl.gov> Mime-Version: 1.0 X-Mailer: Evolution 2.22.1 (2.22.1-2.fc9) Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.2.15, 1.0.148, 0.0.0000 definitions=2010-12-23_09:2010-12-23, 2010-12-23, 1970-01-01 signatures=0 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-6.91 tagged_above=-10 required=5 tests=BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01 X-Spam-Level: X-Archive-Number: 201012/298 X-Sequence-Number: 41750 On Thu, 2010-12-23 at 11:24 -0700, Scott Marlowe wrote: > On Thu, Dec 23, 2010 at 10:37 AM, Przemek Wozniak wrote: > > When testing the IO performance of ioSAN storage device from FusionIO > > (650GB MLC version) one of the things I tried is a set of IO intensive > > operations in Postgres: bulk data loads, updates, and queries calling > > for random IO. So far I cannot make Postgres take advantage of this > > So, were you running a lot of these at once? Or just single threaded? > I get very good io concurrency with lots of parallel postgresql > connections on a 34 disk SAS setup with a battery backed controller. In one test I was running between 1 and 32 clients simultaneously writing lots of data using copy binary. The problem is that with a large RAM buffer it all goes there, and then the background writer, a single postgres process, will issue write requests one at a time I suspect. So the actual IO is effectively serialized by the backend.