Received: from maia.hub.org (unknown [200.46.204.183]) by mail.postgresql.org (Postfix) with ESMTP id D26C0634AC2 for ; Wed, 4 Nov 2009 21:14:27 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 41771-04 for ; Thu, 5 Nov 2009 01:14:20 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail.postnewspapers.com.au (202-89-185-120.static.dsl.amnet.net.au [202.89.185.120]) by mail.postgresql.org (Postfix) with ESMTP id CC887635B04 for ; Wed, 4 Nov 2009 21:12:23 -0400 (AST) Received: from localhost (access [127.0.0.1]) by mail.postnewspapers.com.au (Postfix) with ESMTP id BF6A91E043F; Thu, 5 Nov 2009 09:12:19 +0800 (WST) X-Virus-Scanned: Debian amavisd-new at postnewspapers.com.au Received: from mail.postnewspapers.com.au ([127.0.0.1]) by localhost (access.postnewspapers.com.au [127.0.0.1]) (amavisd-new, port 10024) with LMTP id e29pX0NlWzmB; Thu, 5 Nov 2009 09:12:18 +0800 (WST) Received: from [10.1.1.5] (124-169-36-119.dyn.iinet.net.au [124.169.36.119]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "Craig Ringer", Issuer "POST Certificate Authority" (verified OK)) by mail.postnewspapers.com.au (Postfix) with ESMTPSA id EB4601E043C; Thu, 5 Nov 2009 09:12:17 +0800 (WST) Message-ID: <4AF22671.303@postnewspapers.com.au> Date: Thu, 05 Nov 2009 09:12:17 +0800 From: Craig Ringer User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: Merlin Moncure CC: Jay Manni , "pgsql-performance@postgresql.org" Subject: Re: High Frequency Inserts to Postgres Database vs Writing to a File References: <95652FF3B9EF3045AC4AE8FE5AF8671EED65CF6B@mail2.FireEye.com> <60B0F2124D07B942988329B5B7CA393D01E5B93FF7@mail2.FireEye.com> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-2.599 tagged_above=-10 required=5 tests=BAYES_00=-2.599 X-Spam-Level: X-Archive-Number: 200911/54 X-Sequence-Number: 36222 Merlin Moncure wrote: > Postgres can handle multiple 1000 insert/sec but your hardware most > likely can't handle multiple 1000 transaction/sec if fsync is on. commit_delay or async commit should help a lot there. http://www.postgresql.org/docs/8.3/static/wal-async-commit.html http://www.postgresql.org/docs/8.3/static/runtime-config-wal.html Please do *not* turn fsync off unless you want to lose your data. > If you are bulk inserting 1000+ records/sec all day long, make sure > you have provisioned enough storage for this (that's 86M records/day), plus any index storage, room for dead tuples if you ever issue UPDATEs, etc. -- Craig Ringer