X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 64CFDD8D39 for ; Tue, 13 Sep 2005 11:10:07 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 75580-06 for ; Tue, 13 Sep 2005 14:10:01 +0000 (GMT) Received: from ns.snowman.net (ns.snowman.net [66.92.160.21]) by svr1.postgresql.org (Postfix) with ESMTP id 464F2D862D for ; Tue, 13 Sep 2005 11:10:02 -0300 (ADT) Received: by ns.snowman.net (Postfix, from userid 1000) id 685AB17AD3; Tue, 13 Sep 2005 10:10:31 -0400 (EDT) Date: Tue, 13 Sep 2005 10:10:31 -0400 From: Stephen Frost To: Brandon Black Cc: Greg Stark , pgsql-performance@postgresql.org Subject: Re: Performance considerations for very heavy INSERT traffic Message-ID: <20050913141031.GJ6026@ns.snowman.net> Mail-Followup-To: Brandon Black , Greg Stark , pgsql-performance@postgresql.org References: <84621a6050912140464c380c3@mail.gmail.com> <87slw91x8a.fsf@stark.xeocode.com> <84621a605091220442775249b@mail.gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WI3x0E7Qla8PXOxb" Content-Disposition: inline In-Reply-To: <84621a605091220442775249b@mail.gmail.com> X-Editor: Vim http://www.vim.org/ X-Info: http://www.snowman.net X-Operating-System: Linux/2.4.24ns.3.0 (i686) X-Uptime: 10:08:07 up 94 days, 6:24, 11 users, load average: 0.01, 0.09, 0.09 User-Agent: Mutt/1.5.9i X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.004 required=5 tests=[AWL=0.004] X-Spam-Level: X-Archive-Number: 200509/162 X-Sequence-Number: 14470 --WI3x0E7Qla8PXOxb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Brandon Black (blblack@gmail.com) wrote: > Ideally I'd like to commit the data seperately, as the data could contain= =20 > errors which abort the transaction, but it may come down to batching it a= nd=20 > coding things such that I can catch and discard the offending row and ret= ry=20 > the transaction if it fails (which should be fairly rare I would hope). I= =20 Don't really know if it'd be better, or worse, or what, but another thought to throw out there is to perhaps use savepoints instead of full transactions? Not sure if that's more expensive or cheaper than doing full commits but it might be something to consider. > When I finally get all of this sorted out and working reasonably optimall= y,=20 > I'll be sure to come back and report what techniques/settings did and did= n't=20 > work for this workload. That'd be great, many thanks, Stephen --WI3x0E7Qla8PXOxb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDJt3WrzgMPqB3kigRAtKVAJ4hZygNkds1YkkSHMTzcnq4w4nruQCgnUzG XE5xhwTNvrQjpMchqZil9CA= =cq5c -----END PGP SIGNATURE----- --WI3x0E7Qla8PXOxb--