From: Stephen Frost <sfrost@snowman.net>
To: Charles Gomes <charlesrg@outlook.com>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: Performance on Bulk Insert to Partitioned Table
Date: Thu, 20 Dec 2012 15:02:34 -0500
Message-ID: <20121220200234.GJ12354@tamriel.snowman.net> (raw)
In-Reply-To: <BLU002-W2777115565E1CBCF867857AB370@phx.gbl>
References: <BLU002-W2777115565E1CBCF867857AB370@phx.gbl>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
Charles,
* Charles Gomes (charlesrg@outlook.com) wrote:
> I’m doing 1.2 Billion inserts into a table partitioned in
> 15.
Do you end up having multiple threads writing to the same, underlying,
tables..? If so, I've seen that problem before. Look at pg_locks while
things are running and see if there are 'extend' locks that aren't being
immediately granted.
Basically, there's a lock that PG has on a per-relation basis to extend
the relation (by a mere 8K..) which will block other writers. If
there's a lot of contention around that lock, you'll get poor
performance and it'll be faster to have independent threads writing
directly to the underlying tables. I doubt rewriting the trigger in C
will help if the problem is the extent lock.
If you do get this working well, I'd love to hear what you did to
accomplish that. Note also that you can get bottle-necked on the WAL
data, unless you've taken steps to avoid that WAL.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 197B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: sfrost@snowman.net, charlesrg@outlook.com
Subject: Re: Performance on Bulk Insert to Partitioned Table
In-Reply-To: <20121220200234.GJ12354@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox