From: Tom Lane <tgl@sss.pgh.pa.us>
To: scott.marlowe <scott.marlowe@ihs.com>
Cc: Kevin White <kwhite@digital-ics.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Really bad insert performance: what did I do wrong?
Date: Fri, 21 Feb 2003 23:23:47 -0500
Message-ID: <2703.1045887827@sss.pgh.pa.us> (raw)
In-Reply-To: <Pine.LNX.4.33.0302211049090.17876-100000@css120.ihs.com>
References: <Pine.LNX.4.33.0302211049090.17876-100000@css120.ihs.com>
"scott.marlowe" <scott.marlowe@ihs.com> writes:
> 3: Inserting ALL 700,000 rows in one transaction is probably not optimal.
> Try putting a test in every 1,000 or 10,000 rows to toss a "commit;begin;"
> pair at the database while loading. Inserting all 700,000 rows at once
> means postgresql can't recycle the transaction logs, so you'll have
> 700,000 rows worth of data in the transaction logs waiting for you to
> commit at the end.
That was true in 7.1.0, but we got rid of that behavior *very* quickly
(by 7.1.3, according to the release notes). Long transactions do not
currently stress the WAL storage any more than the same amount of work
in short transactions.
Which is not to say that there's anything wrong with divvying the work
into 1000-row-or-so transactions. I agree that that's enough to push
the per-transaction overhead down into the noise.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: tgl@sss.pgh.pa.us, scott.marlowe@ihs.com, kwhite@digital-ics.com
Subject: Re: Really bad insert performance: what did I do wrong?
In-Reply-To: <2703.1045887827@sss.pgh.pa.us>
References: <Pine.LNX.4.33.0302211049090.17876-100000@css120.ihs.com> <2703.1045887827@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox