From: Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
To: PgSQL Performance ML <pgsql-performance@postgresql.org>
Subject: Re: LOCK TABLE & speeding up mass data loads
Date: Mon, 27 Jan 2003 15:15:07 +0530
Message-ID: <3E354CFB.32732.A38120C@localhost> (raw)
In-Reply-To: <1043658500.818.398.camel@haggis>
References: <Pine.NEB.4.51.0301270808260.596@angelic.cynic.net>
On 27 Jan 2003 at 3:08, Ron Johnson wrote:
> Here's what I'd like to see:
> COPY table [ ( column [, ...] ) ]
> FROM { 'filename' | stdin }
> [ [ WITH ]
> [ BINARY ]
> [ OIDS ]
> [ DELIMITER [ AS ] 'delimiter' ]
> [ NULL [ AS ] 'null string' ] ]
> [COMMIT EVERY ... ROWS WITH LOGGING] <<<<<<<<<<<<<
> [SKIP ... ROWS] <<<<<<<<<<<<<
>
> This way, if I'm loading 25M rows, I can have it commit every, say,
> 1000 rows, and if it pukes 1/2 way thru, then when I restart the
> COPY, it can SKIP past what's already been loaded, and proceed apace.
IIRc, there is a hook to \copy, not the postgreSQL command copy for how many
transactions you would like to see. I remember to have benchmarked that and
concluded that doing copy in one transaction is the fastest way of doing it.
DOn't have a postgresql installation handy, me being in linux, but this is
definitely possible..
Bye
Shridhar
--
I still maintain the point that designing a monolithic kernel in 1991 is
afundamental error. Be thankful you are not my student. You would not get
ahigh grade for such a design :-)(Andrew Tanenbaum to Linus Torvalds)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: shridhar_daithankar@persistent.co.in
Subject: Re: LOCK TABLE & speeding up mass data loads
In-Reply-To: <3E354CFB.32732.A38120C@localhost>
References: <Pine.NEB.4.51.0301270808260.596@angelic.cynic.net> <3E354CFB.32732.A38120C@localhost>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox