Received: from localhost (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 39B96476142 for ; Mon, 27 Jan 2003 04:44:46 -0500 (EST) Received: from www.pspl.co.in (www.pspl.co.in [202.54.11.65]) by postgresql.org (Postfix) with ESMTP id D8A28476135 for ; Mon, 27 Jan 2003 04:44:43 -0500 (EST) Received: (from root@localhost) by www.pspl.co.in (8.11.6/8.11.6) id h0R9il417579 for ; Mon, 27 Jan 2003 15:14:47 +0530 Received: from daithan (daithan.intranet.pspl.co.in [192.168.7.161]) by www.pspl.co.in (8.11.6/8.11.0) with ESMTP id h0R9il717574 for ; Mon, 27 Jan 2003 15:14:47 +0530 From: "Shridhar Daithankar" To: PgSQL Performance ML Date: Mon, 27 Jan 2003 15:15:07 +0530 MIME-Version: 1.0 Subject: Re: LOCK TABLE & speeding up mass data loads Reply-To: shridhar_daithankar@persistent.co.in Message-ID: <3E354CFB.32732.A38120C@localhost> References: In-reply-to: <1043658500.818.398.camel@haggis> X-mailer: Pegasus Mail for Windows (v4.02) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by AMaViS new-20020517 X-Archive-Number: 200301/344 X-Sequence-Number: 991 On 27 Jan 2003 at 3:08, Ron Johnson wrote: > Here's what I'd like to see: > COPY table [ ( column [, ...] ) ] > FROM { 'filename' | stdin } > [ [ WITH ] > [ BINARY ] > [ OIDS ] > [ DELIMITER [ AS ] 'delimiter' ] > [ NULL [ AS ] 'null string' ] ] > [COMMIT EVERY ... ROWS WITH LOGGING] <<<<<<<<<<<<< > [SKIP ... ROWS] <<<<<<<<<<<<< > > This way, if I'm loading 25M rows, I can have it commit every, say, > 1000 rows, and if it pukes 1/2 way thru, then when I restart the > COPY, it can SKIP past what's already been loaded, and proceed apace. IIRc, there is a hook to \copy, not the postgreSQL command copy for how many transactions you would like to see. I remember to have benchmarked that and concluded that doing copy in one transaction is the fastest way of doing it. DOn't have a postgresql installation handy, me being in linux, but this is definitely possible.. Bye Shridhar -- I still maintain the point that designing a monolithic kernel in 1991 is afundamental error. Be thankful you are not my student. You would not get ahigh grade for such a design :-)(Andrew Tanenbaum to Linus Torvalds)