X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id 31026D1D2E3 for ; Wed, 24 Mar 2004 00:50:27 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 15123-04 for ; Tue, 23 Mar 2004 20:50:27 -0400 (AST) Received: from candle.pha.pa.us (candle.pha.pa.us [207.106.42.251]) by svr1.postgresql.org (Postfix) with ESMTP id 21C93D1EC49 for ; Tue, 23 Mar 2004 20:50:22 -0400 (AST) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.11.6) id i2O0oOn19677; Tue, 23 Mar 2004 19:50:24 -0500 (EST) From: Bruce Momjian Message-Id: <200403240050.i2O0oOn19677@candle.pha.pa.us> Subject: Re: [Retrieved]RE: backup and recovery In-Reply-To: <5.1.0.14.2.20040323170131.05e1e290@imap.eldocomp.com> To: Naomi Walker Date: Tue, 23 Mar 2004 19:50:24 -0500 (EST) Cc: "Mark M. Huber" , "pgsql-admin@postgresql.org" X-Mailer: ELM [version 2.4ME+ PL108 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200403/310 X-Sequence-Number: 12879 Naomi Walker wrote: > > I'm not sure of the correct protocol for getting things on the "todo" > list. Whom shall we beg? > Uh, you just ask and we discuss it on the list. Are you using INSERTs from pg_dump? I assume so because COPY uses a single transaction per command. Right now with pg_dump -d I see: -- -- Data for Name: has_oids; Type: TABLE DATA; Schema: public; Owner: postgres -- INSERT INTO has_oids VALUES (1); INSERT INTO has_oids VALUES (1); INSERT INTO has_oids VALUES (1); INSERT INTO has_oids VALUES (1); Seems that should be inside a BEGIN/COMMIT for performance reasons, and to have the same behavior as COPY (fail if any row fails). Commands? As far as skipping on errors, I am unsure on that one, and if we put the INSERTs in a transaction, we will have no way of rolling back only the few inserts that fail. --------------------------------------------------------------------------- > > > >That brings up a good point. It would be extremely helpful to add two > >parameters to pg_dump. One, to add how many rows to insert before a > >commit, and two, to live through X number of errors before dying (and > >putting the "bad" rows in a file). > > > > > >At 10:15 AM 3/19/2004, Mark M. Huber wrote: > > >What it was that I guess the pg_dump makes one large transaction and our > > >shell script wizard wrote a perl program to add a commit transaction > > >every 500 rows or what every you set. Also I should have said that we were > > >doing the recovery with the insert statements created from pg_dump. So... > > >my 500000 row table recovery took < 10 Min. > > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073