Received: from localhost (maia-3.hub.org [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id 60AFE9F9C61 for ; Tue, 28 Aug 2007 10:41:04 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 02068-06 for ; Tue, 28 Aug 2007 10:40:51 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.5 Received: from web58113.mail.re3.yahoo.com (web58113.mail.re3.yahoo.com [68.142.236.136]) by postgresql.org (Postfix) with SMTP id 629BE9F9A5E for ; Tue, 28 Aug 2007 10:40:59 -0300 (ADT) Received: (qmail 11289 invoked by uid 60001); 28 Aug 2007 13:40:57 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID; b=vLgxBGSYmKp+r4pqoWwm+zzpS6EuSanooGueX4jI3ueNjVW8ZlP4cmeCrJbeBoARKDaNkETTWW4dSCgIKpq2N80uQ1Ea2bToQRMkhwuloG8oKHgSnc0d3gvE5OLOiMLrUvhwfIgmg8x0fMCNHhyecZgTCO3AAoskuvx4cZ6EwLw=; X-YMail-OSG: zh7HXgAVM1mDQZY_sc0EoghvdZUwr9BwaNPxviLCrH7N9vjYY8ejTmj_CISWwzsmigSjYD4n5eyKsegJa56bKzxfiEc6jfGzsDuxvl35iyqPpgDUXnTdoqzGGA-- Received: from [221.221.225.41] by web58113.mail.re3.yahoo.com via HTTP; Tue, 28 Aug 2007 06:40:57 PDT X-Mailer: YahooMailRC/651.48 YahooMailWebService/0.7.119 Date: Tue, 28 Aug 2007 06:40:57 -0700 (PDT) From: qinghuamail-postgresqlfans@yahoo.com Subject: =?gb2312?B?u9i4tKO6IFtBRE1JTl0ge1NwYW19INeqt6KjuiBbQURNSU5dIENvcHkgY21k?= =?gb2312?B?IGVycm9y?= To: Dimitri Fontaine , pgsql-admin@postgresql.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-682187024-1188308457=:7607" Message-ID: <781705.7607.qm@web58113.mail.re3.yahoo.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.002 tagged_above=0 required=5 tests=BAYES_50=0.001, HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 200708/346 X-Sequence-Number: 26677 --0-682187024-1188308457=:7607 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable thanks ,i read the source of pgloader,and have decided to rewrite my perl = script.=0A=0A=0A=0Athanks again.=0A=0A----- =E5=8E=9F=E5=A7=8B=E9=82=AE=E4= =BB=B6 ----=0A=E5=8F=91=E4=BB=B6=E4=BA=BA=EF=BC=9A Dimitri Fontaine =0A=E6=94=B6=E4=BB=B6=E4=BA=BA=EF=BC=9A pgsql-admin@postgr= esql.org; qinghuamail-postgresqlfans@yahoo.com=0A=E5=B7=B2=E5=8F=91=E9=80= =81=EF=BC=9A 2007/8/28(=E5=91=A8=E4=BA=8C), =E4=B8=8B=E5=8D=884:03:57=0A=E4= =B8=BB=E9=A2=98=EF=BC=9A Re: [ADMIN] {Spam} =E8=BD=AC=E5=8F=91=EF=BC=9A [AD= MIN] Copy cmd error=0A=0AHi,=0A=0ALe mardi 28 ao=C3=BBt 2007, qinghuamail-p= ostgresqlfans@yahoo.com a =C3=A9crit :=0A> hi, after reading the source of = pgloader,i have a idea,use perl to create=0A> the flat file, and, when erro= r,reject the row,seek the next line,continue=0A> to copy.=0A=0AThat's prett= y much what pgloader already does for you: if COPY errors out, =0Apgloader = split the data in two groups and try to COPY again each of them, =0Auntil t= he dichotomy isolates the one row causing COPY the error.=0AThen the given = row is appended to the configured reject_data file.=0A=0ASee those options = for details:=0A http://pgloader.projects.postgresql.org/=0A=0Acopy_every = =0A=0AWhen issuing COPY PostgreSQL commands, pgloader will not make a singl= e big =0ACOPY attempt, but copy copy_every lines at a time.=0AThis paramete= r is optionnal and defaults to 10000.=0A=0Areject_log =0A In case of errors= processing input data, a human readable log per rejected =0Ainput data lin= e is produced into the reject_log file. =0A=0A reject_data =0A In case of e= rrors processing input data, the rejected input line is appended =0Ato the = reject_data file.=0A=0AMaybe this feature should be better highlighted from= the DESCRIPTION section =0Aof the manpage (which is also the website).=0A= =0ARegards,=0A-- =0Adim=0A=0A=0A=0A=0A=0A=0A=0A ______________________= _____________________________________ =0A=E9=9B=85=E8=99=8E=E9=82=AE=E7=AE= =B1=EF=BC=8C=E4=BB=A5=E5=AE=89=E5=85=A8=E8=91=97=E7=A7=B0=EF=BC=8C=E6=98=AF= =E5=80=BC=E5=BE=97=E4=BF=A1=E8=B5=96=E7=9A=84=E9=82=AE=E7=AE=B1=E4=B8=93=E5= =AE=B6=EF=BC=81 =0Ahttp://mail.yahoo.com.cn/ --0-682187024-1188308457=:7607 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
thanks ,i read the source of pgloader,and have deci= ded to rewrite my perl  script.
=0A
=0Athanks again.

----- =E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6 ----
=E5=8F=91=E4=BB=B6= =E4=BA=BA=EF=BC=9A Dimitri Fontaine <dfontaine@hi-media.com>
=E6= =94=B6=E4=BB=B6=E4=BA=BA=EF=BC=9A pgsql-admin@postgresql.org; qinghuamail-p= ostgresqlfans@yahoo.com
=E5=B7=B2=E5=8F=91=E9=80=81=EF=BC=9A 2007/8/28(= =E5=91=A8=E4=BA=8C), =E4=B8=8B=E5=8D=884:03:57
=E4=B8=BB=E9=A2=98=EF=BC= =9A Re: [ADMIN] {Spam} =E8=BD=AC=E5=8F=91=EF=BC=9A [ADMIN] Copy cmd error
Hi,

Le mardi 28 ao=C3=BBt 2007, qinghuamail-postgresqlfan= s@yahoo.com a =C3=A9crit :
> hi, after reading the source of pgl= oader,i have a idea,use perl to create
> the flat file, and, when err= or,reject the row,seek the next line,continue
> to copy.

That'= s pretty much what pgloader already does for you: if COPY errors out,
p= gloader split the data in two groups and try to COPY again each of them, until the dichotomy isolates the one row causing COPY the error.
Then = the given row is appended to the configured reject_data file.

See those options for details:
  http://pgloader.pro= jects.postgresql.org/

copy_every

When issuing COPY Postg= reSQL commands, pgloader will not make a single big
COPY attempt, but c= opy copy_every lines at a time.
This parameter is optionnal and defaults= to 10000.

reject_log
In case of errors processing input data, = a human readable log per rejected
input data line is produced into the = reject_log file.

reject_data
In case of errors processing inp= ut data, the rejected input line is appended
to the reject_data file.
Maybe this feature should be better highlighted from the DESCRIPTION = section
of the manpage (which is also the website).

Regards,
= --
dim


=0A=0A=0A=0A
=E9=9B=85= =E8=99=8E=E9=82=AE=E7=AE=B1=EF=BC=8C=E4=BB=A5=E5=AE=89=E5=85=A8=E8=91=97=E7= =A7=B0=EF=BC=8C=E6=98=AF=E5=80=BC=E5=BE=97=E4=BF=A1=E8=B5=96=E7=9A=84=E9=82= =AE=E7=AE=B1=E4=B8=93=E5=AE=B6=EF=BC=81 --0-682187024-1188308457=:7607--