pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: qinghuamail-postgresqlfans@yahoo.com
To: Dimitri Fontaine <dfontaine@hi-media.com>
To: pgsql-admin@postgresql.org
Subject:  回复: [ADMIN] {Spam} 转发: [ADMIN] Copy cmd error
Date: Tue, 28 Aug 2007 06:40:57 -0700 (PDT)
Message-ID: <781705.7607.qm@web58113.mail.re3.yahoo.com> (raw)

thanks ,i read the source of pgloader,and have decided to rewrite my perl  script.



thanks again.

----- 原始邮件 ----
发件人: Dimitri Fontaine <dfontaine@hi-media.com>
收件人: pgsql-admin@postgresql.org; qinghuamail-postgresqlfans@yahoo.com
已发送: 2007/8/28(周二), 下午4:03:57
主题: Re: [ADMIN] {Spam} 转发: [ADMIN] Copy cmd error

Hi,

Le mardi 28 août 2007, qinghuamail-postgresqlfans@yahoo.com a écrit :
> hi, after reading the source of pgloader,i have a idea,use perl to create
> the flat file, and, when error,reject the row,seek the next line,continue
> to copy.

That's pretty much what pgloader already does for you: if COPY errors out, 
pgloader split the data in two groups and try to COPY again each of them, 
until the dichotomy isolates the one row causing COPY the error.
Then the given row is appended to the configured reject_data file.

See those options for details:
  http://pgloader.projects.postgresql.org/

copy_every 

When issuing COPY PostgreSQL commands, pgloader will not make a single big 
COPY attempt, but copy copy_every lines at a time.
This parameter is optionnal and defaults to 10000.

reject_log 
 In case of errors processing input data, a human readable log per rejected 
input data line is produced into the reject_log file. 

 reject_data 
 In case of errors processing input data, the rejected input line is appended 
to the reject_data file.

Maybe this feature should be better highlighted from the DESCRIPTION section 
of the manpage (which is also the website).

Regards,
-- 
dim







      ___________________________________________________________ 
雅虎邮箱,以安全著称,是值得信赖的邮箱专家! 
http://mail.yahoo.com.cn/

Message-ID: <781705.7607.qm@web58113.mail.re3.yahoo.com>
Permalink:  ../781705.7607.qm@web58113.mail.re3.yahoo.com/
Also on:    postgresql.org/message-id/781705.7607.qm@web58113.mail.re3.yahoo.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-admin@postgresql.org
  Cc: qinghuamail-postgresqlfans@yahoo.com, dfontaine@hi-media.com
  Subject: Re:  回复: [ADMIN] {Spam} 转发: [ADMIN] Copy cmd error
  In-Reply-To: <781705.7607.qm@web58113.mail.re3.yahoo.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox