pg.ddx.io pgsql-sql@postgresql.org mailing list archive
help / color / mirror / Atom feedCOPY command problems
4+ messages / 3 participants
[nested] [flat]
* COPY command problems
@ 2002-12-24 10:23 Nikola Ivacic <nikola@rs-pi.com>
0 siblings, 1 reply; 4+ messages in thread
From: Nikola Ivacic @ 2002-12-24 10:23 UTC (permalink / raw)
To: pgsql-sql
I need help
Has somebody experienced difficulties using COPY command on large files.
I have a large (250MB) file and each time I insert records I've got one or more (< 30 of cca 1079000) corrupted
records. The number of corrupted records is not constant (i.e. 1, 30, 7, 23 etc..)
p.s. the file is tab-nl separated and corruption is done in the middle of record field
(not the end or begining)
lp
Nikola
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: COPY command problems
@ 2002-12-24 15:24 Tom Lane <tgl@sss.pgh.pa.us>
parent: Nikola Ivacic <nikola@rs-pi.com>
0 siblings, 1 reply; 4+ messages in thread
From: Tom Lane @ 2002-12-24 15:24 UTC (permalink / raw)
To: Nikola Ivacic <nikola@rs-pi.com>; +Cc: pgsql-sql
"Nikola Ivacic" <nikola@rs-pi.com> writes:
> Has somebody experienced difficulties using COPY command on large files.
> I have a large (250MB) file and each time I insert records I've got one or =
> more (< 30 of cca 1079000) corrupted
> records. The number of corrupted records is not constant (i.e. 1, 30, 7, 23=
> etc..)
I'd bet on flaky hardware --- have you run memory and disk tests? If
the COPY data is passing across a network, then network problems are
also worthy of suspicion.
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: COPY command problems
@ 2002-12-24 15:55 Nikola Ivacic <nikola@rs-pi.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 4+ messages in thread
From: Nikola Ivacic @ 2002-12-24 15:55 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Ales Kuclar <ales.kuclar@rs-pi.com>; pgsql-sql
It must be internal error:
two reasons:
1.) the original file is OK (I checked with grep + there is no network
envolved)
2.) Error has strange patern: it substitutes 0x31 with 0x21 (1 with !) also
0x34 with 0x24 (4 with $)
and 0x39 with 0x29 (9 with ) )
so I guess you are right.
can you suggest some tools for FreeBSD
to test RAM, because I think the hard disk is ok.
p.s. right now I am testing it with splited file
thanks
Nikola
----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Nikola Ivacic" <nikola@rs-pi.com>
Cc: <pgsql-sql@postgresql.org>
Sent: Tuesday, December 24, 2002 4:24 PM
Subject: Re: [SQL] COPY command problems
> "Nikola Ivacic" <nikola@rs-pi.com> writes:
> > Has somebody experienced difficulties using COPY command on large files.
> > I have a large (250MB) file and each time I insert records I've got one
or =
> > more (< 30 of cca 1079000) corrupted
> > records. The number of corrupted records is not constant (i.e. 1, 30, 7,
23=
> > etc..)
>
> I'd bet on flaky hardware --- have you run memory and disk tests? If
> the COPY data is passing across a network, then network problems are
> also worthy of suspicion.
>
> regards, tom lane
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: COPY command problems
@ 2002-12-27 11:17 Rajesh Kumar Mallah. <mallah@trade-india.com>
parent: Nikola Ivacic <nikola@rs-pi.com>
0 siblings, 0 replies; 4+ messages in thread
From: Rajesh Kumar Mallah. @ 2002-12-27 11:17 UTC (permalink / raw)
To: Nikola Ivacic <nikola@rs-pi.com>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Ales Kuclar <ales.kuclar@rs-pi.com>; pgsql-sql
http://www.memtest86.com/
may be useful.
regds
mallah.
On Tuesday 24 December 2002 09:25 pm, Nikola Ivacic wrote:
> It must be internal error:
> two reasons:
> 1.) the original file is OK (I checked with grep + there is no network
> envolved)
> 2.) Error has strange patern: it substitutes 0x31 with 0x21 (1 with !) also
> 0x34 with 0x24 (4 with $)
> and 0x39 with 0x29 (9 with ) )
>
> so I guess you are right.
>
> can you suggest some tools for FreeBSD
> to test RAM, because I think the hard disk is ok.
>
> p.s. right now I am testing it with splited file
>
> thanks
>
> Nikola
>
> ----- Original Message -----
> From: "Tom Lane" <tgl@sss.pgh.pa.us>
> To: "Nikola Ivacic" <nikola@rs-pi.com>
> Cc: <pgsql-sql@postgresql.org>
> Sent: Tuesday, December 24, 2002 4:24 PM
> Subject: Re: [SQL] COPY command problems
>
> > "Nikola Ivacic" <nikola@rs-pi.com> writes:
> > > Has somebody experienced difficulties using COPY command on large
> > > files. I have a large (250MB) file and each time I insert records I've
> > > got one
>
> or =
>
> > > more (< 30 of cca 1079000) corrupted
> > > records. The number of corrupted records is not constant (i.e. 1, 30,
> > > 7,
>
> 23=
>
> > > etc..)
> >
> > I'd bet on flaky hardware --- have you run memory and disk tests? If
> > the COPY data is passing across a network, then network problems are
> > also worthy of suspicion.
> >
> > regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
--
Rajesh Kumar Mallah,
Project Manager (Development)
Infocom Network Limited, New Delhi
phone: +91(11)6152172 (221) (L) ,9811255597 (M)
Visit http://www.trade-india.com ,
India's Leading B2B eMarketplace.
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2002-12-27 11:17 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2002-12-24 10:23 COPY command problems Nikola Ivacic <nikola@rs-pi.com>
2002-12-24 15:24 ` Tom Lane <tgl@sss.pgh.pa.us>
2002-12-24 15:55 ` Nikola Ivacic <nikola@rs-pi.com>
2002-12-27 11:17 ` Rajesh Kumar Mallah. <mallah@trade-india.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox