pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedLarge file support needed? Trying to identify root of error.
4+ messages / 3 participants
[nested] [flat]
* Large file support needed? Trying to identify root of error.
@ 2004-07-19 19:28 Kris Kiger <kris@musicrebellion.com>
2004-07-19 20:00 ` Re: Large file support needed? Trying to identify root of error. Tom Lane <tgl@sss.pgh.pa.us>
2004-07-19 20:07 ` Re: Large file support needed? Trying to identify root of Scott Marlowe <smarlowe@qwest.net>
0 siblings, 2 replies; 4+ messages in thread
From: Kris Kiger @ 2004-07-19 19:28 UTC (permalink / raw)
To: pgsql-admin
I've got a database that is a single table with 5 integers, a timestamp
with time zone, and a boolean. The table is 170 million rows in length.
The contents of the tar'd dump file it produced using:
pg_dump -U postgres -Ft test > test_backup.tar
is: 8.dat (approximately 8GB), a toc, and restore.sql.
No errors are reported on dump, however, when a restore is attempted I get:
ERROR: unexpected message type 0x58 during COPY from stdin
CONTEXT: COPY test_table, line 86077128: ""
ERROR: could not send data to client: Broken pipe
CONTEXT: COPY test_table, line 86077128: ""
I am doing the dump & restore on the same machine.
Any ideas? If the file is too large, is there anyway postgres could
break it up into smaller chunks for the tar when backing up? Thanks for
the help!
Kris
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Large file support needed? Trying to identify root of error.
2004-07-19 19:28 Large file support needed? Trying to identify root of error. Kris Kiger <kris@musicrebellion.com>
@ 2004-07-19 20:00 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-07-20 15:09 ` Re: Large file support needed? Trying to identify root of Kris Kiger <kris@musicrebellion.com>
1 sibling, 1 reply; 4+ messages in thread
From: Tom Lane @ 2004-07-19 20:00 UTC (permalink / raw)
To: Kris Kiger <kris@musicrebellion.com>; +Cc: pgsql-admin
Kris Kiger <kris@musicrebellion.com> writes:
> I've got a database that is a single table with 5 integers, a timestamp
> with time zone, and a boolean. The table is 170 million rows in length.
> The contents of the tar'd dump file it produced using:
> pg_dump -U postgres -Ft test > test_backup.tar
> is: 8.dat (approximately 8GB), a toc, and restore.sql.
Try -Fc instead. I have some recollection that tar format has a
hard-wired limit on the size of individual members.
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Large file support needed? Trying to identify root of
2004-07-19 19:28 Large file support needed? Trying to identify root of error. Kris Kiger <kris@musicrebellion.com>
2004-07-19 20:00 ` Re: Large file support needed? Trying to identify root of error. Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-07-20 15:09 ` Kris Kiger <kris@musicrebellion.com>
0 siblings, 0 replies; 4+ messages in thread
From: Kris Kiger @ 2004-07-20 15:09 UTC (permalink / raw)
To: pgsql-admin
Thanks Tom, -Fc worked great! I appreciate your help.
Kris
Tom Lane wrote:
>Kris Kiger <kris@musicrebellion.com> writes:
>
>
>>I've got a database that is a single table with 5 integers, a timestamp
>>with time zone, and a boolean. The table is 170 million rows in length.
>> The contents of the tar'd dump file it produced using:
>> pg_dump -U postgres -Ft test > test_backup.tar
>>is: 8.dat (approximately 8GB), a toc, and restore.sql.
>>
>>
>
>Try -Fc instead. I have some recollection that tar format has a
>hard-wired limit on the size of individual members.
>
> regards, tom lane
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Large file support needed? Trying to identify root of
2004-07-19 19:28 Large file support needed? Trying to identify root of error. Kris Kiger <kris@musicrebellion.com>
@ 2004-07-19 20:07 ` Scott Marlowe <smarlowe@qwest.net>
1 sibling, 0 replies; 4+ messages in thread
From: Scott Marlowe @ 2004-07-19 20:07 UTC (permalink / raw)
To: Kris Kiger <kris@musicrebellion.com>; +Cc: pgsql-admin
On Mon, 2004-07-19 at 13:28, Kris Kiger wrote:
> I've got a database that is a single table with 5 integers, a timestamp
> with time zone, and a boolean. The table is 170 million rows in length.
> The contents of the tar'd dump file it produced using:
> pg_dump -U postgres -Ft test > test_backup.tar
> is: 8.dat (approximately 8GB), a toc, and restore.sql.
>
> No errors are reported on dump, however, when a restore is attempted I get:
>
> ERROR: unexpected message type 0x58 during COPY from stdin
> CONTEXT: COPY test_table, line 86077128: ""
> ERROR: could not send data to client: Broken pipe
> CONTEXT: COPY test_table, line 86077128: ""
>
> I am doing the dump & restore on the same machine.
>
> Any ideas? If the file is too large, is there anyway postgres could
> break it up into smaller chunks for the tar when backing up? Thanks for
> the help!
How, exactly, are you restoring? Doing things like:
cat file|pg_restore ...
can cause problems because cat is often limited to 2 gigs on many OSes.
Just use a redirect:
psql dbname <file
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2004-07-20 15:09 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-07-19 19:28 Large file support needed? Trying to identify root of error. Kris Kiger <kris@musicrebellion.com>
2004-07-19 20:00 ` Re: Large file support needed? Trying to identify root of error. Tom Lane <tgl@sss.pgh.pa.us>
2004-07-20 15:09 ` Re: Large file support needed? Trying to identify root of Kris Kiger <kris@musicrebellion.com>
2004-07-19 20:07 ` Re: Large file support needed? Trying to identify root of Scott Marlowe <smarlowe@qwest.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox