Date: Tue, 26 Oct 2010 13:41:44 +0200
From: =?utf-8?Q?Martin_Povolny?= <martin.povolny@solnet.cz>
To: pgsql-admin@postgresql.org
Subject: 
 =?utf-8?Q?large_database=3A_problems_with_pg=5Fdump_and_pg=5Frestore?=
Mime-Version: 1.0
Content-Type: multipart/alternative;
	boundary=mimepart_4cc6be7a60314_19e4..fdbeb90dc5e6
User-Agent: InChange
Message-Id: <E1PAhuU-0006Fa-DI@ns.solnet.cz>


--mimepart_4cc6be7a60314_19e4..fdbeb90dc5e6
Content-Type: text/plain; charset=utf-8

Hallo,

I have some quite grave problems with dumping and restoring large databases (>4GB of
dump).

I had 5 databases, 4 dumped ok, the 5th, the largest failed dumping: I was unable to
make a dump in the default 'tar' format. I got this message:

pg_dump: [tar archiver] archive member too large for tar format

I got over this issue by using the 'custom' format.

Unfortunately later on I was only able to restore 3 of the 5 databases -- any of the
2 dumps that would get over 4GB in the 'tar' format would fail.

/var/tmp# ls -l dumps/
total 16294020
-rw-r--r-- 1 root root  742611968 2010-10-16 20:36 archiv1.dump
-rw-r--r-- 1 root root  317352448 2010-10-16 20:37 archiv2.dump
-rw-r--r-- 1 root root 1137477632 2010-10-16 20:41 archiv3.dump
-rw-r--r-- 1 root root 3712833536 2010-10-16 20:57 archiv4.dump
-rw-r--r-- 1 root root 8735429632 2010-10-16 21:35 archiv5.dump
-rw-r--r-- 1 root root 1253325453 2010-10-16 22:59 bb.dump

archiv1-4 went OK, archiv5 -- the largest in 'tar' format -- failed and the 'bb.dump'
which is in the 'custom' format, failed too.

I got these messages:

for the archiv5 in the 'tar' format:

pg_restore: [tar archivář] nalezena poškozená tar hlavička v STEX
(předpokládáno 100, vypočteno 34044) pozice souboru 7750193152

sorry, it's in my native locale, but is says "found a corrupted tar header in STEX
(expected 100, calculated 34044) file position 7750193152

for the bb.dump in the 'custom' format:

pg_restore: [vlastní archivář] unexpected end of file

'vlastní archivář' is again in my locale, it should be in English "own archiver"

Later I tried to utilize the -I and -i switches of pg_restore to restore data that
are in the archive behing the table that was not restored. But got the same error
message.

The dump was created on postgresql-8.3  8.3.3-1~bpo40+1 from debian backports. I was
trying to restore on this version and later on using postgresql-8.4 8.4.5-1~bpo50+1
from debian backports, finally I tried 64bit version
of postgresql-8.4 8.4.5-0ubuntu10.10. No change, still the same error messages.

I welcome any help and/or hints on this issue as I need to dump and restore several
large databases.

Regards,
--
Mgr. Martin Povolný, soLNet, s.r.o.,
+420777714458, martin.povolny@solnet.cz

--mimepart_4cc6be7a60314_19e4..fdbeb90dc5e6
Content-Type: text/html; charset=utf-8

Hallo,&nbsp;<div><br></div><div>I have some quite grave problems with dumping and restoring large databases (&gt;4GB of dump).<br><br></div><div>I had 5 databases, 4 dumped ok, the 5th, the largest failed dumping: I was unable to make a dump in the default 'tar' format. I got this message:</div><div><br></div><div>pg_dump: [tar archiver] archive member too large for tar format</div><div><br></div><div>I got over this issue by using the 'custom' format.</div><div><br></div><div>Unfortunately later on I was only able to restore 3 of the 5 databases -- any of the 2 dumps that would get over 4GB in the 'tar' format would fail.</div><div><br></div><div><div>/var/tmp# ls -l dumps/</div><div>total 16294020</div><div>-rw-r--r-- 1 root root &nbsp;742611968 2010-10-16 20:36 archiv1.dump</div><div>-rw-r--r-- 1 root root &nbsp;317352448 2010-10-16 20:37 archiv2.dump</div><div>-rw-r--r-- 1 root root 1137477632 2010-10-16 20:41 archiv3.dump</div><div>-rw-r--r-- 1 root root 3712833536 2010-
 10-16 20:57 archiv4.dump</div><div>-rw-r--r-- 1 root root 8735429632 2010-10-16 21:35 archiv5.dump</div><div>-rw-r--r-- 1 root root 1253325453 2010-10-16 22:59 bb.dump</div></div><div><br></div><div>archiv1-4 went OK, archiv5 -- the largest in 'tar' format -- failed and the 'bb.dump' which is in the 'custom' format, failed too.</div><div><br></div><div>I got these messages:</div><div><br></div><div>for the archiv5 in the 'tar' format:&nbsp;</div><div><br></div><div><div>pg_restore: [tar archivář] nalezena poškozená tar hlavička v STEX (předpokládáno 100, vypočteno 34044) pozice souboru 7750193152</div></div><div><br></div><div>sorry, it's in my native locale, but is says "found a corrupted tar header in STEX (expected 100, calculated 34044) file position&nbsp;7750193152</div><div><br></div><div>for the bb.dump in the 'custom' format:</div><div><br></div><div><div>pg_restore: [vlastní archivář] unexpected end of file</div></div><div><br></div><div>'vlastní archiv
 ář' is again in my locale, it should be in English "own archiver"</div><div><br></div><div>Later I tried to utilize the -I and -i switches of pg_restore to restore data that are in the archive behing the table that was not restored. But got the same error message.</div><div><br></div><div>The dump was created on&nbsp;postgresql-8.3 &nbsp;8.3.3-1~bpo40+1 from debian backports. I was trying to restore on this version and later on using postgresql-8.4&nbsp;8.4.5-1~bpo50+1 from debian backports, finally I tried 64bit version of&nbsp;postgresql-8.4&nbsp;8.4.5-0ubuntu10.10. No change, still the same error messages.</div><div><br></div><div>I welcome any help and/or hints on this issue as I need to dump and restore several large databases.</div><div><br></div><div>Regards,</div><div><br>--<br>Mgr. Martin Povolný, soLNet, s.r.o.,<br>+420777714458, martin.povolny@solnet.cz</div>
--mimepart_4cc6be7a60314_19e4..fdbeb90dc5e6--