Received: from localhost (unknown [200.46.204.183])
	by mail.postgresql.org (Postfix) with ESMTP id CC80264FCD9
	for <pgsql-performance-postgresql.org@mail.postgresql.org>;
	Fri, 26 Dec 2008 16:18:58 -0400 (AST)
Received: from mail.postgresql.org ([200.46.204.86])
	by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024)
	with ESMTP id 72375-10
	for <pgsql-performance-postgresql.org@mail.postgresql.org>;
	Fri, 26 Dec 2008 16:18:56 -0400 (AST)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
	by mail.postgresql.org (Postfix) with ESMTP id BF0BF64FCB1
	for <pgsql-performance@postgresql.org>;
	Fri, 26 Dec 2008 16:18:55 -0400 (AST)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.14.2/8.14.2) with ESMTP id mBQKIoq0014876;
	Fri, 26 Dec 2008 15:18:50 -0500 (EST)
To: "Merlin Moncure" <mmoncure@gmail.com>
cc: "Ted Allen" <tallen@blackducksoftware.com>,
	"pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Re: Troubles dumping a very large table. 
In-reply-to: <b42b73150812261150k6b17b819i704b6b7720243eb8@mail.gmail.com> 
References: <495271EB.80508@blackducksoftware.com>
	<12778.1230140996@sss.pgh.pa.us>
	<FD2D13C2E4A95C4499BF8AA8BAB85AEC20DD41D698@BDSEXCH2K7CLS.blackducksoftware.com>
	<13141.1230313118@sss.pgh.pa.us>
	<b42b73150812261150k6b17b819i704b6b7720243eb8@mail.gmail.com>
Comments: In-reply-to "Merlin Moncure" <mmoncure@gmail.com>
	message dated "Fri, 26 Dec 2008 14:50:32 -0500"
Date: Fri, 26 Dec 2008 15:18:50 -0500
Message-ID: <14875.1230322730@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests=none
X-Spam-Level: 
X-Archive-Number: 200812/217
X-Sequence-Number: 32253

"Merlin Moncure" <mmoncure@gmail.com> writes:
> On Fri, Dec 26, 2008 at 12:38 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Yeah, the average expansion of bytea data in COPY format is about 3X :-(
>> So you need to get the max row length down to around 300mb.  I'm curious
>> how you got the data in to start with --- were the values assembled on
>> the server side?

> Wouldn't binary style COPY be more forgiving in this regard?  (if so,
> the OP might have better luck running COPY BINARY)...

Yeah, if he's willing to use COPY BINARY directly.  AFAIR there is not
an option to get pg_dump to use it.  But maybe "pg_dump -s" together
with a manual dump of the table data is the right answer.  It probably
beats shoving some of the rows aside as he's doing now...

			regards, tom lane