Content-Type: text/plain; charset="iso-8859-15"
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: pg_dump / copy bugs with "big lines" ?
From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Alvaro Herrera" <alvherre@2ndquadrant.com>
CC: "Tom Lane" <tgl@sss.pgh.pa.us>,"Robert Haas" <robertmhaas@gmail.com>,"Jim
 Nasby" <Jim.Nasby@bluetreble.com>,"Ronan Dunklau"
 <ronan.dunklau@dalibo.com>,"pgsql-hackers" <pgsql-hackers@postgresql.org>
In-Reply-To: <7c0836cc-ae43-445e-9c83-ab569e2525df@mm>
Date: Wed, 16 Mar 2016 14:02:07 +0100
Message-ID: <28a1f376-e006-4ecf-93f5-133737652c5c@mm>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

	Daniel Verite wrote:

> # \copy bigtext2 from '/var/tmp/bigtext.sql'
> ERROR:  54000: out of memory
> DETAIL:  Cannot enlarge string buffer containing 1073741808 bytes by 8191
> more bytes.
> CONTEXT:  COPY bigtext2, line 1
> LOCATION:  enlargeStringInfo, stringinfo.c:278

To go past that problem, I've tried tweaking the StringInfoData
used for COPY FROM, like the original patch does in CopyOneRowTo.=20

It turns out that it fails a bit later when trying to make a tuple
from the big line, in heap_form_tuple():

  tuple =3D (HeapTuple) palloc0(HEAPTUPLESIZE + len);

which fails because (HEAPTUPLESIZE + len) is again considered
an invalid size, the  size being 1468006476 in my test.

At this point it feels like a dead end, at least for the idea that extending
StringInfoData might suffice to enable COPYing such large rows.

Best regards,
--=20
Daniel V=E9rit=E9
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite


--=20
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers