From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Daniel Verite <daniel@manitou-mail.org>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Jim Nasby <Jim.Nasby@bluetreble.com>
Cc: Ronan Dunklau <ronan.dunklau@dalibo.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: pg_dump / copy bugs with "big lines" ?
Date: Tue, 29 Nov 2016 14:15:20 -0300
Message-ID: <20161129171520.6s6xnr47vgaejykl@alvherre.pgsql> (raw)
In-Reply-To: <9cfb98ae-e4a4-4654-bf1a-d37e7d27c075@manitou-mail.org>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Daniel Verite wrote:
> If we consider what would happen with the latest patch on a platform
> with sizeof(int)=8 and a \copy invocation like this:
>
> \copy (select big,big,big,big,big,big,big,big,...... FROM
> (select lpad('', 1024*1024*200) as big) s) TO /dev/null
>
> if we put enough copies of "big" in the select-list to grow over 2GB,
> and then over 4GB.
Oh, right, I was forgetting that.
> On a platform with sizeof(int)=4 this should normally fail over 2GB with
> "Cannot enlarge string buffer containing $X bytes by $Y more bytes"
>
> I don't have an ILP64 environment myself to test, but I suspect
> it would malfunction instead of cleanly erroring out on such
> platforms.
I suspect nobody has such platforms, as they are mostly defunct. But I
see an easy way to fix it, which is to use sizeof(int32).
> Also, without this limit, we can "COPY FROM/TO file" really huge rows, 4GB
> and beyond, like I tried successfully during the tests mentioned upthread
> (again with len as int64 on x86_64).
> So such COPYs would succeed or fail depending on whether they deal with
> a file or a network connection.
> Do we want this difference in behavior?
Such a patch would be for master only.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@2ndquadrant.com, daniel@manitou-mail.org, tomas.vondra@2ndquadrant.com, craig@2ndquadrant.com, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, Jim.Nasby@bluetreble.com, ronan.dunklau@dalibo.com
Subject: Re: pg_dump / copy bugs with "big lines" ?
In-Reply-To: <20161129171520.6s6xnr47vgaejykl@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox