From: Daniel Verite <daniel@manitou-mail.org>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Jim Nasby <Jim.Nasby@bluetreble.com>
Cc: Ronan Dunklau <ronan.dunklau@dalibo.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: pg_dump / copy bugs with "big lines" ?
Date: Thu, 03 Mar 2016 17:19:55 +0100
Message-ID: <7c0836cc-ae43-445e-9c83-ab569e2525df@mm> (raw)
In-Reply-To: <27428.1456868923@sss.pgh.pa.us>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Tom Lane wrote:
> BTW, is anyone checking the other side of this, ie "COPY IN" with equally
> wide rows? There doesn't seem to be a lot of value in supporting dump
> if you can't reload ...
Indeed, the lines bigger than 1GB can't be reloaded :(
My test: with a stock 9.5.1 plus Alvaro's patch with my additional
int64 fix mentioned upthread, creating this table and filling
all columns with 200MB of text each:
create table bigtext(t1 text ,t2 text, t3 text, t4 text,
t5 text, t6 text, t7 text);
# \copy bigtext to '/var/tmp/bigtext.sql'
That part does work as expected, producing this huge single line:
$ wc /var/tmp/bigtext.sql
1 7 1468006407 /var/tmp/bigtext.sql
But reloading it fails:
# create table bigtext2 (like bigtext);
CREATE TABLE
# copy bigtext2 from '/var/tmp/bigtext.sql';
ERROR: 54000: out of memory
DETAIL: Cannot enlarge string buffer containing 1073676288 bytes by 65536
more bytes.
CONTEXT: COPY bigtext2, line 1
LOCATION: enlargeStringInfo, stringinfo.c:278
# \copy bigtext2 from '/var/tmp/bigtext.sql'
ERROR: 54000: out of memory
DETAIL: Cannot enlarge string buffer containing 1073741808 bytes by 8191
more bytes.
CONTEXT: COPY bigtext2, line 1
LOCATION: enlargeStringInfo, stringinfo.c:278
Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: daniel@manitou-mail.org, tgl@sss.pgh.pa.us, alvherre@2ndquadrant.com, robertmhaas@gmail.com, Jim.Nasby@bluetreble.com, ronan.dunklau@dalibo.com
Subject: Re: pg_dump / copy bugs with "big lines" ?
In-Reply-To: <7c0836cc-ae43-445e-9c83-ab569e2525df@mm>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox