From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@commandprompt.com>
Cc: Johann Spies <jspies@sun.ac.za>
Cc: pgsql-admin@postgresql.org
Subject: Re: ERROR: invalid memory alloc request size
Date: Mon, 28 May 2007 11:18:37 -0400
Message-ID: <28065.1180365517@sss.pgh.pa.us> (raw)
In-Reply-To: <20070528144651.GA7795@alvh.no-ip.org>
References: <11217.1179153258@sss.pgh.pa.us>
<20070515105042.GA8121@burger.sun.ac.za>
<5221.1179337804@sss.pgh.pa.us>
<20070517132033.GA9482@sun.ac.za>
<25608.1179412804@sss.pgh.pa.us>
<20070518122942.GA28931@sun.ac.za>
<27574.1179501021@sss.pgh.pa.us>
<20070521140718.GC10742@sun.ac.za>
<10687.1179758440@sss.pgh.pa.us>
<20070528134559.GC5615@sun.ac.za>
<20070528144651.GA7795@alvh.no-ip.org>
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Johann Spies wrote:
>> #3 0x00000000005ff1c8 in perform_default_encoding_conversion (
>> src=0x2b881e87f040 "\"2007-04-20\" \"18:07:06\" \"192.168.0.100\" \"73\" \"http://www.kerkbode.co.za/kerkbode/images/blank.gif\\\"\"\n\"2007-04-20\" \"18:07:06\" \"192.168.0.100\" \"69\" \"http://develop.christians.co.za/phpAdsNew/adx.js\"\n\"20"..., len=337739539, is_client_to_server=5 '\005') at mbutils.c:461
> Interesting. You are passing a single 300 MB string here, not
> 90-something char lines. The problem is probably in newlines. Why are
> they not getting used as line separators escapes me.
Stripped of gdb's own backslashing, the data seems to be
...images/blank.gif\""
"2007...
which leads me to think that the dump was generated on the assumption
that backslash is the escape character, but is being reloaded under a
different assumption. Once you got out-of-sync on that one data value,
you'd have a mess --- all the intended line-ending newlines would be
inside quotes, up till you hit another URL with an embedded quote
(or newline?). CSV is not exactly a robust representation :-(
So the short answer seems to be that we were all misled by the large
alloc-request number into thinking there was some data-corruption
behavior involved, when we should have been quizzing Johann on exactly
what COPY parameters he was giving.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: tgl@sss.pgh.pa.us, alvherre@commandprompt.com, jspies@sun.ac.za
Subject: Re: ERROR: invalid memory alloc request size
In-Reply-To: <28065.1180365517@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox