Received: from localhost (maia-3.hub.org [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id BDDA89FB298 for ; Mon, 28 May 2007 12:18:47 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 12929-08 for ; Mon, 28 May 2007 12:18:39 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id B6E179FB290 for ; Mon, 28 May 2007 12:18:39 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.14.1/8.14.1) with ESMTP id l4SFIbpE028066; Mon, 28 May 2007 11:18:37 -0400 (EDT) To: Alvaro Herrera cc: Johann Spies , pgsql-admin@postgresql.org Subject: Re: ERROR: invalid memory alloc request size In-reply-to: <20070528144651.GA7795@alvh.no-ip.org> References: <11217.1179153258@sss.pgh.pa.us> <20070515105042.GA8121@burger.sun.ac.za> <5221.1179337804@sss.pgh.pa.us> <20070517132033.GA9482@sun.ac.za> <25608.1179412804@sss.pgh.pa.us> <20070518122942.GA28931@sun.ac.za> <27574.1179501021@sss.pgh.pa.us> <20070521140718.GC10742@sun.ac.za> <10687.1179758440@sss.pgh.pa.us> <20070528134559.GC5615@sun.ac.za> <20070528144651.GA7795@alvh.no-ip.org> Comments: In-reply-to Alvaro Herrera message dated "Mon, 28 May 2007 10:46:51 -0400" Date: Mon, 28 May 2007 11:18:37 -0400 Message-ID: <28065.1180365517@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200705/292 X-Sequence-Number: 25612 Alvaro Herrera writes: > Johann Spies wrote: >> #3 0x00000000005ff1c8 in perform_default_encoding_conversion ( >> src=0x2b881e87f040 "\"2007-04-20\" \"18:07:06\" \"192.168.0.100\" \"73\" \"http://www.kerkbode.co.za/kerkbode/images/blank.gif\\\"\"\n\"2007-04-20\" \"18:07:06\" \"192.168.0.100\" \"69\" \"http://develop.christians.co.za/phpAdsNew/adx.js\"\n\"20"..., len=337739539, is_client_to_server=5 '\005') at mbutils.c:461 > Interesting. You are passing a single 300 MB string here, not > 90-something char lines. The problem is probably in newlines. Why are > they not getting used as line separators escapes me. Stripped of gdb's own backslashing, the data seems to be ...images/blank.gif\"" "2007... which leads me to think that the dump was generated on the assumption that backslash is the escape character, but is being reloaded under a different assumption. Once you got out-of-sync on that one data value, you'd have a mess --- all the intended line-ending newlines would be inside quotes, up till you hit another URL with an embedded quote (or newline?). CSV is not exactly a robust representation :-( So the short answer seems to be that we were all misled by the large alloc-request number into thinking there was some data-corruption behavior involved, when we should have been quizzing Johann on exactly what COPY parameters he was giving. regards, tom lane