Received: from localhost (unknown [200.46.204.184]) by developer.postgresql.org (Postfix) with ESMTP id D80972E0044 for ; Tue, 22 Apr 2008 18:55:33 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 49351-07 for ; Tue, 22 Apr 2008 18:55:22 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by developer.postgresql.org (Postfix) with ESMTP id 79B8B2E004E for ; Tue, 22 Apr 2008 18:55:27 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.14.2/8.14.2) with ESMTP id m3MLtNDf027807; Tue, 22 Apr 2008 17:55:23 -0400 (EDT) To: "Adonias Malosso" cc: pgsql-performance@postgresql.org Subject: Re: [PERFORMANCE] Error loading 37G CSV file "invalid string enlargement request size 65536" In-reply-to: <8a5d3c890804221405m69eb84b0k76f8e08b7f41952c@mail.gmail.com> References: <8a5d3c890804221405m69eb84b0k76f8e08b7f41952c@mail.gmail.com> Comments: In-reply-to "Adonias Malosso" message dated "Tue, 22 Apr 2008 18:05:35 -0300" Date: Tue, 22 Apr 2008 17:55:23 -0400 Message-ID: <27806.1208901323@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200804/333 X-Sequence-Number: 29826 "Adonias Malosso" writes: > I´m running a copy for a 37G CSV and receiving the following error: > "invalid string enlargement request size 65536" AFAICS this would only happen if you've got an individual line of COPY data exceeding 1GB. (PG versions later than 8.2 give a slightly more helpful "out of memory" error in such a case.) Most likely, that was not your intention, and the real problem is incorrect quoting/escaping in the CSV file, causing COPY to think that a large number of physical lines should be read as one logical line. regards, tom lane