Received: from desertmoon.com (desertmoon.com [198.182.124.2]) by hub.org (8.9.3/8.9.3) with SMTP id KAA89825 for ; Mon, 3 May 1999 10:00:01 -0400 (EDT) (envelope-from pierre@desertmoon.com) From: pierre@desertmoon.com Received: (qmail 8000 invoked by uid 100); 3 May 1999 14:06:53 -0000 Message-ID: <19990503140653.7998.qmail@desertmoon.com> Subject: Re: [SQL] Slow Inserts Again To: fmorton@base2inc.com (Frank Morton) Date: Mon, 3 May 1999 07:06:53 -0700 (MST) Cc: vadim@krs.ru, pgsql-sql@hub.org In-Reply-To: <06c001be956b$e35db0c0$8355e5ce@base2inc.com> from "Frank Morton" at May 3, 99 08:50:21 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Hmmm I've had problems with punctuation and stuff when importing large quantities of test into my DB. But I've always had success using copy. Have you tried using perl to munge your data and escape the appropriate characters? I've always used the following to import data into a clean DB. copy fubar from '/home/pierre/data/fubar.txt' using delimiters ','; How are you building your import files? That is how are you putting your data together? For me simply using a regex of: s/'/''/g and s/,/\\,/g on each text field BEFORE I dump it into my data file is sufficient to allow it to be imported using the copy command. So...for a table that has three varchar columns, A/B/C my data file might look like: However\, I''m here.,Don''t take me serisouly.,Hi there! The above would be imported correctly. I may be missing something as I just started reading this thread, but I hope this helps... -=pierre > > >> This last attempt, I bracket each insert statement with > > ^^^^^^^^^^^^^^^^^^^^^ > >> "begin;" and "end;". > > > >Why _each_? > >Enclose ALL statements by begin; & end; to insert ALL data > >in SINGLE transaction: > > This was suggested by someone on the list so that all > 150,000 inserts would not be treated as one large transaction. > > Like I said before, I have tried all suggestions without success. > > > > > >