pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: pierre@desertmoon.com
To: fmorton@base2inc.com
Cc: vadim@krs.ru
Cc: pgsql-sql@hub.org
Subject: Re: [SQL] Slow Inserts Again
Date: Mon, 3 May 1999 07:06:53 -0700 (MST)
Message-ID: <19990503140653.7998.qmail@desertmoon.com> (raw)
In-Reply-To: <06c001be956b$e35db0c0$8355e5ce@base2inc.com>

Hmmm
  I've had problems with punctuation and stuff when importing large quantities
  of test into my DB. But I've always had success using copy. Have you tried
  using perl to munge your data and escape the appropriate characters?

  I've always used the following to import data into a clean DB.

  copy fubar from '/home/pierre/data/fubar.txt' using delimiters ',';

  How are you building your import files? That is how are you putting your data 
  together? 

  For me simply using a regex of: s/'/''/g  and s/,/\\,/g on each text field BEFORE I 
  dump it into my data file is sufficient to allow it to be imported using the copy
  command.

  So...for a table that has three varchar columns, A/B/C my data file might look
  like:

  However\, I''m here.,Don''t take me serisouly.,Hi there!

  The above would be imported correctly. I may be missing something as I just
  started reading this thread, but I hope this helps...

  -=pierre

> 
> >> This last attempt, I bracket each insert statement with
> >                               ^^^^^^^^^^^^^^^^^^^^^
> >> "begin;" and "end;".
> >
> >Why _each_?
> >Enclose ALL statements by begin; & end; to insert ALL data
> >in SINGLE transaction:
> 
> This was suggested by someone on the list so that all
> 150,000 inserts would not be treated as one large transaction.
> 
> Like I said before, I have tried all suggestions without success.
> 
> 
> 
> 
> 
> 




view thread (16+ messages)  latest in thread

Message-ID: <19990503140653.7998.qmail@desertmoon.com>
Permalink:  ../19990503140653.7998.qmail@desertmoon.com/
Also on:    postgresql.org/message-id/19990503140653.7998.qmail@desertmoon.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-sql@postgresql.org
  Cc: pierre@desertmoon.com, fmorton@base2inc.com, vadim@krs.ru, pgsql-sql@hub.org
  Subject: Re: [SQL] Slow Inserts Again
  In-Reply-To: <19990503140653.7998.qmail@desertmoon.com>
  References: <06c001be956b$e35db0c0$8355e5ce@base2inc.com> <19990503140653.7998.qmail@desertmoon.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox