From: Ross J. Reedstrom <reedstrm@rice.edu>
To: Abdul Karim <karim@thehub.co.uk>
Cc: pgsql-sql@postgresql.org
Subject: Re: importing in sql
Date: Wed, 12 Jul 2000 12:55:57 -0500
Message-ID: <20000712125557.A3092@rice.edu> (raw)
In-Reply-To: <396A481C.D8D4E21@thehub.co.uk>
References: <396A481C.D8D4E21@thehub.co.uk>
On Mon, Jul 10, 2000 at 05:03:08PM -0500, Abdul Karim wrote:
> Hi, I am trying to import loads of data into postgres, I am having
> trouble with a field which is a sequence. I know how to import data
> using a delimiter with the copy command. But I need to know how I
> increase the sequence on each line of import.
>
> My Table has the following format.
>
> field1 serial primary key,
> field2 char (50),
> field3 char(50),
> field4 int,
> field5 date
>
> The file has the following format
>
> field2|field3|field4|field5|
>
> Each field is separated by a | (pipe), How do I insert the sequence
> number before field2? I have like 30 files in this format and I need to
> import them in one table with each line having unique number. Has
> anyone come across similar situation? any help would be greatly
> appreciated.
>
you don't mention your operating system, but on Linux I'd do this
with awk:
awk '{print NR"|"$0}' data_file > data_file.out
You mention 30 files: to get them sequential, I'd do:
cat files1 file2 [...] | awk '{print NR"|"$0}' | split -C 1m - dataout
Use some glob pattern for the 'cat' command that puts the files in the
order you want (even if it's just listing them all individually)
Awk will then number them, and split will generate file with at most 1
Meg of lines in them. If you're not worried about doing an all in one
bulk load, skip the split.
After this is all loaded, be sure to set the sequence associated with
the serial filed you loading into:
SELECT setval('table_field_seq',max(field)) from table;
Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-sql@postgresql.org
Cc: reedstrm@rice.edu, karim@thehub.co.uk
Subject: Re: importing in sql
In-Reply-To: <20000712125557.A3092@rice.edu>
References: <396A481C.D8D4E21@thehub.co.uk> <20000712125557.A3092@rice.edu>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox