pg.ddx.io pgsql-sql@postgresql.org mailing list archive
help / color / mirror / Atom feedimporting in sql
3+ messages / 3 participants
[nested] [flat]
* importing in sql
@ 2000-07-10 22:03 Abdul Karim <karim@thehub.co.uk>
2000-07-12 17:55 ` Re: importing in sql Ross J. Reedstrom <reedstrm@rice.edu>
2000-07-12 18:19 ` Re: importing in sql jmr@computing.com
0 siblings, 2 replies; 3+ messages in thread
From: Abdul Karim @ 2000-07-10 22:03 UTC (permalink / raw)
To: pgsql-sql
Hi, I am trying to import loads of data into postgres, I am having
trouble with a field which is a sequence. I know how to import data
using a delimiter with the copy command. But I need to know how I
increase the sequence on each line of import.
My Table has the following format.
field1 serial primary key,
field2 char (50),
field3 char(50),
field4 int,
field5 date
The file has the following format
field2|field3|field4|field5|
Each field is separated by a | (pipe), How do I insert the sequence
number before field2? I have like 30 files in this format and I need to
import them in one table with each line having unique number. Has
anyone come across similar situation? any help would be greatly
appreciated.
Please let me know if you need more info.
Thanks in advance.
--
Abdul Karim
The Hub Communications Company Ltd.
The Farmhouse
Syon Park
Middlesex TW8 8JF
T: 020 8560 9222 (ext 243)
F: 020 8560 9333
Email: mailto:Karim@thehub.co.uk
URL: http://www.thehub.co.uk
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: importing in sql
2000-07-10 22:03 importing in sql Abdul Karim <karim@thehub.co.uk>
@ 2000-07-12 17:55 ` Ross J. Reedstrom <reedstrm@rice.edu>
1 sibling, 0 replies; 3+ messages in thread
From: Ross J. Reedstrom @ 2000-07-12 17:55 UTC (permalink / raw)
To: Abdul Karim <karim@thehub.co.uk>; +Cc: pgsql-sql
On Mon, Jul 10, 2000 at 05:03:08PM -0500, Abdul Karim wrote:
> Hi, I am trying to import loads of data into postgres, I am having
> trouble with a field which is a sequence. I know how to import data
> using a delimiter with the copy command. But I need to know how I
> increase the sequence on each line of import.
>
> My Table has the following format.
>
> field1 serial primary key,
> field2 char (50),
> field3 char(50),
> field4 int,
> field5 date
>
> The file has the following format
>
> field2|field3|field4|field5|
>
> Each field is separated by a | (pipe), How do I insert the sequence
> number before field2? I have like 30 files in this format and I need to
> import them in one table with each line having unique number. Has
> anyone come across similar situation? any help would be greatly
> appreciated.
>
you don't mention your operating system, but on Linux I'd do this
with awk:
awk '{print NR"|"$0}' data_file > data_file.out
You mention 30 files: to get them sequential, I'd do:
cat files1 file2 [...] | awk '{print NR"|"$0}' | split -C 1m - dataout
Use some glob pattern for the 'cat' command that puts the files in the
order you want (even if it's just listing them all individually)
Awk will then number them, and split will generate file with at most 1
Meg of lines in them. If you're not worried about doing an all in one
bulk load, skip the split.
After this is all loaded, be sure to set the sequence associated with
the serial filed you loading into:
SELECT setval('table_field_seq',max(field)) from table;
Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: importing in sql
2000-07-10 22:03 importing in sql Abdul Karim <karim@thehub.co.uk>
@ 2000-07-12 18:19 ` jmr@computing.com
1 sibling, 0 replies; 3+ messages in thread
From: jmr@computing.com @ 2000-07-12 18:19 UTC (permalink / raw)
To: karim@thehub.co.uk; +Cc: pgsql-sql
karim> The file has the following format
karim>
karim> field2|field3|field4|field5|
karim>
karim> Each field is separated by a | (pipe), How do I insert the sequence
karim> number before field2? I have like 30 files in this format and I need to
karim> import them in one table with each line having unique number. Has
karim> anyone come across similar situation? any help would be greatly
karim> appreciated.
I'd create a new (temporary) table with only these 4 fields, and use
COPY to load it. Then, insert the data into the real table from the
temp one. Let postgres assign the serial numbers.
Something like this:
insert into <foo> (field2, field3, ...) select * from tmptable;
Then you can drop your temp table.
Jim Rowan
DCSI
jmr@computing.com
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2000-07-12 18:19 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2000-07-10 22:03 importing in sql Abdul Karim <karim@thehub.co.uk>
2000-07-12 17:55 ` Ross J. Reedstrom <reedstrm@rice.edu>
2000-07-12 18:19 ` jmr@computing.com
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox