From: Tim Pizey <tim@paneris.co.uk>
To: Herouth Maoz <herouth@oumail.openu.ac.il>
Cc: pgsql-sql@postgreSQL.org
Cc: bloomsbury_development@messageboards.paneris.org
Subject: Re: [SQL] Newbie dbadmin out of his league
Date: Thu, 02 Sep 1999 11:36:17 +0100
Message-ID: <3.0.3.32.19990902113617.00732f1c@popmail.dircon.co.uk> (raw)
In-Reply-To: <l0313030ab3f2f95055be@[147.233.159.109]>
References: <7648.935788758@sss.pgh.pa.us>
<3.0.3.32.19990827211232.006f4c40@popmail.dircon.co.uk>
At 18:45 01/09/99 +0300, Herouth Maoz wrote:
>At 00:19 +0300 on 28/08/1999, Tom Lane wrote:
>
>
>> > Presumably I again have to take control of the key values,
>> > drop the index
>> > copy from tab delimited file containing hard ids
>> > create id sequence
>> > create index
>> > modify id definition
>>
>> Yup, that's about what you need to do. You can leave the "DEFAULT"
>> clause where it is, since it won't be invoked during a COPY that's
>> supplying non-default values for the ID column. (A good thing too,
>> since I don't think we support ALTER TABLE ADD DEFAULT...)
>
>Hmmm. If it were I, I would have tackled it in a slightly different way:
>
>COPY the data into a temporary table, that doesn't have the id numbers at
>all. Thus you don't have to have a counter on the client side, that knows
>the last id that's already on the table, etc, etc., and you also don't have
>to transfer several extra bytes per row through the postgres port.
>
Does the volume of data really matter, I was assuming that it was the
indexing that was taking the time. Many of my field sizes are generous and
padded with spaces.
>Then, when you have a temp table, you can add the values to the main table
>with an
>
>INSERT INTO main_table (field1, field2, field3)
>SELECT field1, field2, field3
>FROM temp_table;
>
>If you don't mention the field that carries the default in this INSERT
>statement, it will invoke the default. Dropping the index may still be a
>good idea. The temp table shouldn't have an index anyways.
>
>This would save you at least the three last steps in your "recipe".
>
Thanks a lot for this, I will try it next.
At the moment the job is going much faster, but is getting stuck on the
index creation.
I am turning off fsync (-o -F) during the copy, which copies all 2000000
records in one COPY. Should I keep it off during the index creation?
It looks increasingly as though this process is going to take an un
acceptable amount of time and resources on a live server.
What is the recommended way of creating a db on another machine and then
hot swapping the live and the new dbs?
yours
Tim Pizey
Happy to take part:/
/www.paneris.co.uk/
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-sql@postgresql.org
Cc: tim@paneris.co.uk, herouth@oumail.openu.ac.il, bloomsbury_development@messageboards.paneris.org
Subject: Re: [SQL] Newbie dbadmin out of his league
In-Reply-To: <3.0.3.32.19990902113617.00732f1c@popmail.dircon.co.uk>
References: <7648.935788758@sss.pgh.pa.us> <3.0.3.32.19990827211232.006f4c40@popmail.dircon.co.uk> <3.0.3.32.19990902113617.00732f1c@popmail.dircon.co.uk>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox