From: Stef <svb@ucs.co.za>
To: pgsql-performance@postgresql.org
Subject: Slow loads when indexes added.
Date: Tue, 15 Mar 2005 17:23:45 +0200
Message-ID: <20050315172345.7e444432@svb.ucs.co.za> (raw)
Hi all,
I suspect this problem/bug has been dealt with already, but I couldn't
find anything in the mail archives.
I'm using postgres 7.3, and I managed to recreate the problem using the attached
files.
The database structure is in slow_structure.sql
After creating the database, using this script, I ran run_before_load__fast.sql
Then I created a load file using create_loadfile.sh (It creates a file called load.sql)
I timed the loading of this file, and it loaded in 1 min 11.567 sec
Then I recreated the database from slow_structure.sql, ran run_before_load__slow.sql,
and then loaded the same load.sql and it took 3 min 51.293 sec which is about 6 times slower.
I tried the same thing on postgres 8.0.0 to see if it does the same thing, but there it
was consistently slow : 3 min 31.367 sec
The only way I got the load.sql to load fast on postgres 8.0.0, was by not creating
any of the foreign key constraints that point to the "main" table, and then enabling them
afterwards. This gave me the fastest time overall : 1 min 4.911 sec
My problem is that on the postgres 7.3.4 database I'm working with, a load process that
used to take 40 minutes, now takes 4 hours, because of 3 rows data being loaded into
a table (similar in setup to the "main" table in the example) before the indexes were created.
(This happens automatically when you dump and re-import the database (7.3.4))
Is there a way to get it to load fast again on the 7.3 database without dropping the foreign
key constraints (After running run_before_load_slow.sql) ?
And, if someone knows off-hand, what's happening here?
TIA
Kind Regards
Stefan
Attachments:create_loadfile.sh (text/x-sh, 865B) download
#!/bin/bash
for x in one two three four five six seven eight nine ten
do
for y in eleven twelve thirteen fourteen fifteen sixteen seventeen eighteen nineteen twenty
do
for z in `seq 1 100`
do
echo "insert into main (c_text,d_text) values ('${x}','${y}');" >> load.sql
echo "insert into a(id) values (currval('public.main_id_seq'::text));" >> load.sql
echo "insert into b(id) values (currval('public.main_id_seq'::text));" >> load.sql
echo "insert into e(id) values (currval('public.main_id_seq'::text));" >> load.sql
echo "insert into f(id) values (currval('public.main_id_seq'::text));" >> load.sql
echo "insert into g(id) values (currval('public.main_id_seq'::text));" >> load.sql
echo "insert into h(id) values (currval('public.main_id_seq'::text));" >> load.sql
done
done
done
run_before_load__fast.sql (application/octet-stream, 236B) download
run_before_load__slow.sql (application/octet-stream, 236B) download
slow_structure.sql (application/octet-stream, 7.5K) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: svb@ucs.co.za
Subject: Re: Slow loads when indexes added.
In-Reply-To: <20050315172345.7e444432@svb.ucs.co.za>
References: <20050315172345.7e444432@svb.ucs.co.za>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox