pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
alter table TBL add constraint TBL_FK foreign key ... very slow
4+ messages / 4 participants
[nested] [flat]

* alter table TBL add constraint TBL_FK foreign key ... very slow
@ 2003-01-01 03:32  Minghann Ho <Minghann.Ho@mcs.vuw.ac.nz>
  0 siblings, 2 replies; 4+ messages in thread

From: Minghann Ho @ 2003-01-01 03:32 UTC (permalink / raw)
  To: pgsql-performance

Hi all,

I've experienced very slow performance to add foreign key constraints using 
ALTER TABLE ADD CONSTRAINT FOREIGN KEY ...

After using COPY ... FROM to load the base tables, I started to build the 
referential integrity between tables.
I have 3 tables: T1 (6 million records), T2 (1.5 million records) and T3 (0.8 
million records).
One of the RI - foreign key (T1 -> T2) constraint took about 70 hrs to build.
The other RI - foreign key (T1 -> T3) constraint took about 200 hrs and yet 
completed!! (compound foreign key)

I tried to use small subset of the tables of T2 and T3 to do the testing.
An estimation show that it need about 960 hrs to build the RI - foreign key 
constraints on table T1 -> T3 !!!

I've read in the archives that some people suffered slow performance of this 
problem in Aug 2000, but there was no further information about the solution.

Please anyone who has experience in this issues can give me some hint. 

Thanks

Hans



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* alter table TBL add constraint TBL_FK foreign key ... very slow
@ 2003-01-01 03:38  Minghann Ho <mho@mcs.vuw.ac.nz>
  0 siblings, 0 replies; 4+ messages in thread

From: Minghann Ho @ 2003-01-01 03:38 UTC (permalink / raw)
  To: pgsql-performance; pgsql-performance-owner@postgresql.org; pgsql-admin@postgresql.org

Hi all,

I've experienced very slow performance to add foreign key constraints using 
ALTER TABLE ADD CONSTRAINT FOREIGN KEY ...

After using COPY ... FROM to load the base tables, I started to build the 
referential integrity between tables.
I have 3 tables: T1 (6 million records), T2 (1.5 million records) and T3 (0.8 
million records).
One of the RI - foreign key (T1 -> T2) constraint took about 70 hrs to build.
The other RI - foreign key (T1 -> T3) constraint took about 200 hrs and yet 
completed!! (compound foreign key)

I tried to use small subset of the tables of T2 and T3 to do the testing.
An estimation show that it need about 960 hrs to build the RI - foreign key 
constraints on table T1 -> T3 !!!

I've read in the archives that some people suffered slow performance of this 
problem in Aug 2000, but there was no further information about the solution.

Please anyone who has experience in this issues can give me some hint. 

Thanks

Hans



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: alter table TBL add constraint TBL_FK foreign key ...
@ 2003-01-01 03:38  Stephan Szabo <sszabo@megazone23.bigpanda.com>
  parent: Minghann Ho <Minghann.Ho@mcs.vuw.ac.nz>
  1 sibling, 0 replies; 4+ messages in thread

From: Stephan Szabo @ 2003-01-01 03:38 UTC (permalink / raw)
  To: Minghann Ho <Minghann.Ho@mcs.vuw.ac.nz>; +Cc: pgsql-performance

On Wed, 1 Jan 2003, Minghann Ho wrote:

> I've experienced very slow performance to add foreign key constraints using
> ALTER TABLE ADD CONSTRAINT FOREIGN KEY ...
>
> After using COPY ... FROM to load the base tables, I started to build the
> referential integrity between tables.
> I have 3 tables: T1 (6 million records), T2 (1.5 million records) and T3 (0.8
> million records).
> One of the RI - foreign key (T1 -> T2) constraint took about 70 hrs to build.
> The other RI - foreign key (T1 -> T3) constraint took about 200 hrs and yet
> completed!! (compound foreign key)
>
> I tried to use small subset of the tables of T2 and T3 to do the testing.
> An estimation show that it need about 960 hrs to build the RI - foreign key
> constraints on table T1 -> T3 !!!

It's running the constraint check for each row in the foreign key table.
Rather than using a call to the function and a select for each row, it
could probably be done in a single select with a not exists subselect, but
that hasn't been done yet.  There's also been talk about allowing some
mechanism to allow the avoidance of the create time check, but I don't
think any concensus was reached.




^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: alter table TBL add constraint TBL_FK foreign key
@ 2003-01-01 08:14  Ron Johnson <ron.l.johnson@cox.net>
  parent: Minghann Ho <Minghann.Ho@mcs.vuw.ac.nz>
  1 sibling, 0 replies; 4+ messages in thread

From: Ron Johnson @ 2003-01-01 08:14 UTC (permalink / raw)
  To: pgsql-performance

On Tue, 2002-12-31 at 21:32, Minghann Ho wrote:
> Hi all,
> 
> I've experienced very slow performance to add foreign key constraints using 
> ALTER TABLE ADD CONSTRAINT FOREIGN KEY ...
> 
> After using COPY ... FROM to load the base tables, I started to build the 
> referential integrity between tables.
> I have 3 tables: T1 (6 million records), T2 (1.5 million records) and T3 (0.8 
> million records).
> One of the RI - foreign key (T1 -> T2) constraint took about 70 hrs to build.
> The other RI - foreign key (T1 -> T3) constraint took about 200 hrs and yet 
> completed!! (compound foreign key)
> 
> I tried to use small subset of the tables of T2 and T3 to do the testing.
> An estimation show that it need about 960 hrs to build the RI - foreign key 
> constraints on table T1 -> T3 !!!
> 
> I've read in the archives that some people suffered slow performance of this 
> problem in Aug 2000, but there was no further information about the solution.
> 
> Please anyone who has experience in this issues can give me some hint. 

Silly question: Are T2 & T3 compound-key indexed on the relevant foreign
key fields (in the exact order that they are mentioned in the ADD
CONSTRAINT command)?  Otherwise, for each record in T1, it is scanning
T2 1.5M times (9E12 record reads!), with a similar formula for T1->T3.

-- 
+------------------------------------------------------------+
| Ron Johnson, Jr.     mailto:ron.l.johnson@cox.net          |
| Jefferson, LA  USA   http://members.cox.net/ron.l.johnson  |
|                                                            |
| "Basically, I got on the plane with a bomb. Basically, I   |
|  tried to ignite it. Basically, yeah, I intended to damage |
|  the plane."                                               |
|    RICHARD REID, who tried to blow up American Airlines    |
|                  Flight 63                                 |
+------------------------------------------------------------+




^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2003-01-01 08:14 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-01-01 03:32 alter table TBL add constraint TBL_FK foreign key ... very slow Minghann Ho <Minghann.Ho@mcs.vuw.ac.nz>
2003-01-01 03:38 ` Re: alter table TBL add constraint TBL_FK foreign key ... Stephan Szabo <sszabo@megazone23.bigpanda.com>
2003-01-01 08:14 ` Re: alter table TBL add constraint TBL_FK foreign key Ron Johnson <ron.l.johnson@cox.net>
2003-01-01 03:38 alter table TBL add constraint TBL_FK foreign key ... very slow Minghann Ho <mho@mcs.vuw.ac.nz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox