pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Re: [SQL] Joining bug????
16+ messages / 8 participants
[nested] [flat]

* Re: [SQL] Joining bug????
@ 1998-10-27 03:18  Gene Selkov Jr. <selkovjr@mcs.anl.gov>
  parent: pierre <pierre@desertmoon.com>
  1 sibling, 1 reply; 16+ messages in thread

From: Gene Selkov Jr. @ 1998-10-27 03:18 UTC (permalink / raw)
  To: pierre <pierre@desertmoon.com>; pgsql-sql


> I've been attempting to get my DB tuned to handle joins as best as 
> possible. However, no matter which way I try to perform the joins, if I 
> attempt to join more than two or three tables the joins becomes 
> unbearably long not matter how many indexes I create in however many ways.

Ditto. Never saw a quadruple join succeed, even with relatively small tables.

> My only solution was to create a hybrid table that contains the join of 
> all of the tables I'm searching on with multi-keyed indexes. This is a 
> VERY kludgy solution that makes changing the keys to my DB hard to change.

The solution I use may be the worst imaginable kludge, but it works great: I do the joins (as well as other set operations) on the client side. Perl hashes are very good for that.

--Gene



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Joining bug????
@ 1998-10-27 03:39  pierre <pierre@desertmoon.com>
  0 siblings, 2 replies; 16+ messages in thread

From: pierre @ 1998-10-27 03:39 UTC (permalink / raw)
  To: pgsql-sql

I've been attempting to get my DB tuned to handle joins as best as 
possible. However, no matter which way I try to perform the joins, if I 
attempt to join more than two or three tables the joins becomes 
unbearably long not matter how many indexes I create in however many ways.

My only solution was to create a hybrid table that contains the join of 
all of the tables I'm searching on with multi-keyed indexes. This is a 
VERY kludgy solution that makes changing the keys to my DB hard to change.

IS there a problem with postgresql in performing joins? Is there a fix? 
What gives?

-=pierre



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-27 16:24  David Hartwig <daveh@insightdist.com>
  parent: pierre <pierre@desertmoon.com>
  1 sibling, 1 reply; 16+ messages in thread

From: David Hartwig @ 1998-10-27 16:24 UTC (permalink / raw)
  To: pierre <pierre@desertmoon.com>; +Cc: pgsql-sql



pierre wrote:

> I've been attempting to get my DB tuned to handle joins as best as
> possible. However, no matter which way I try to perform the joins, if I
> attempt to join more than two or three tables the joins becomes
> unbearably long not matter how many indexes I create in however many ways.
>
> My only solution was to create a hybrid table that contains the join of
> all of the tables I'm searching on with multi-keyed indexes. This is a
> VERY kludgy solution that makes changing the keys to my DB hard to change.
>
> IS there a problem with postgresql in performing joins? Is there a fix?
> What gives?

There are many reasons for a query to take more time than expected.
PostgreSQL can do reasonably well joining many tables.   It is impossible for
anyone to even guess without more specifics.

Try submitting the specific query and the result of an EXPLAIN.
Example:
   EXPLAIN SELECT foo FROM bar;

Table sizes, indices, and time elapsed are also helpful.

BTW, in case you don't know, the "VACUUM ANALYZE" statement need to be run
occasionally to facilitate the query planner.





^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-27 17:00  pierre@desertmoon.com
  parent: David Hartwig <daveh@insightdist.com>
  0 siblings, 2 replies; 16+ messages in thread

From: pierre@desertmoon.com @ 1998-10-27 17:00 UTC (permalink / raw)
  To: daveh@insightdist.com; +Cc: pgsql-sql

> 
> 
> 
> pierre wrote:
> 
> > I've been attempting to get my DB tuned to handle joins as best as
> > possible. However, no matter which way I try to perform the joins, if I
> > attempt to join more than two or three tables the joins becomes
> > unbearably long not matter how many indexes I create in however many ways.
> >
> > My only solution was to create a hybrid table that contains the join of
> > all of the tables I'm searching on with multi-keyed indexes. This is a
> > VERY kludgy solution that makes changing the keys to my DB hard to change.
> >
> > IS there a problem with postgresql in performing joins? Is there a fix?
> > What gives?
> 
> There are many reasons for a query to take more time than expected.
> PostgreSQL can do reasonably well joining many tables.   It is impossible for
> anyone to even guess without more specifics.
> 
> Try submitting the specific query and the result of an EXPLAIN.
> Example:
>    EXPLAIN SELECT foo FROM bar;
> 
> Table sizes, indices, and time elapsed are also helpful.

This query takes about 30seconds...way too long for my needs...I've got
the query down to 2-5 seconds by using a hybrid table that is the join
between the pos and pcat tables. All _id fields are indexed, as is the
keyword field.

explain select p.prod_name from prod p, pos o, pcat c, pkey k
where p.prod_id = o.pos_prod_id and
o.pos_os_id = 2 and
o.pos_prod_id = c.pcat_prod_id and
c.pcat_cat_id = 6 and
c.pcat_prod_id = k.pkey_prod_id and
k.pkey_keyword = 'photoshop';
NOTICE:  QUERY PLAN:

Nested Loop  (cost=8.10 size=2 width=28)
  ->  Nested Loop  (cost=6.05 size=1 width=12)
        ->  Nested Loop  (cost=4.05 size=1 width=8)
              ->  Index Scan on o  (cost=2.05 size=1 width=4)
              ->  Index Scan on c  (cost=2.00 size=2 width=4)
        ->  Index Scan on k  (cost=2.00 size=2 width=4)
  ->  Index Scan on p  (cost=2.05 size=85442 width=16)


select count(*) from prod;
EXPLAIN
count
-----
85443
(1 row)


select count(*) from pos;
 count
------
132564
(1 row)


select count(*) from pcat;
 count
------
337251
(1 row)



select count(*) from pkey;
 count
------
492550
(1 row)

EOF
> 
> BTW, in case you don't know, the "VACUUM ANALYZE" statement need to be run
> occasionally to facilitate the query planner.
> 

All data is newley imported and indexed. So I shouldn't have to perform
this. This web site that is using this DB is still in debug mode.

Thanks for taking a look.

-=pierre



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-27 18:21  David Hartwig <daveh@insightdist.com>
  parent: pierre@desertmoon.com
  1 sibling, 1 reply; 16+ messages in thread

From: David Hartwig @ 1998-10-27 18:21 UTC (permalink / raw)
  To: pierre@desertmoon.com; +Cc: pgsql-sql

Nice query.   You mentioned having indexes on id columns etc.    Are the indices
single column, or multi-column?   You may want to consider making some of them
multi-column.   The EXPLAIN does indicate usage of your indices, but it may not be
optimal.   If the distribution of your data in the indices is such that the each
index scan must go through large chunks of data to qualify the other attributes in
the join, it will run slow.   By adding some well placed multi-part indices, the
index scans will be narrower in their scan sets.    Or not.

Also,  I could be wrong, but, I have found even batch loaded data needs a VACUUM
ANALYZE to gather distributions statistics.

pierre@desertmoon.com wrote:

> >
> >
> >
> > pierre wrote:
> >
> > > I've been attempting to get my DB tuned to handle joins as best as
> > > possible. However, no matter which way I try to perform the joins, if I
> > > attempt to join more than two or three tables the joins becomes
> > > unbearably long not matter how many indexes I create in however many ways.
> > >
> > > My only solution was to create a hybrid table that contains the join of
> > > all of the tables I'm searching on with multi-keyed indexes. This is a
> > > VERY kludgy solution that makes changing the keys to my DB hard to change.
> > >
> > > IS there a problem with postgresql in performing joins? Is there a fix?
> > > What gives?
> >
> > There are many reasons for a query to take more time than expected.
> > PostgreSQL can do reasonably well joining many tables.   It is impossible for
> > anyone to even guess without more specifics.
> >
> > Try submitting the specific query and the result of an EXPLAIN.
> > Example:
> >    EXPLAIN SELECT foo FROM bar;
> >
> > Table sizes, indices, and time elapsed are also helpful.
>
> This query takes about 30seconds...way too long for my needs...I've got
> the query down to 2-5 seconds by using a hybrid table that is the join
> between the pos and pcat tables. All _id fields are indexed, as is the
> keyword field.
>
> explain select p.prod_name from prod p, pos o, pcat c, pkey k
> where p.prod_id = o.pos_prod_id and
> o.pos_os_id = 2 and
> o.pos_prod_id = c.pcat_prod_id and
> c.pcat_cat_id = 6 and
> c.pcat_prod_id = k.pkey_prod_id and
> k.pkey_keyword = 'photoshop';
> NOTICE:  QUERY PLAN:
>
> Nested Loop  (cost=8.10 size=2 width=28)
>   ->  Nested Loop  (cost=6.05 size=1 width=12)
>         ->  Nested Loop  (cost=4.05 size=1 width=8)
>               ->  Index Scan on o  (cost=2.05 size=1 width=4)
>               ->  Index Scan on c  (cost=2.00 size=2 width=4)
>         ->  Index Scan on k  (cost=2.00 size=2 width=4)
>   ->  Index Scan on p  (cost=2.05 size=85442 width=16)
>
> select count(*) from prod;
> EXPLAIN
> count
> -----
> 85443
> (1 row)
>
> select count(*) from pos;
>  count
> ------
> 132564
> (1 row)
>
> select count(*) from pcat;
>  count
> ------
> 337251
> (1 row)
>
> select count(*) from pkey;
>  count
> ------
> 492550
> (1 row)
>
> EOF
> >
> > BTW, in case you don't know, the "VACUUM ANALYZE" statement need to be run
> > occasionally to facilitate the query planner.
> >
>
> All data is newley imported and indexed. So I shouldn't have to perform
> this. This web site that is using this DB is still in debug mode.
>
> Thanks for taking a look.
>
> -=pierre




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-27 18:40  David Hartwig <daveh@insightdist.com>
  parent: pierre@desertmoon.com
  1 sibling, 0 replies; 16+ messages in thread

From: David Hartwig @ 1998-10-27 18:40 UTC (permalink / raw)
  To: pierre@desertmoon.com; +Cc: pgsql-sql

I would try the following indices:

prod - (prod_id)
pos -   (pos_os_id, pos_prod_id)
pcat: - (pcat_prod_id, pcat_cat_id)
pkey - ( pkey_keyword, pkey_prod_id)

Various permutations of order may also effect the outcome.

pierre@desertmoon.com wrote:

> >
> >
> >
> > pierre wrote:
> >
> > > I've been attempting to get my DB tuned to handle joins as best as
> > > possible. However, no matter which way I try to perform the joins, if I
> > > attempt to join more than two or three tables the joins becomes
> > > unbearably long not matter how many indexes I create in however many ways.
> > >
> > > My only solution was to create a hybrid table that contains the join of
> > > all of the tables I'm searching on with multi-keyed indexes. This is a
> > > VERY kludgy solution that makes changing the keys to my DB hard to change.
> > >
> > > IS there a problem with postgresql in performing joins? Is there a fix?
> > > What gives?
> >
> > There are many reasons for a query to take more time than expected.
> > PostgreSQL can do reasonably well joining many tables.   It is impossible for
> > anyone to even guess without more specifics.
> >
> > Try submitting the specific query and the result of an EXPLAIN.
> > Example:
> >    EXPLAIN SELECT foo FROM bar;
> >
> > Table sizes, indices, and time elapsed are also helpful.
>
> This query takes about 30seconds...way too long for my needs...I've got
> the query down to 2-5 seconds by using a hybrid table that is the join
> between the pos and pcat tables. All _id fields are indexed, as is the
> keyword field.
>
> explain select p.prod_name from prod p, pos o, pcat c, pkey k
> where p.prod_id = o.pos_prod_id and
> o.pos_os_id = 2 and
> o.pos_prod_id = c.pcat_prod_id and
> c.pcat_cat_id = 6 and
> c.pcat_prod_id = k.pkey_prod_id and
> k.pkey_keyword = 'photoshop';
> NOTICE:  QUERY PLAN:
>
> Nested Loop  (cost=8.10 size=2 width=28)
>   ->  Nested Loop  (cost=6.05 size=1 width=12)
>         ->  Nested Loop  (cost=4.05 size=1 width=8)
>               ->  Index Scan on o  (cost=2.05 size=1 width=4)
>               ->  Index Scan on c  (cost=2.00 size=2 width=4)
>         ->  Index Scan on k  (cost=2.00 size=2 width=4)
>   ->  Index Scan on p  (cost=2.05 size=85442 width=16)
>
> select count(*) from prod;
> EXPLAIN
> count
> -----
> 85443
> (1 row)
>
> select count(*) from pos;
>  count
> ------
> 132564
> (1 row)
>
> select count(*) from pcat;
>  count
> ------
> 337251
> (1 row)
>
> select count(*) from pkey;
>  count
> ------
> 492550
> (1 row)
>
> EOF
> >
> > BTW, in case you don't know, the "VACUUM ANALYZE" statement need to be run
> > occasionally to facilitate the query planner.
> >
>
> All data is newley imported and indexed. So I shouldn't have to perform
> this. This web site that is using this DB is still in debug mode.
>
> Thanks for taking a look.
>
> -=pierre




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-27 19:04  pierre@desertmoon.com
  parent: David Hartwig <daveh@insightdist.com>
  0 siblings, 0 replies; 16+ messages in thread

From: pierre@desertmoon.com @ 1998-10-27 19:04 UTC (permalink / raw)
  To: daveh@insightdist.com; +Cc: pgsql-sql

> 
> Nice query.   You mentioned having indexes on id columns etc.    Are the indices
> single column, or multi-column?   You may want to consider making some of them
> multi-column.   The EXPLAIN does indicate usage of your indices, but it may not be
> optimal.   If the distribution of your data in the indices is such that the each
> index scan must go through large chunks of data to qualify the other attributes in
> the join, it will run slow.   By adding some well placed multi-part indices, the
> index scans will be narrower in their scan sets.    Or not.
> 

My apologies I should have been a bit more specific. The indices are multi 
column, _id first prod_id second. Also  A search for just the keyword
photoshop returns just 35 or so values. I was under the impression that postgresql
used the bottom up method of optimization. I.e. the most narrow criteria
should be declared last, why doesn't the explain show the use of the keyword 
first?

> Also,  I could be wrong, but, I have found even batch loaded data needs a VACUUM
> ANALYZE to gather distributions statistics.

I'll try just about anything at this point....oh...and most of these tables 
have been clustered by their major indexes.

-=pierre




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* RE: [SQL] Joining bug????
@ 1998-10-27 23:21  Jackson, DeJuan <djackson@cpsgroup.com>
  0 siblings, 0 replies; 16+ messages in thread

From: Jackson, DeJuan @ 1998-10-27 23:21 UTC (permalink / raw)
  To: Howie <caffeine@toodarkpark.org>; Gene Selkov Jr. <selkovjr@mcs.anl.gov>; +Cc: pierre <pierre@desertmoon.com>; pgsql-sql

> On Mon, 26 Oct 1998, Gene Selkov Jr. wrote:
> 
> > 
> > > I've been attempting to get my DB tuned to handle joins as best as
> 
> > > possible. However, no matter which way I try to perform the joins,
> if I 
> > > attempt to join more than two or three tables the joins becomes 
> > > unbearably long not matter how many indexes I create in however
> many ways.
> > 
> > Ditto. Never saw a quadruple join succeed, even with relatively
> small tables.
> 
> Ive been playing with the idea of moving a client from MySQL to
> Postgres (
> views, triggers, and subselects would _really_ make my life easier ).
> 
> ---[ CUT ]---
> SELECT
>         account.accnum,
>         account.accname,
>         ((customers.firstname || ' ') || customers.lastname) as
> Customer,
>         acctype.descr,
>         account.balance,
>         account.status,
>         billdate.next
> FROM
>         account,customers,acctype,billdate
> WHERE
>         account.custnum=customers.custnum AND
>         account.acctype=acctype.accid AND
>         account.accnum=billdate.accnum AND
>         account.accnum<20
> ORDER BY
>         account.accnum
> ---[ CUT ]---
> 
> works perfectly, takes about 2 seconds to complete.  granted i have
> ( sometimes unique ) indexes on all the join columns.
> 
> account has 7k rows, customers has 5k rows, acctype has 12 rows,
> billdate has 7k rows.
> 
> a somewhat modified, real-world query ends up crashing postgres,
> though:
> 
> ---[ CUT ]---
> select 
>         account.accnum, 
>         account.accname, 
>         account.totalfee, 
>         billdate.next, 
>         ((customers.firstname || ' ') || customers.lastname) as
> Customer,
>         customers.company, 
>         customers.street,
>         acctype.yearly
> FROM 
>         account,customers,acctype,billdate
> where
>         account.totalfee>0.00 AND
>         billtype=1 OR (billcc1stmo=1 AND
> account.created=billdate.last)
> AND
try:         (billdate.billtype=1 OR (billdate.billcc1stmo=1 AND
account.created=billdate.last)) AND

>         balance<>0.00 AND
>         billdate.next>='1998-01-01' AND 
>         billdate.next<='1998-01-05' AND 
>         account.status<>'C' AND
>         billdate.accnum=account.accnum AND
>         account.custnum=customers.custnum AND
>         account.acctype=acctype.accid
> ---[ CUT ]---
> 
> 
> > > My only solution was to create a hybrid table that contains the
> join of 
> > > all of the tables I'm searching on with multi-keyed indexes. This
> is a 
> > > VERY kludgy solution that makes changing the keys to my DB hard to
> change.
> > 
> > The solution I use may be the worst imaginable kludge, but it works
> > great: I do the joins (as well as other set operations) on the
> client
> > side. Perl hashes are very good for that. 
> 
> try making a view.  from what others have said, views save the query
> plan
> and have usually, at least for me, been _alot_ faster than normal
> queries.
> 
> ---
> Howie <caffeine@toodarkpark.org>   URL: http://www.toodarkpark.org    
>         "Oh my god, they killed init!  YOU BASTARDS!"
> 



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-27 23:48  Howie <caffeine@toodarkpark.org>
  parent: Gene Selkov Jr. <selkovjr@mcs.anl.gov>
  0 siblings, 1 reply; 16+ messages in thread

From: Howie @ 1998-10-27 23:48 UTC (permalink / raw)
  To: Gene Selkov Jr. <selkovjr@mcs.anl.gov>; +Cc: pierre <pierre@desertmoon.com>; pgsql-sql

On Mon, 26 Oct 1998, Gene Selkov Jr. wrote:

> 
> > I've been attempting to get my DB tuned to handle joins as best as 
> > possible. However, no matter which way I try to perform the joins, if I 
> > attempt to join more than two or three tables the joins becomes 
> > unbearably long not matter how many indexes I create in however many ways.
> 
> Ditto. Never saw a quadruple join succeed, even with relatively small tables.

Ive been playing with the idea of moving a client from MySQL to Postgres (
views, triggers, and subselects would _really_ make my life easier ).

---[ CUT ]---
SELECT
        account.accnum,
        account.accname,
        ((customers.firstname || ' ') || customers.lastname) as Customer,
        acctype.descr,
        account.balance,
        account.status,
        billdate.next
FROM
        account,customers,acctype,billdate
WHERE
        account.custnum=customers.custnum AND
        account.acctype=acctype.accid AND
        account.accnum=billdate.accnum AND
        account.accnum<20
ORDER BY
        account.accnum
---[ CUT ]---

works perfectly, takes about 2 seconds to complete.  granted i have
( sometimes unique ) indexes on all the join columns.

account has 7k rows, customers has 5k rows, acctype has 12 rows,
billdate has 7k rows.

a somewhat modified, real-world query ends up crashing postgres, though:

---[ CUT ]---
select 
        account.accnum, 
        account.accname, 
        account.totalfee, 
        billdate.next, 
        ((customers.firstname || ' ') || customers.lastname) as Customer,
        customers.company, 
        customers.street,
        acctype.yearly
FROM 
        account,customers,acctype,billdate
where
        account.totalfee>0.00 AND
        billtype=1 OR (billcc1stmo=1 AND account.created=billdate.last)
AND
        balance<>0.00 AND
        billdate.next>='1998-01-01' AND 
        billdate.next<='1998-01-05' AND 
        account.status<>'C' AND
        billdate.accnum=account.accnum AND
        account.custnum=customers.custnum AND
        account.acctype=acctype.accid
---[ CUT ]---


> > My only solution was to create a hybrid table that contains the join of 
> > all of the tables I'm searching on with multi-keyed indexes. This is a 
> > VERY kludgy solution that makes changing the keys to my DB hard to change.
> 
> The solution I use may be the worst imaginable kludge, but it works
> great: I do the joins (as well as other set operations) on the client
> side. Perl hashes are very good for that. 

try making a view.  from what others have said, views save the query plan
and have usually, at least for me, been _alot_ faster than normal queries.

---
Howie <caffeine@toodarkpark.org>   URL: http://www.toodarkpark.org    
        "Oh my god, they killed init!  YOU BASTARDS!"




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-28 01:38  pierre <pierre@desertmoon.com>
  0 siblings, 1 reply; 16+ messages in thread

From: pierre @ 1998-10-28 01:38 UTC (permalink / raw)
  To: David Hartwig <daveh@insightdist.com>; +Cc: pgsql-sql

>Nice query.   You mentioned having indexes on id columns etc.    Are the 
>indices
>single column, or multi-column?   You may want to consider making some of 
>them
>multi-column.   The EXPLAIN does indicate usage of your indices, but it 
>may not be
>optimal.   If the distribution of your data in the indices is such that 
>the each
>index scan must go through large chunks of data to qualify the other 
>attributes in
>the join, it will run slow.   By adding some well placed multi-part 
>indices, the
>index scans will be narrower in their scan sets.    Or not.
>
>Also,  I could be wrong, but, I have found even batch loaded data needs a 
>VACUUM
>ANALYZE to gather distributions statistics.
>
>pierre@desertmoon.com wrote:
>
>> >

Thanks David. The vacuum analyze did the trick. I made the invalid 
assumption that the statistics would be up to date just after a copy and 
index creation. They were not. As soon as I ran the vacuum across all my 
tables the explains changed and I got a MAJOR speed increase and the most 
complicated query takes no more than 4-8 seconds. This is perfect.

Perhaps this (bug??) should be documented? I've seen documentation 
relating to the vacuum analyze, but I always made that invalid 
assumption. *sigh* Ah well you live and you learn. :)

-=pierre



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-28 02:19  Bruce Momjian <maillist@candle.pha.pa.us>
  parent: pierre <pierre@desertmoon.com>
  0 siblings, 0 replies; 16+ messages in thread

From: Bruce Momjian @ 1998-10-28 02:19 UTC (permalink / raw)
  To: pierre@desertmoon.com; +Cc: daveh@insightdist.com; pgsql-sql

> Thanks David. The vacuum analyze did the trick. I made the invalid 
> assumption that the statistics would be up to date just after a copy and 
> index creation. They were not. As soon as I ran the vacuum across all my 
> tables the explains changed and I got a MAJOR speed increase and the most 
> complicated query takes no more than 4-8 seconds. This is perfect.
> 
> Perhaps this (bug??) should be documented? I've seen documentation 
> relating to the vacuum analyze, but I always made that invalid 
> assumption. *sigh* Ah well you live and you learn. :)

I have beefed up the FAQ:

   For column-specific optimization statistics, use vacuum analyze.
   Vacuum analyze is important for complex multi-join queries, so the
   optimizer can estimate the number of rows returned from each table,
   and choose the proper join order. The backend does not keep track of
   column statistics on its own, and vacuum analyze must be run to
   collect them periodically.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-28 11:43  Jan Wieck <jwieck@debis.com>
  parent: Howie <caffeine@toodarkpark.org>
  0 siblings, 0 replies; 16+ messages in thread

From: jwieck@debis.com @ 1998-10-28 11:43 UTC (permalink / raw)
  To: caffeine@toodarkpark.org; +Cc: selkovjr@mcs.anl.gov; pierre@desertmoon.com; pgsql-sql

> try making a view.  from what others have said, views save the query plan
> and have usually, at least for me, been _alot_ faster than normal queries.

    Views don't save query plans. They are parsetree templates in
    query rewrite rules.

>
> ---
> Howie <caffeine@toodarkpark.org>   URL: http://www.toodarkpark.org
>         "Oh my god, they killed init!  YOU BASTARDS!"


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#======================================== jwieck@debis.com (Jan Wieck) #





^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* RE: [SQL] Joining bug????
@ 1998-10-28 17:11  Jackson, DeJuan <djackson@cpsgroup.com>
  0 siblings, 1 reply; 16+ messages in thread

From: Jackson, DeJuan @ 1998-10-28 17:11 UTC (permalink / raw)
  To: jwieck@debis.com; caffeine@toodarkpark.org; +Cc: selkovjr@mcs.anl.gov; pierre@desertmoon.com; pgsql-sql

> > try making a view.  from what others have said, views save the query
> plan
> > and have usually, at least for me, been _alot_ faster than normal
> queries.
> 
>     Views don't save query plans. They are parsetree templates in
>     query rewrite rules.
> 
So, could you explain what counts for the speed increase.  Is the parse
stage really that costly?
	-DEJ



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* RE: [SQL] Joining bug????
@ 1998-10-28 17:49  Jackson, DeJuan <djackson@cpsgroup.com>
  0 siblings, 1 reply; 16+ messages in thread

From: Jackson, DeJuan @ 1998-10-28 17:49 UTC (permalink / raw)
  To: jwieck@debis.com; +Cc: caffeine@toodarkpark.org; selkovjr@mcs.anl.gov; pierre@desertmoon.com; pgsql-sql


>     BTW:  You  might want to take a look at the new documentation
>     of the rule system to understand how views work in detail.
> 
I think I'll do that.  I'd love to be able to help you for the rule
rewrite for v6.5 .

>     It could only be the parse stage. After the  rule  system  is
>     through  with the query, the parsetree given to the optimizer
>     is exactly the same  as  if  you  typed  in  the  query  that
>     accesses the real tables.  Having a view
> 
>         CREATE VIEW v1 AS SELECT t1.a, t2.b FROM t1, t2
>             WHERE t1.a = t2.a;
> 
>     and doing a
> 
>         SELECT * FROM v1 WHERE a = 'something';
> 
>     is 100% the same as
> 
>         SELECT t1.a, t2.b FROM t1, t2
>             WHERE t1.a = 'something'
>               AND t1.a = t2.a;
> 
>     The  optimizer  will  get exactly the same parsetree and will
>     generate the same plan then. So the timing difference  cannot
>     be in the optimizer or executor.
> 
>     But  it's  interesting.  I'll  do some tests on it and try to
>     find out whether parsing of complicated WHERE clauses is that
>     costly  that  it  can be the reason. Seems the rule system is
>     quicker than the parser :-)
> 
What kind of speed increases are people seeing in their queries?
I've actually never created a view in PostgreSQL (say he with head hung
low).


> Jan
	-DEJ




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-28 17:56  Jan Wieck <jwieck@debis.com>
  parent: Jackson, DeJuan <djackson@cpsgroup.com>
  0 siblings, 0 replies; 16+ messages in thread

From: jwieck@debis.com @ 1998-10-28 17:56 UTC (permalink / raw)
  To: djackson@cpsgroup.com; +Cc: jwieck@debis.com; caffeine@toodarkpark.org; selkovjr@mcs.anl.gov; pierre@desertmoon.com; pgsql-sql

>
> > > try making a view.  from what others have said, views save the query
> > plan
> > > and have usually, at least for me, been _alot_ faster than normal
> > queries.
> >
> >     Views don't save query plans. They are parsetree templates in
> >     query rewrite rules.
> >
> So, could you explain what counts for the speed increase.  Is the parse
> stage really that costly?

    BTW:  You  might want to take a look at the new documentation
    of the rule system to understand how views work in detail.

    It could only be the parse stage. After the  rule  system  is
    through  with the query, the parsetree given to the optimizer
    is exactly the same  as  if  you  typed  in  the  query  that
    accesses the real tables.  Having a view

        CREATE VIEW v1 AS SELECT t1.a, t2.b FROM t1, t2
            WHERE t1.a = t2.a;

    and doing a

        SELECT * FROM v1 WHERE a = 'something';

    is 100% the same as

        SELECT t1.a, t2.b FROM t1, t2
            WHERE t1.a = 'something'
              AND t1.a = t2.a;

    The  optimizer  will  get exactly the same parsetree and will
    generate the same plan then. So the timing difference  cannot
    be in the optimizer or executor.

    But  it's  interesting.  I'll  do some tests on it and try to
    find out whether parsing of complicated WHERE clauses is that
    costly  that  it  can be the reason. Seems the rule system is
    quicker than the parser :-)


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#======================================== jwieck@debis.com (Jan Wieck) #





^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: [SQL] Joining bug????
@ 1998-10-28 18:38  Jan Wieck <jwieck@debis.com>
  parent: Jackson, DeJuan <djackson@cpsgroup.com>
  0 siblings, 0 replies; 16+ messages in thread

From: jwieck@debis.com @ 1998-10-28 18:38 UTC (permalink / raw)
  To: djackson@cpsgroup.com; +Cc: jwieck@debis.com; caffeine@toodarkpark.org; selkovjr@mcs.anl.gov; pierre@desertmoon.com; pgsql-sql

> >     But  it's  interesting.  I'll  do some tests on it and try to
> >     find out whether parsing of complicated WHERE clauses is that
> >     costly  that  it  can be the reason. Seems the rule system is
> >     quicker than the parser :-)
> >
> What kind of speed increases are people seeing in their queries?
> I've actually never created a view in PostgreSQL (say he with head hung
> low).

    Used  a  view that is a join of 5 tables. There is absolutely
    no difference between using the view or the equivalent select
    from the tables directly in what -s prints as times.

    There   must  be  something  else  that  causes  the  speedup
    reported.  Maybe another ordering in the qualifications  when
    using  the  view.   The  view  qual's  are  added last to the
    original queries qual. Putting them the other  way  round  in
    the query might produce another join order.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#======================================== jwieck@debis.com (Jan Wieck) #





^ permalink  raw  reply  [nested|flat] 16+ messages in thread


end of thread, other threads:[~1998-10-28 18:38 UTC | newest]

Thread overview: 16+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
1998-10-27 03:39 Joining bug???? pierre <pierre@desertmoon.com>
1998-10-27 03:18 ` Gene Selkov Jr. <selkovjr@mcs.anl.gov>
1998-10-27 23:48   ` Howie <caffeine@toodarkpark.org>
1998-10-28 11:43     ` Jan Wieck <jwieck@debis.com>
1998-10-27 16:24 ` David Hartwig <daveh@insightdist.com>
1998-10-27 17:00   ` pierre@desertmoon.com
1998-10-27 18:21     ` David Hartwig <daveh@insightdist.com>
1998-10-27 19:04       ` pierre@desertmoon.com
1998-10-27 18:40     ` David Hartwig <daveh@insightdist.com>
1998-10-27 23:21 RE: [SQL] Joining bug???? Jackson, DeJuan <djackson@cpsgroup.com>
1998-10-28 01:38 Re: [SQL] Joining bug???? pierre <pierre@desertmoon.com>
1998-10-28 02:19 ` Bruce Momjian <maillist@candle.pha.pa.us>
1998-10-28 17:11 RE: [SQL] Joining bug???? Jackson, DeJuan <djackson@cpsgroup.com>
1998-10-28 17:56 ` Jan Wieck <jwieck@debis.com>
1998-10-28 17:49 RE: [SQL] Joining bug???? Jackson, DeJuan <djackson@cpsgroup.com>
1998-10-28 18:38 ` Jan Wieck <jwieck@debis.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox