pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Herouth Maoz <herouth@oumail.openu.ac.il>
To: Michael Olivier <molivier@yahoo.com>
Cc: pgsql-sql@postgreSQL.org
Subject: Re: [SQL] optimizing 2-table join w/millions of rows
Date: Mon, 23 Nov 1998 15:09:09 +0200
Message-ID: <l03110700b27f0a10d780@[147.233.159.109]> (raw)
In-Reply-To: <19981123030514.15338.rocketmail@send106.yahoomail.com>

At 5:05 +0200 on 23/11/98, Michael Olivier wrote:


>
> > How about:
> >
> > SELECT U.acctname
> > FROM   usertest U, bgndtest B
> > WHERE  B.acctname = U.acctname
> >   AND  B.part_needed=3 AND B.loc_needed=5
> >   AND  U.part=2 AND U.loc=3;
>
> Yes, that looks equivalent. My problem is this is too slow an
> operation as I'm benchmarking it right now. And if I add less-than or
> greater-than comparisons, the performance goes _way_ down from there.
> How can I get the best performance out of this kind of operation?
>
> Is there any way to force postgres to hold certain tables in memory
> all the time? As I said, cost of memory isn't an issue, but
> performance is.

This is very strange. As a general rule, a simple join should always be
faster than a query with a subquery. Especially when that subquery is "in"
and not "exists".

In principle, if there is an index on the part, loc and acctname (separate
indices) in usertest, and on part_needed, loc_needed and acctname in
bgndtest, the most optimal query should be: restrict each of the tables by
the literal comparisons, and then match the acctname.

I don't know who wrote the optimiser, and I'm not sure he is on this list.
There are optimizer settings which can be changed, such as SET R_PLANS ON.

In any case, if you see in "real life" that a subquery actually works
better for you, then avoid the join. No reason to do both:

SELECT acctname
FROM bgndtest
WHERE part_needed=3 and loc_needed=5
  AND acctname IN (
      SELECT acctname
      FROM usertest
      WHERE part=2 AND loc=3
  );

Or better yet:

SELECT acctname
FROM bgndtest B
WHERE part_needed=3 and loc_needed=5
  AND EXISTS (
      SELECT *
      FROM usertest U
      WHERE part=2 AND loc=3
        AND U.acctname = B.acctname
  );

Of course, you may do it the other way around. It depends on which of your
tables is supposed to have more records, and which of the comparisons is
actually a "=" comparison, which is very important, because less-than and
greater-than don't use the indices.

What I mean by "the other way around" is:

SELECT acctname
FROM usertest U
WHERE part=2 and loc=3
  AND EXISTS (
      SELECT *
      FROM bgndtest B
      WHERE part_needed=3 and loc_needed=5
        AND U.acctname = B.acctname
  );

As for your question - there is no way to keep an entire table in memory,
unless you have a ram disk.

Herouth

--
Herouth Maoz, Internet developer.
Open University of Israel - Telem project
http://telem.openu.ac.il/~herutma





view thread (6+ messages)  latest in thread

Message-ID: <l03110700b27f0a10d780@[147.233.159.109]>
Permalink:  ../l03110700b27f0a10d780@%5B147.233.159.109%5D/
Also on:    postgresql.org/message-id/l03110700b27f0a10d780@[147.233.159.109]

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-sql@postgresql.org
  Cc: herouth@oumail.openu.ac.il, molivier@yahoo.com
  Subject: Re: [SQL] optimizing 2-table join w/millions of rows
  In-Reply-To: <l03110700b27f0a10d780@[147.233.159.109]>
  References: <19981123030514.15338.rocketmail@send106.yahoomail.com> <l03110700b27f0a10d780@[147.233.159.109]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox