From: Herouth Maoz <herouth@oumail.openu.ac.il>
To: Michael Olivier <molivier@yahoo.com>
Cc: pgsql-sql@postgreSQL.org
Subject: Re: [SQL] optimizing 2-table join w/millions of rows
Date: Mon, 23 Nov 1998 15:09:09 +0200
Message-ID: <l03110700b27f0a10d780@[147.233.159.109]> (raw)
In-Reply-To: <19981123030514.15338.rocketmail@send106.yahoomail.com>
At 5:05 +0200 on 23/11/98, Michael Olivier wrote:
>
> > How about:
> >
> > SELECT U.acctname
> > FROM usertest U, bgndtest B
> > WHERE B.acctname = U.acctname
> > AND B.part_needed=3 AND B.loc_needed=5
> > AND U.part=2 AND U.loc=3;
>
> Yes, that looks equivalent. My problem is this is too slow an
> operation as I'm benchmarking it right now. And if I add less-than or
> greater-than comparisons, the performance goes _way_ down from there.
> How can I get the best performance out of this kind of operation?
>
> Is there any way to force postgres to hold certain tables in memory
> all the time? As I said, cost of memory isn't an issue, but
> performance is.
This is very strange. As a general rule, a simple join should always be
faster than a query with a subquery. Especially when that subquery is "in"
and not "exists".
In principle, if there is an index on the part, loc and acctname (separate
indices) in usertest, and on part_needed, loc_needed and acctname in
bgndtest, the most optimal query should be: restrict each of the tables by
the literal comparisons, and then match the acctname.
I don't know who wrote the optimiser, and I'm not sure he is on this list.
There are optimizer settings which can be changed, such as SET R_PLANS ON.
In any case, if you see in "real life" that a subquery actually works
better for you, then avoid the join. No reason to do both:
SELECT acctname
FROM bgndtest
WHERE part_needed=3 and loc_needed=5
AND acctname IN (
SELECT acctname
FROM usertest
WHERE part=2 AND loc=3
);
Or better yet:
SELECT acctname
FROM bgndtest B
WHERE part_needed=3 and loc_needed=5
AND EXISTS (
SELECT *
FROM usertest U
WHERE part=2 AND loc=3
AND U.acctname = B.acctname
);
Of course, you may do it the other way around. It depends on which of your
tables is supposed to have more records, and which of the comparisons is
actually a "=" comparison, which is very important, because less-than and
greater-than don't use the indices.
What I mean by "the other way around" is:
SELECT acctname
FROM usertest U
WHERE part=2 and loc=3
AND EXISTS (
SELECT *
FROM bgndtest B
WHERE part_needed=3 and loc_needed=5
AND U.acctname = B.acctname
);
As for your question - there is no way to keep an entire table in memory,
unless you have a ram disk.
Herouth
--
Herouth Maoz, Internet developer.
Open University of Israel - Telem project
http://telem.openu.ac.il/~herutma
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-sql@postgresql.org
Cc: herouth@oumail.openu.ac.il, molivier@yahoo.com
Subject: Re: [SQL] optimizing 2-table join w/millions of rows
In-Reply-To: <l03110700b27f0a10d780@[147.233.159.109]>
References: <19981123030514.15338.rocketmail@send106.yahoomail.com> <l03110700b27f0a10d780@[147.233.159.109]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox