Message-ID: <4EB1A419.1080901@consistentstate.com>
Date: Wed, 02 Nov 2011 14:12:09 -0600
From: Brian Fehrle <brianf@consistentstate.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US;
	rv:1.9.2.18) Gecko/20110616 Lightning/1.0b2 Thunderbird/3.1.11
MIME-Version: 1.0
To: pgsql-performance@postgresql.org
Subject: two table join just not fast enough.
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi all,

I've got a query that I need to squeeze as much speed out of as I can.

When I execute this query, the average time it takes is about 190 ms. I 
increased my work_mem from 1 MB to 50MB and it decreased the timing down 
to an average of 170 ms, but that's still not fast enough. This query is 
executed extremely frequently, so much of it should be easily cached.

Some settings
work_mem = 50MB
shared_buffers = 5GB

I've made sure that the two tables are recently analyzed (with 
default_stats to 100, 400, and 1500 even), and table bloat is low (150 
meg table has 7 megs bloat).

Here's the query:
SELECT yankee.my_id
FROM yankee
INNER JOIN hotel_zulu
     ON hotel_zulu.my_id = yankee.zulu_id
         AND hotel_zulu.type IN ('string1', 'string2', 'string3', 'string4')
WHERE yankee.your_id=402513;

And here is a query plan.

Hash Join  (cost=17516.470..26386.660 rows=27624 width=4) (actual time=309.194..395.135 rows=12384 loops=1)
     Hash Cond: (yankee.alpha = hotel_zulu.quebec)
   ->   Bitmap Heap Scan on yankee  (cost=1066.470..8605.770 rows=27624 width=20) (actual time=5.178..34.693 rows=26963 loops=1)
           Recheck Cond: (mike = 402513)
         ->   Bitmap Index Scan on hotel_alpha  (cost=0.000..1059.570 rows=27624 width=0) (actual time=4.770..4.770 rows=26967 loops=1)
                 Index Cond: (mike = 402513)
   ->   Hash  (cost=14465.080..14465.080 rows=114154 width=16) (actual time=303.717..303.717 rows=129908 loops=1)
           Buckets: 4096  Batches: 8  Memory Usage: 784kB
         ->   Bitmap Heap Scan on hotel_zulu  (cost=2461.770..14465.080 rows=114154 width=16) (actual time=25.642..185.253 rows=129908 loops=1)
                 Recheck Cond: ((two)::golf = ANY ('xray'::golf[]))
               ->   Bitmap Index Scan on kilo  (cost=0.000..2433.230 rows=114154 width=0) (actual time=23.887..23.887 rows=130292 loops=1)
                       Index Cond: ((two)::golf = ANY ('xray'::golf[]))


One thing I notice is the rows estimated is 27624 and the actual rows 
returned is 12384. Quite a bit different.

Table 2 (known here as hotel_zulu) is being joined on zulu_id to the 
first table, and then a where clause on the column 'type'. There are 
single column indexes on each of these columns, and any multi column 
index I put on these are just ignored by the planner.

Any thoughts on ways to tweak this?

- Brian F