To: Thomas Kappler <tkappler@googlemail.com>
cc: pgsql-performance@postgresql.org
Subject: Re: Slow query with self-join, group by, 100m rows 
In-reply-to: 
 <CAOij6AxT+TQOx3_Q_1QFKNvDr5pr-z4zu8FX5nKjUj99bOH5_Q@mail.gmail.com>
References: 
 <CAOij6AxT+TQOx3_Q_1QFKNvDr5pr-z4zu8FX5nKjUj99bOH5_Q@mail.gmail.com>
Comments: In-reply-to Thomas Kappler <tkappler@googlemail.com>
	message dated "Tue, 20 Sep 2011 19:43:27 +0200"
Date: Wed, 21 Sep 2011 11:02:42 -0400
Message-ID: <23039.1316617362@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Thomas Kappler <tkappler@googlemail.com> writes:
> The query we want to do is (with example values):

> select t.cid, count(distinct t1.subject_id)
> from termindex t1, termindex t2
> where t1.cid=20642 and t1.indextype=636 and t2.indextype=636 and
> t2.subject_id=t1.subject_id
> group by t2.cid;

The EXPLAIN output you provided doesn't appear to match this query (in
particular, I don't see the indextype restrictions being checked
anyplace in the plan).

One quick-and-dirty thing that might help is to raise work_mem enough
so that (1) you get a hash aggregation not a sort/group one, and (2)
if there are still any sorts being done, they don't spill to disk.
That will probably be a higher number than would be prudent to install
as a global setting, but you could SET it locally in the current
session before issuing the expensive query.

			regards, tom lane