Date: Fri, 12 Jan 2007 09:16:31 +0100
From: Tobias Brox <tobias@nordicbet.com>
To: pgsql-performance@postgresql.org
Subject: Planner statistics, correlations
Message-ID: <20070112081631.GA10636@oppetid.no>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Organization: Group Nordicbet
User-Agent: Mutt/1.5.9i

We have a table with a timestamp attribute (event_time) and a state flag
which usually changes value around the event_time (it goes to 4).  Now
we have more than two years of events in the database, and around 5k of
future events.

It is important to frequently pick out "overdue events", say:

  select * from events where state<>4 and event_time<now()

This query would usually yield between 0 and 100 rows - however, the
planner doesn't see the correlation betewen state and event_time - since
most of the events have event_time<now, the planner also assumes most of
the events with state<>4 has event_time<now, so the expected number of
rows is closer to 5k.  This matters, because I have a query with joins,
and I would really benefit from nested loops.

(I've tried replacing "now()" above with different timestamps from the
future and the past.  I'm using pg 8.2)

Any suggestions?