From: Igor Lobanov <ilobanov@swsoft.com>
To: Richard Huxton <dev@archonet.com>
To: pgsql-performance@postgresql.org
Subject: Re: Querying distinct values from a large table
Date: Tue, 30 Jan 2007 15:33:11 +0600
Message-ID: <45BF10D7.3040203@swsoft.com> (raw)
In-Reply-To: <45BF0C13.3070003@archonet.com>
References: <45BF02DE.7080605@swsoft.com>
<45BF0C13.3070003@archonet.com>
Richard Huxton wrote:
>> I have rather large table with about 5 millions of rows and a dozen of
>> columns. Let's suppose that columns are named 'a', 'b', 'c' etc. I
>> need to query distinct pairs of ('a';'b') from this table.
>
> What version of PostgreSQL is it?
8.1.4
> How many distinct values are you getting back from your 5 million rows?
> If there are too many, an index isn't going to help.
No more than 10,000.
> Can you share the EXPLAIN ANALYSE output? You might want to try
> increasing work_mem for this one query to speed any sorting.
Real table and colum names are obfuscated because of NDA, sorry.
explain analyze select distinct a, b from tbl
EXPLAIN ANALYZE output is:
Unique (cost=500327.32..525646.88 rows=1848 width=6) (actual
time=52719.868..56126.356 rows=5390 loops=1)
-> Sort (cost=500327.32..508767.17 rows=3375941 width=6) (actual
time=52719.865..54919.989 rows=3378864 loops=1)
Sort Key: a, b
-> Seq Scan on tbl (cost=0.00..101216.41 rows=3375941
width=6) (actual time=16.643..20652.610 rows=3378864 loops=1)
Total runtime: 57307.394 ms
> How often is the table updated? Clustering might buy you some
> improvements (but not a huge amount I suspect).
It is updated once per 3-5 seconds.
And one more thing. I don't know if it helps, but column 'a' can have
value from a limited set: 0, 1 or 2. Column 'b' is also an integer
(foreign key, actually).
--
Igor Lobanov
Internal Development Engineer
SWsoft, Inc.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: ilobanov@swsoft.com, dev@archonet.com
Subject: Re: Querying distinct values from a large table
In-Reply-To: <45BF10D7.3040203@swsoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox