From: Joe Mirabal <jmmirabal@gmail.com>
To: Gregory Stark <stark@enterprisedb.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: josh@agliodbs.com, pgsql-performance@postgresql.org, "Craig James" <craig_james@emolecules.com>
Subject: Re: count * performance issue
Date: Mon, 10 Mar 2008 16:54:23 -0400
Message-ID: <12ba3bf00803101354w15b023e1o237f564a1b01105f@mail.gmail.com> (raw)
In-Reply-To: <87r6eia9tz.fsf@oxford.xeocode.com>
References: <f966c2ee0803052142l6a105678jfe2da92a0d9371@mail.gmail.com>
<47D00DB2.7040007@emolecules.com>
<18451.1204824562@sss.pgh.pa.us>
<200803071727.36146.josh@agliodbs.com>
<13345.1204940295@sss.pgh.pa.us>
<87r6eia9tz.fsf@oxford.xeocode.com>
Gregory,
I just joined this listserv and was happy to see this posting. I have a
400GB table that I have indexed (building the index took 27 hours) , Loading
the table with 10 threads took 9 hours. I run queries on the data nad get
immediate max and min as well as other aggrgate functions very quickly,
however a select count(*) of the table takes forever usually nearly an hour
or more.
Do you have any tuning recommendations. We in our warehouse use the
count(*) as our verification of counts by day/month's etc and in Netezza its
immediate. I tried by adding oids. BUT the situation I learned was that
adding the oids in the table adds a significasnt amount of space to the data
AND the index.
As you may gather from this we are relatively new on Postgres.
Any suggestions you can give me would be most helpful.
Cheers,
Joe
On Mon, Mar 10, 2008 at 11:16 AM, Gregory Stark <stark@enterprisedb.com>
wrote:
> "Tom Lane" <tgl@sss.pgh.pa.us> writes:
>
> > Well, scanning an index to get a count might be significantly faster
> > than scanning the main table, but it's hardly "instantaneous". It's
> > still going to take time proportional to the table size.
>
> Hm, Mark's comment about bitmap indexes makes that not entirely true. A
> bitmap
> index can do RLE compression which makes the relationship between the size
> of
> the table and the time taken to scan the index more complex. In the
> degenerate
> case where there are no concurrent updates (assuming you can determine
> that
> quickly) it might actually be constant time.
>
> > Unless they keep a central counter of the number of index entries;
> > which would have all the same serialization penalties we've talked
> > about before...
>
> Bitmap indexes do in fact have concurrency issues -- arguably they're just
> a
> baroque version of this central counter in this case.
>
> --
> Gregory Stark
> EnterpriseDB http://www.enterprisedb.com
> Ask me about EnterpriseDB's Slony Replication support!
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
--
Mirabili et Veritas
Joe Mirabal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jmmirabal@gmail.com, stark@enterprisedb.com, tgl@sss.pgh.pa.us, craig_james@emolecules.com
Subject: Re: count * performance issue
In-Reply-To: <12ba3bf00803101354w15b023e1o237f564a1b01105f@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox