pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Steve Lane <slane@moyergroup.com>
To: pgsql-admin@postgresql.org
Subject: Re: Index (re)-creation speed
Date: Fri, 06 Feb 2004 01:03:48 -0600
Message-ID: <BC489874.9895%slane@moyergroup.com> (raw)
In-Reply-To: <24649.1076049015@sss.pgh.pa.us>

On 2/6/04 12:30 AM, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> Steve Lane <slane@moyergroup.com> writes:
>> When I drop and rebuild the indexes, they take oddly varying amounts of time
>> to rebuild. I rebuilt them in the following order, with the following rough
>> times. I took a guess that the speed of the rebuild might be related to the
>> number of distinct values in the column -- this seems true in some cases but
>> not in others. Here are the times:
> 
>> id_response       38 secs (86000 distinct)
>> id_topic          33 secs (6 distinct)
>> id_survey         13 secs (1 distinct)
>> id_code           39 secs (1444 distinct)
>> id_item           40 secs (65 distinct)
>> id_administration 13 secs (1 distinct)
> 
> How many rows altogether in this table?  What sort_mem are you using
> (and have you tried altering it)?

Sorry, left out important info. I cleaned the table first (it's a temporary
copy), then imported one run of data, 86,000 rows.

I haven't checked sort_mem, will do so.
> 
> When I read your previous message I was wondering about different
> datatypes having different comparison costs, but since these are all
> integer columns that's clearly not the issue here.  I think you may
> be seeing something related to the number of initial runs created in the
> sorting algorithm --- with only one distinct value, there'd always be
> just one run and no need for any merge passes.
> 
> (Why are you bothering to index columns with only one or a few values in
> them, anyway?  Such an index will be useless for searches ...)

Again, I left out some useful details. Id_survey and id_administration will
be identical for a single run of data (which can generate 10-100K inserts).
And my tests were just one data set, imported into a cleaned table. Still,
the distinctness of these columns will be low -- on the order of one
distinct value per 10^4 rows. The others have levels of distinctness
proportional to what the above chart shows -- id_response is unique,
id_topic will have 4-8 values per 10^4 records, and so on.

And still, none of this explains to me why the index on id_item drove the
COPY cost up so apparently dramatically. I tried the COPY again both with
and without that one index and in both cases it caused COPY to go from 5 to
40+ seconds.

-- sgl




view thread (4+ messages)  latest in thread

Message-ID: <BC489874.9895%slane@moyergroup.com>
Permalink:  ../BC489874.9895%25slane@moyergroup.com/
Also on:    postgresql.org/message-id/BC489874.9895%slane@moyergroup.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: slane@moyergroup.com
  Subject: Re: Index (re)-creation speed
  In-Reply-To: <BC489874.9895%slane@moyergroup.com>
  References: <24649.1076049015@sss.pgh.pa.us> <BC489874.9895%slane@moyergroup.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox