From: Janet Jacobsen <jsjacobsen@lbl.gov>
To: pgsql-performance@postgresql.org
Subject: performance using table partitions in Postgres 8.2.6
Date: Mon, 07 Apr 2008 09:57:34 -0700
Message-ID: <47FA527E.4020101@lbl.gov> (raw)
Hi. I am looking for information about using table partitions in Postgres,
in particular w.r.t. performance when querying a partitioned table.
I implemented table partitioning following the documentation, which
is quite good and easy to follow (Chapter 5.9). I am doing some
testing, so at this point my test dataset has 1M records; in the table,
each row will have an autoincrement integer index, two numerics,
one integer (DMID), and one smallint.
I created five master tables and created a different number of
partitions for each one (2, 4, 8, 16, 32). I am using a range partition
for the integer, DMID, which represents a file index. The values of
DMID range from 0 to 180360. I also create and index for DMID.
I don't understand the timing results that I am getting. I got these
times by averaging the results of querying the database from within
a loop in a Perl script:
no. of partitions constraint_exclusion off constraint_exclusion on
2 0.597 ms 0.427 ms
4 0.653 ms 0.414 ms
8 0.673 ms 0.654 ms
16 1.068 ms 1.014 ms
32 2.301 ms 1.537 ms
I expected that the query time would decrease as the number of
partitions increases, but that's not what I am seeing. I get better results
(0.29 ms) if I simply index DMID and don't use the partitions.
When I run "explain analyze"on a query, the results (with and without
constraint_exclusion set) indicate that fewer partitions are being scanned
when constraint_exclusion is set to on.
I am testing table partitioning in Postgres against table partitioning
using MySQL. The results for MySQL make sense: more partitions,
faster query times.
The underlying application is a file index. It is expected that groups
of files in selected ranges of DMID values will be accessed more
often, but this is not the key implementation issue.
This is basically a "write once, read often" database. We expect that
the database will grow to 50M records in a few years, and I thought
that using range partitions for the DMID value might decrease the
query time.
Should I be using many more partitions? Am I expecting too much in
terms of performance when using partitions? Do these results point to
some obvious implementation error?
Thank you for any help/suggestions you can give.
Janet Jacobsen
--
Janet Jacobsen
NERSC Analytics/HPCRD Visualization Group
Lawrence Berkeley National Laboratory
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jsjacobsen@lbl.gov
Subject: Re: performance using table partitions in Postgres 8.2.6
In-Reply-To: <47FA527E.4020101@lbl.gov>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox