From: Heikki Linnakangas <heikki@enterprisedb.com>
To: Pablo Alcaraz <pabloa@laotraesquina.com.ar>
Cc: pgsql-performance@postgresql.org
Subject: Re: tables with 300+ partitions
Date: Tue, 30 Oct 2007 18:56:50 +0000
Message-ID: <47277E72.6010101@enterprisedb.com> (raw)
In-Reply-To: <47277341.7070704@laotraesquina.com.ar>
References: <47277341.7070704@laotraesquina.com.ar>
Pablo Alcaraz wrote:
> I had a big big big table. I tried to divide it in 300 partitions with
> 30M rows each one. The problem was when I used the table to insert
> information: the perfomance was LOW.
That's very vague. What exactly did you do? Just inserted a few rows, or
perhaps a large bulk load of millions of rows? What was the bottleneck,
disk I/O or CPU usage? How long did the operation take, and how long did
you expect it to take?
> I did some testing. I created a 300 partitioned empty table. Then, I
> inserted some rows on it and the perfomance was SLOW too.
>
> SLOW = 1% perfomance compared with a non partitioned table. That is too
> much.
>
> Then, I did a 10 partitioned table version with 30M rows each one and I
> inserted rows there. The performance was the same that the no
> partitioned table version.
That suggests that the CPU time is spent in planning the query, possibly
in constraint exclusion. But that's a very different scenario from
having millions of rows in each partition.
> I suspect there is a lock problem there. I think every SQL command do a
> lock to ALL the partitions so the perfomance with concurrent inserts and
> updates are far worst than the no partitioned version.
Every query takes an AccessShareLock on each partition, but that doesn't
prevent concurrent inserts or updates, and acquiring the locks isn't
very expensive. In other words: no, that's not it.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: heikki@enterprisedb.com, pabloa@laotraesquina.com.ar
Subject: Re: tables with 300+ partitions
In-Reply-To: <47277E72.6010101@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox