From: Dave Crooke <dcrooke@gmail.com>
To: Alvaro Herrera <alvherre@commandprompt.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Greg Stark <gsstark@mit.edu>
Cc: Pierre C <lists@peufeu.com>
Cc: Greg Smith <greg@2ndquadrant.com>
Cc: Paul McGarry <paul@paulmcgarry.com>
Cc: pgsql-performance@postgresql.org
Subject: Block at a time ...
Date: Tue, 16 Mar 2010 18:58:50 -0500
Message-ID: <ca24673e1003161658g6a0fa30dg2c158db395ac08@mail.gmail.com> (raw)
I agree with Tom, any reordering attempt is at best second guessing the
filesystem and underlying storage.
However, having the ability to control the extent size would be a worthwhile
improvement for systems that walk and chew gum (write to lots of tables)
concurrently.
I'm thinking of Oracle's AUTOEXTEND settings for tablespace datafiles .... I
think the ideal way to do it for PG would be to make the equivalent
configurable in postgresql.conf system wide, and allow specific per-table
settings in the SQL metadata, similar to auto-vacuum.
An awesomely simple alternative is to just specify the extension as e.g. 5%
of the existing table size .... it starts by adding one block at a time for
tiny tables, and once your table is over 20GB, it ends up adding a whole 1GB
file and pre-allocating it. Very little wasteage.
Cheers
Dave
On Tue, Mar 16, 2010 at 4:49 PM, Alvaro Herrera
<alvherre@commandprompt.com>wrote:
> Tom Lane escribió:
> > Alvaro Herrera <alvherre@commandprompt.com> writes:
> > > Maybe it would make more sense to try to reorder the fsync calls
> > > instead.
> >
> > Reorder to what, though? You still have the problem that we don't know
> > much about the physical layout on-disk.
>
> Well, to block numbers as a first step.
>
> However, this reminds me that sometimes we take the block-at-a-time
> extension policy too seriously. We had a customer that had a
> performance problem because they were inserting lots of data to TOAST
> tables, causing very frequent extensions. I kept wondering whether an
> allocation policy that allocated several new blocks at a time could be
> useful (but I didn't try it). This would also alleviate fragmentation,
> thus helping the physical layout be more similar to logical block
> numbers.
>
> --
> Alvaro Herrera
> http://www.CommandPrompt.com/
> PostgreSQL Replication, Consulting, Custom Development, 24x7 support
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: dcrooke@gmail.com, alvherre@commandprompt.com, tgl@sss.pgh.pa.us, gsstark@mit.edu, lists@peufeu.com, greg@2ndquadrant.com, paul@paulmcgarry.com
Subject: Re: Block at a time ...
In-Reply-To: <ca24673e1003161658g6a0fa30dg2c158db395ac08@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox