From: Simon Riggs <simon@2ndQuadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Scott Carey <scott@richrelevance.com>
Cc: Greg Smith <gsmith@gregsmith.com>
Cc: Jignesh K. Shah <J.K.Shah@sun.com>
Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: Proposal of tunable fix for scalability of 8.4
Date: Wed, 18 Mar 2009 07:48:38 +0000
Message-ID: <1237362518.3953.181.camel@ebony.2ndQuadrant> (raw)
In-Reply-To: <5077.1237046989@sss.pgh.pa.us>
References: <alpine.GSO.2.01.0903121613270.1925@westnet.com>
<C5DED7A7.33B2%scott@richrelevance.com>
<603c8f070903121829r31cf1472ha9876e5c480a542e@mail.gmail.com>
<12292.1236964604@sss.pgh.pa.us>
<49BB699D.1070809@enterprisedb.com>
<5077.1237046989@sss.pgh.pa.us>
On Sat, 2009-03-14 at 12:09 -0400, Tom Lane wrote:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
> > WALInsertLock is also quite high on Jignesh's list. That I've seen
> > become the bottleneck on other tests too.
>
> Yeah, that's been seen to be an issue before. I had the germ of an idea
> about how to fix that:
>
> ... with no lock, determine size of WAL record ...
> obtain WALInsertLock
> identify WAL start address of my record, advance insert pointer
> past record end
> *release* WALInsertLock
> without lock, copy record into the space just reserved
>
> The idea here is to allow parallelization of the copying of data into
> the buffers. The hold time on WALInsertLock would be very short. Maybe
> it could even become a spinlock, though I'm not sure, because the
> "advance insert pointer" bit is more complicated than it looks (you have
> to allow for the extra overhead when crossing a WAL page boundary).
>
> Now the fly in the ointment is that there would need to be some way to
> ensure that we didn't write data out to disk until it was valid; in
> particular how do we implement a request to flush WAL up to a particular
> LSN value, when maybe some of the records before that haven't been fully
> transferred into the buffers yet? The best idea I've thought of so far
> is shared/exclusive locks on the individual WAL buffer pages, with the
> rather unusual behavior that writers of the page would take shared lock
> and only the reader (he who has to dump to disk) would take exclusive
> lock. But maybe there's a better way. Currently I don't believe that
> dumping a WAL buffer (WALWriteLock) blocks insertion of new WAL data,
> and it would be nice to preserve that property.
Yeh, that's just what we'd discussed previously:
http://markmail.org/message/gectqy3yzvjs2hru#query:Reworking%20WAL%
20locking+page:1+mid:gectqy3yzvjs2hru+state:results
Are you thinking of doing this for 8.4? :-)
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: simon@2ndQuadrant.com, tgl@sss.pgh.pa.us, heikki.linnakangas@enterprisedb.com, robertmhaas@gmail.com, scott@richrelevance.com, gsmith@gregsmith.com, J.K.Shah@sun.com, Kevin.Grittner@wicourts.gov
Subject: Re: Proposal of tunable fix for scalability of 8.4
In-Reply-To: <1237362518.3953.181.camel@ebony.2ndQuadrant>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox