From: Simon Riggs <simon@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alvaro Herrera <alvherre@commandprompt.com>
Cc: Jakub Ouhrabka <kuba@comgate.cz>
Cc: pgsql-performance@postgresql.org
Subject: Re: Linux/PostgreSQL scalability issue - problem with 8 cores
Date: Fri, 25 Jan 2008 23:27:09 +0000
Message-ID: <1201303629.4257.554.camel@ebony.site> (raw)
In-Reply-To: <28297.1199753665@sss.pgh.pa.us>
References: <flii57$748$1@news.hub.org>
<7083.1199379570@sss.pgh.pa.us>
<477D1D59.7050002@comgate.cz>
<4782063C.7070105@comgate.cz>
<20080107151234.GG4723@alvh.no-ip.org>
<28297.1199753665@sss.pgh.pa.us>
On Mon, 2008-01-07 at 19:54 -0500, Tom Lane wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > Perhaps it would make sense to try to take the "fast path" in
> > SIDelExpiredDataEntries with only a shared lock rather than exclusive.
>
> I think the real problem here is that sinval catchup processing is well
> designed to create contention :-(.
Thinking some more about handling TRUNCATEs...
> Some ideas for improving matters:
>
> 1. Try to avoid having all the backends hit the queue at once. Instead
> of SIGUSR1'ing everybody at the same time, maybe hit only the process
> with the oldest message pointer, and have him hit the next oldest after
> he's done reading the queue.
>
> 2. Try to take more than one message off the queue per SInvalLock cycle.
> (There is a tuning tradeoff here, since it would mean holding the lock
> for longer at a time.)
>
> 3. Try to avoid having every backend run SIDelExpiredDataEntries every
> time through ReceiveSharedInvalidMessages. It's not critical to delete
> entries until the queue starts getting full --- maybe we could rejigger
> the logic so it only happens once when somebody notices the queue is
> getting full, or so that only the guy(s) who had nextMsgNum == minMsgNum
> do it, or something like that?
(2) is unnecessary if we can reduce the number of Exclusive lockers so
that repeated access to the backend's messages is not contended.
(1) would do this, but seems like it would be complex. We can reduce the
possibility of multiple re-signals though.
(3) seems like the easiest route, as long as we get a reasonable
algorithm for reducing the access rate to a reasonable level.
I'm posting a patch for discussion to -patches now that will do this. It
seems straightforward enough to include in 8.3, but that may rise a few
eyebrows, but read the patch first.
--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: simon@2ndquadrant.com, tgl@sss.pgh.pa.us, alvherre@commandprompt.com, kuba@comgate.cz
Subject: Re: Linux/PostgreSQL scalability issue - problem with 8 cores
In-Reply-To: <1201303629.4257.554.camel@ebony.site>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox