From: Bruce Momjian <bruce@momjian.us>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alvaro Herrera <alvherre@commandprompt.com>
Cc: Jakub Ouhrabka <kuba@comgate.cz>
Cc: pgsql-performance@postgresql.org
Subject: Re: Linux/PostgreSQL scalability issue - problem with 8 cores
Date: Fri, 21 Mar 2008 21:44:29 -0400 (EDT)
Message-ID: <200803220144.m2M1iTe16630@momjian.us> (raw)
In-Reply-To: <28297.1199753665@sss.pgh.pa.us>
Added to TODO:
* Improve performance of shared invalidation queue for multiple CPUs
http://archives.postgresql.org/pgsql-performance/2008-01/msg00023.php
---------------------------------------------------------------------------
Tom Lane wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > Perhaps it would make sense to try to take the "fast path" in
> > SIDelExpiredDataEntries with only a shared lock rather than exclusive.
>
> I think the real problem here is that sinval catchup processing is well
> designed to create contention :-(. Once we've decided that the message
> queue is getting too full, we SIGUSR1 all the backends at once (or as
> fast as the postmaster can do it anyway), then they all go off and try
> to touch the sinval queue. Backends that haven't awoken even once
> since the last time will have to process the entire queue contents,
> and they're all trying to do that at the same time. What's worse, they
> take and release the SInvalLock once for each message they take off the
> queue. This isn't so horrid for one-core machines (since each process
> will monopolize the CPU for probably less than one timeslice while it's
> catching up) but it's pretty obvious where all the contention is coming
> from on an 8-core.
>
> Some ideas for improving matters:
>
> 1. Try to avoid having all the backends hit the queue at once. Instead
> of SIGUSR1'ing everybody at the same time, maybe hit only the process
> with the oldest message pointer, and have him hit the next oldest after
> he's done reading the queue.
>
> 2. Try to take more than one message off the queue per SInvalLock cycle.
> (There is a tuning tradeoff here, since it would mean holding the lock
> for longer at a time.)
>
> 3. Try to avoid having every backend run SIDelExpiredDataEntries every
> time through ReceiveSharedInvalidMessages. It's not critical to delete
> entries until the queue starts getting full --- maybe we could rejigger
> the logic so it only happens once when somebody notices the queue is
> getting full, or so that only the guy(s) who had nextMsgNum == minMsgNum
> do it, or something like that?
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: bruce@momjian.us, tgl@sss.pgh.pa.us, alvherre@commandprompt.com, kuba@comgate.cz
Subject: Re: Linux/PostgreSQL scalability issue - problem with 8 cores
In-Reply-To: <200803220144.m2M1iTe16630@momjian.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox