pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedSynchronization primitives (Was: Re: An example of bugs for Hot Standby)
8+ messages / 4 participants
[nested] [flat]
* Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
@ 2010-01-20 18:00 Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:45 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Simon Riggs <simon@2ndQuadrant.com>
0 siblings, 2 replies; 8+ messages in thread
From: Heikki Linnakangas @ 2010-01-20 18:00 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
Andres Freund wrote:
> On Wednesday 20 January 2010 17:59:36 Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> I realize its way too late in the cycle for that, but why dont we start
>>> using some library for easy cross platform atomic ops?
>> (1) there probably isn't one that does exactly what we want, works
>> everywhere, and has the right license;
>> (2) what actual gain would we get? We've already done the work.
> That there might be some other instructions were interested in?
> Like really atomic increment?
This reminds me of something I've been pondering for some time:
Streaming Replication introduces a few places with a polling pattern
like this (in pseudocode):
while()
{
/* Check if variable in shared has advanced beoynd X */
SpinLockAcquire()
localvar = sharedvar;
SpinLockRelease()
if (localvar > X)
break;
/* Not yet. Sleep
pg_usleep(100);
}
For example, startup process polls like that to wait for walreceiver to
write & flush new WAL to be replayed. And in master, walsender polls
like that for new WAL to be generated, so that it can be sent to
standby. Hot standby also has a polling loop where it waits for a
transaction a transaction to die, though I'm not sure if that can be fit
into the same model.
That's OK for asynchronous replication, but unacceptable for synchronous
mode.
It would be nice to have a new synchronization primitive for that.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
@ 2010-01-20 18:16 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:49 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
1 sibling, 1 reply; 8+ messages in thread
From: Tom Lane @ 2010-01-20 18:16 UTC (permalink / raw)
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
> Streaming Replication introduces a few places with a polling pattern
> like this (in pseudocode):
> while()
> {
> /* Check if variable in shared has advanced beoynd X */
> SpinLockAcquire()
> localvar = sharedvar;
> SpinLockRelease()
> if (localvar > X)
> break;
> /* Not yet. Sleep
> pg_usleep(100);
> }
I trust there's a CHECK_FOR_INTERRUPTS in there ...
> It would be nice to have a new synchronization primitive for that.
Maybe. The lock, the variable, the comparison operation, and the sleep
time all seem rather specific to each application. Not sure that it'd
really buy much to try to turn it into a generic subroutine.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
@ 2010-01-20 18:49 ` Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 19:07 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Heikki Linnakangas @ 2010-01-20 18:49 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
Tom Lane wrote:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
>> Streaming Replication introduces a few places with a polling pattern
>> like this (in pseudocode):
>
>> while()
>> {
>> /* Check if variable in shared has advanced beoynd X */
>> SpinLockAcquire()
>> localvar = sharedvar;
>> SpinLockRelease()
>> if (localvar > X)
>> break;
>
>> /* Not yet. Sleep
>> pg_usleep(100);
>> }
>
> I trust there's a CHECK_FOR_INTERRUPTS in there ...
>
>> It would be nice to have a new synchronization primitive for that.
>
> Maybe. The lock, the variable, the comparison operation, and the sleep
> time all seem rather specific to each application. Not sure that it'd
> really buy much to try to turn it into a generic subroutine.
My point is that we should replace such polling loops with something
non-polling, using wait/signal or semaphores or something. That gets
quite a bit more complex. You'd probably still have the loop, but
instead of pg_usleep() you'd call some new primitive function that waits
until the shared variable changes.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:49 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
@ 2010-01-20 19:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 19:22 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2010-01-20 19:07 UTC (permalink / raw)
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
> My point is that we should replace such polling loops with something
> non-polling, using wait/signal or semaphores or something. That gets
> quite a bit more complex. You'd probably still have the loop, but
> instead of pg_usleep() you'd call some new primitive function that waits
> until the shared variable changes.
Maybe someday --- it's certainly not something we need to mess with for
8.5. As Simon comments, getting it to work nicely in the face of corner
cases (like processes dying unexpectedly) could be a lot of work.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:49 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 19:07 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
@ 2010-01-20 19:22 ` Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 20:10 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) David Fetter <david@fetter.org>
0 siblings, 1 reply; 8+ messages in thread
From: Heikki Linnakangas @ 2010-01-20 19:22 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
Tom Lane wrote:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
>> My point is that we should replace such polling loops with something
>> non-polling, using wait/signal or semaphores or something. That gets
>> quite a bit more complex. You'd probably still have the loop, but
>> instead of pg_usleep() you'd call some new primitive function that waits
>> until the shared variable changes.
>
> Maybe someday --- it's certainly not something we need to mess with for
> 8.5. As Simon comments, getting it to work nicely in the face of corner
> cases (like processes dying unexpectedly) could be a lot of work.
Agreed, polling is good enough for 8.5.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:49 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 19:07 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 19:22 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
@ 2010-01-20 20:10 ` David Fetter <david@fetter.org>
2010-01-20 20:18 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: David Fetter @ 2010-01-20 20:10 UTC (permalink / raw)
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
On Wed, Jan 20, 2010 at 09:22:49PM +0200, Heikki Linnakangas wrote:
> Tom Lane wrote:
> > Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
> >> My point is that we should replace such polling loops with something
> >> non-polling, using wait/signal or semaphores or something. That gets
> >> quite a bit more complex. You'd probably still have the loop, but
> >> instead of pg_usleep() you'd call some new primitive function that waits
> >> until the shared variable changes.
> >
> > Maybe someday --- it's certainly not something we need to mess with for
> > 8.5. As Simon comments, getting it to work nicely in the face of corner
> > cases (like processes dying unexpectedly) could be a lot of work.
>
> Agreed, polling is good enough for 8.5.
Is this a TODO yet?
Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:49 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 19:07 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 19:22 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 20:10 ` Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby) David Fetter <david@fetter.org>
@ 2010-01-20 20:18 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 8+ messages in thread
From: Tom Lane @ 2010-01-20 20:18 UTC (permalink / raw)
To: David Fetter <david@fetter.org>; +Cc: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; Andres Freund <andres@anarazel.de>; Simon Riggs <simon@2ndquadrant.com>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
David Fetter <david@fetter.org> writes:
> On Wed, Jan 20, 2010 at 09:22:49PM +0200, Heikki Linnakangas wrote:
>>> My point is that we should replace such polling loops with something
>>> non-polling, using wait/signal or semaphores or something.
> Is this a TODO yet?
It hardly seems concrete enough for a TODO item.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Synchronization primitives (Was: Re: An example of bugs for Hot Standby)
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
@ 2010-01-20 18:45 ` Simon Riggs <simon@2ndQuadrant.com>
1 sibling, 0 replies; 8+ messages in thread
From: Simon Riggs @ 2010-01-20 18:45 UTC (permalink / raw)
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org, Hiroyuki Yamada <yamada@kokolink.net>
On Wed, 2010-01-20 at 20:00 +0200, Heikki Linnakangas wrote:
> Hot standby also has a polling loop where it waits for a
> transaction a transaction to die, though I'm not sure if that can be
> fit into the same model
I prefer that in the context of HS because the Startup process is
waiting for things to die. Given that their death may not be handled
sweetly, I would not wish to rely on that to wake Startup.
In the other two cases you mention all processes are working together
normally and we aren't expecting the other processes to die.
--
Simon Riggs www.2ndQuadrant.com
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2010-01-20 20:18 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-01-20 18:00 Synchronization primitives (Was: Re: An example of bugs for Hot Standby) Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 18:16 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:49 ` Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 19:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 19:22 ` Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2010-01-20 20:10 ` David Fetter <david@fetter.org>
2010-01-20 20:18 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-01-20 18:45 ` Simon Riggs <simon@2ndQuadrant.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox