pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Dennis Gearon <gearond@fireserve.net>
To: Jan Wieck <JanWieck@yahoo.com>
Cc: Chris Travers <chris@travelamericas.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alvaro Herrera <alvherre@dcc.uchile.cl>
Cc: Ron Johnson <ron.l.johnson@cox.net>
Cc: pgsql-general@postgresql.org
Subject: Re: Replication Ideas
Date: Tue, 26 Aug 2003 21:33:59 -0700
Message-ID: <3F4C34B7.3050307@fireserve.net> (raw)
In-Reply-To: <3F4C0CAE.3030901@Yahoo.com>
References: <3F483EBD.3020108@travelamericas.com>
	<1061705587.3733.452.camel@haggis>
	<3F4A420E.6090604@travelamericas.com>
	<20030825182441.GB9597@dcc.uchile.cl>
	<3F4A5724.3060002@travelamericas.com>
	<11627.1061845759@sss.pgh.pa.us>
	<3F4A8A7D.30308@travelamericas.com>
	<3F4C0CAE.3030901@Yahoo.com>

Jan Wieck wrote:

> WARNING: This is getting long ...
>
> Postgres-R is a very interesting and inspiring idea. And I've been 
> kicking that concept around for a while now. What I don't like about 
> it is that it requires fundamental changes in the lock mechanism and 
> that it is based on the assumption of very low lock conflict.
>
> <explain-PG-R>
> In Postgres-R a committing transaction sends it's workset (WS - a list 
> of all updates done in this transaction) to the group communication 
> system (GC). The GC guarantees total order, meaning that all nodes 
> will receive all WSs in the same order, no matter how they have been 
> sent.
>
> If a node receives back it's own WS before any error occured, it goes 
> ahead and finalizes the commit. If it receives a foreign WS, it has to 
> apply the whole WS and commit it before it can process anything else. 
> If now a local transaction, in progress or while waiting for it's WS 
> to come back, holds a lock that is required to process such remote WS, 
> the local transaction needs to be aborted to unlock it's resources ... 
> it lost the total order race.
> </explain-PG-R>
>
> Postgres-R requires that all remote WSs are applied and committed 
> before a local transaction can commit. Otherwise it couldn't correctly 
> detect a lock conflict. So there will not be any read ahead. And since 
> the total order really counts here, it cannot apply any two remote WSs 
> in parallel, a race condition could possibly exist and a later WS in 
> the total order runs faster and locks up a previous one, so we have to 
> squeeze all remote WSs through one single replication work process. 
> And all the locally parallel executed transactions that wait for their 
> WSs to come back have to wait until that poor little worker is done 
> with the whole pile. Bye bye concurrency. And I don't know how the GC 
> will deal with the backlog either. Could well choke on it.
>
> I do not see how this will scale well in a multi-SMP-system cluster. 
> At least the serialization of WSs will become a horror if there is 
> significant lock contention like in a standard TPC-C on the district 
> row containing the order number counter. I don't know for sure, but I 
> suspect that with this kind of bottleneck, Postgres-R will have to 
> rollback more than 50% of it's transactions when there are more than 4 
> nodes under heavy load (like in a benchmark run). That will suck ...
>
>
> But ... initially I said that it is an inspiring concept ... soooo ...
>
> I am currently hacking around with some C+PL/TclU+Spread constructs 
> that might form a rude kind of prototype creature.
>
> My changes to the Postgres-R concept are that there will be as many 
> replicating slave processes as there are in summary masters out in the 
> cluster ... yes, it will try to utilize all the CPU's in the cluster! 
> For failover reliability, A committing transaction will hold before 
> finalizing the commit and send it's "I'm ready" to the GC. Every 
> replicator that reaches the same state send's "I'm ready" too. Spread 
> guarantees in SAFE_MESS mode that messages are delivered to all nodes 
> in a group or that at least LEAVE/DISCONNECT messages are deliverd 
> before. So if a node receives more than 50% of "I'm ready", there 
> would be a very small gap where multiple nodes have to fail in the 
> same split second so that the majority of nodes does NOT commit. A 
> node that reported "I'm ready" but lost more than 50% of the cluster 
> before committing has to rollback and rejoin or wait for operator 
> intervention.
>
> Now the idea is to split up the communication into GC distribution 
> groups per transaction. So working master backends and associated 
> replication backends will join/leave a unique group for every 
> transaction in the cluster. This way, the per process communication is 
> reduced to the required minimum.
>
>
> As said, I am hacking on some code ...
>
>
> Jan
>
> Chris Travers wrote:
>
>> Tom Lane wrote:
>>
>>> Chris Travers <chris@travelamericas.com> writes:
>>>  
>>>
>>>> Yes I have. Postgres-r is not a high-availability solution which is 
>>>> capable of transparent failover,
>>>>   
>>>
>>>
>>> What makes you say that?  My understanding is it's supposed to survive
>>> loss of individual servers.
>>>
>>>             regards, tom lane
>>>
>>>
>>>  
>>>
>> My mistake.  I must have gotten them confused with another 
>> (asynchronous) replication project.
>>
>> Best Wishes,
>> Chris Travers
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 9: the planner will ignore your desire to choose an index scan if 
>> your
>>       joining column's datatypes do not match
>
>
>
As my british friends would say, "Bully for you",and I applaud you 
playing, struggling, learning from this for our sakes. Jeez, all I think 
about is me,huh?




view thread (22+ messages)  latest in thread

Message-ID: <3F4C34B7.3050307@fireserve.net>
Permalink:  ../3F4C34B7.3050307@fireserve.net/
Also on:    postgresql.org/message-id/3F4C34B7.3050307@fireserve.net

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: gearond@fireserve.net, JanWieck@yahoo.com, chris@travelamericas.com, tgl@sss.pgh.pa.us, alvherre@dcc.uchile.cl, ron.l.johnson@cox.net, pgsql-general@postgresql.org
  Subject: Re: Replication Ideas
  In-Reply-To: <3F4C34B7.3050307@fireserve.net>
  References: <3F483EBD.3020108@travelamericas.com> <1061705587.3733.452.camel@haggis> <3F4A420E.6090604@travelamericas.com> <20030825182441.GB9597@dcc.uchile.cl> <3F4A5724.3060002@travelamericas.com> <11627.1061845759@sss.pgh.pa.us> <3F4A8A7D.30308@travelamericas.com> <3F4C0CAE.3030901@Yahoo.com> <3F4C34B7.3050307@fireserve.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox