Subject: Re: Proposal for CSN based snapshots
To: Tom Lane <tgl@sss.pgh.pa.us>
References: <51B6C97B.1070000@bluegap.ch>
 <CAM-w4HMG-YeTfWc3r_excej3Kd04q4zDDXR0hxShVbij211ABA@mail.gmail.com>
 <539AD153.9000004@vmware.com> <1409043789.2335.330.camel@jeff-desktop>
 <53FC6545.7000501@vmware.com> <1409116988.2335.463.camel@jeff-desktop>
 <53FD7D65.9040602@vmware.com> <53FD7DCC.5020005@vmware.com>
 <CAPpHfdsCUb4qBUHzfqNsUYALC_LEpHD9qB-ozvS_8esv1edCTw@mail.gmail.com>
 <CANP8+jLAe3Smtq3FF49DMvuxtfoRQcCPQn28GEKuwAb2J41XCA@mail.gmail.com>
 <CA+Tgmob1DmDsvc7iyd0SY5j4SFdesDwoef36bxG1pE_s1JQXOQ@mail.gmail.com>
 <CANP8+jLyhL92eJi7RVX_yvbKjxm9=OBKcmNnodaYmeE+o2i1+Q@mail.gmail.com>
 <CAPpHfdv7BMwGv=OfUg3S-jGVFKqHi79pR_ZK1Wsk-13oZ+cy5g@mail.gmail.com>
 <69041c5a-ec60-3c54-49d0-5df071b27f52@iki.fi>
 <CAPpHfdsGqb1EtmWo7wJZz=TXaBM_xwAEhfr-nNG8+c57Tm_cbQ@mail.gmail.com! >
 <a7e33ad4-cf1e-2e7c-6752-de91302292dc@iki.fi> <5596.1470838147@sss.pgh.!
 pa.us> <79213506-4c9c-7e7a-44c6-ae94484c1159@iki.fi>
 <6906.1470840684@sss.pgh.pa.us>
Cc: Alexander Korotkov <a.korotkov@postgrespro.ru>,
 Simon Riggs <simon@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>,
 Dilip Kumar <dilipbalaut@gmail.com>, Jeff Davis <pgsql@j-davis.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Alvaro Herrera <alvherre@2ndquadrant.com>, Greg Stark <stark@mit.edu>,
 Andres Freund <andres@2ndquadrant.com>,
 Rajeev rastogi <rajeev.rastogi@huawei.com>, Markus Wanner
 <markus@bluegap.ch>, Ants Aasma <ants@cybertec.at>,
 Bruce Momjian <bruce@momjian.us>, obartunov <obartunov@postgrespro.ru>,
 Teodor Sigaev <teodor@postgrespro.ru>
From: Heikki Linnakangas <hlinnaka@iki.fi>
Message-ID: <8c150c9d-6ba2-69fd-e908-ed2fc2775c05@iki.fi>
Date: Wed, 10 Aug 2016 18:09:25 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Icedove/45.1.0
MIME-Version: 1.0
In-Reply-To: <6906.1470840684@sss.pgh.pa.us>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 08/10/2016 05:51 PM, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> On 08/10/2016 05:09 PM, Tom Lane wrote:
>>> Uh, what?  That's not the semantics we have today, and I don't see why
>>> it's necessary or a good idea.  Once the commit is in the WAL stream,
>>> any action taken on the basis of seeing the commit must be later in
>>> the WAL stream.  So what's the problem?
>
>> I was talking about synchronous commits in the above. A synchronous
>> commit is not made visible to other transactions, until the commit WAL
>> record is flushed to disk.
>
> [ thinks for a bit... ]  Oh, OK, that's because we don't treat a
> transaction as committed until its clog bit is set *and* it's not
> marked as running in the PGPROC array.  And sync transactions will
> flush WAL in between.

Right.

> Still, having to invent CSNs seems like a huge loss for this design.
> Personally I'd give up async commit first.  If we had only sync commit,
> the rule could be "xact LSN less than snapshot threshold and less than
> WAL flush position", and we'd not need CSNs.  I know some people like
> async commit, but it's there because it was easy and cheap in our old
> design, not because it's the world's greatest feature and worth giving
> up performance for.

I don't think that's a very popular opinion (I disagree, for one). 
Asynchronous commits are a huge performance boost for some applications. 
The alternative is fsync=off, and I don't want to see more people doing 
that. SSDs have made the penalty of an fsync much smaller, but it's 
still there.

Hmm. There's one more possible way this could all work. Let's have CSN 
== LSN, also for asynchronous commits. A snapshot is the current insert 
position, but also make note of the current flush position, when you 
take a snapshot. Now, when you use the snapshot, if you ever see an XID 
that committed between the snapshot's insert position and the flush 
position, wait for the WAL to be flushed up to the snapshot's insert 
position at that point. With that scheme, an asynchronous commit could 
return to the application without waiting for a flush, but if someone 
actually looks at the changes the transaction made, then that 
transaction would have to wait. Furthermore, we could probably skip that 
waiting too, if the reading transaction is also using 
synchronous_commit=off.

That's slightly different from the current behaviour. A transaction that 
runs with synchronous_commit=on, and reads data that was modified by an 
asynchronous transaction, would take a hit. But I think that would be 
acceptable.

- Heikki


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers