MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Fri, 10 Apr 2020 18:08:59 +0300
From: Alexey Kondratov <a.kondratov@postgrespro.ru>
To: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Kyotaro Horiguchi
 <horikyota.ntt@gmail.com>, a.akenteva@postgrespro.ru,
 a.korotkov@postgrespro.ru, i.kartyshov@postgrespro.ru,
 amit.kapila16@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: [HACKERS] make async slave to wait for lsn to be replayed
In-Reply-To: <759206c1-265f-97da-d135-fcecbced646b@oss.nttdata.com>
References: <20200408.100945.375702947246835193.horikyota.ntt@gmail.com>
 <d3ff2e363af60b345f82396992595a03@postgrespro.ru>
 <13716.1586378146@sss.pgh.pa.us>
 <20200409.161107.580344062273684114.horikyota.ntt@gmail.com>
 <880834ee-4b18-6d5d-9dae-6520c5eb2bef@oss.nttdata.com>
 <27171.1586439221@sss.pgh.pa.us>
 <a8bff0350a27e0a87a6eaf0905d6737f@postgrespro.ru>
 <759206c1-265f-97da-d135-fcecbced646b@oss.nttdata.com>
User-Agent: Roundcube Webmail/1.4.0
Message-ID: <d2ef1c3fa6e660a0cda30a5e433b72a6@postgrespro.ru>
Precedence: bulk

On 2020-04-10 05:25, Fujii Masao wrote:
> On 2020/04/10 3:16, Alexey Kondratov wrote:
>> Just another idea in case if one will still decide to go with a 
>> separate statement + BEGIN integration instead of a function. We could 
>> use parenthesized options list here. This is already implemented for 
>> VACUUM, REINDEX, etc. There was an idea to allow CONCURRENTLY in 
>> REINDEX there [1] and recently this was proposed again for new options 
>> [2], since it is much more extensible from the grammar perspective.
>> 
>> That way, the whole feature may look like:
>> 
>> WAIT (LSN '16/B374D848', TIMEOUT 100);
>> 
>> and/or
>> 
>> BEGIN
>> WAIT (LSN '16/B374D848', WHATEVER_OPTION_YOU_WANT);
>> ...
>> COMMIT;
>> 
>> It requires only one reserved keyword 'WAIT'. The advantage of this 
>> approach is that it can be extended to support xid, timestamp, csn or 
>> anything else, that may be invented in the future, without affecting 
>> the grammar.
>> 
>> What do you think?
>> 
>> Personally, I find this syntax to be more convenient and 
>> human-readable compared with function call:
>> 
>> SELECT pg_wait_for_lsn('16/B374D848');
>> BEGIN;
> 
> I can imagine that some users want to specify the LSN to wait for,
> from the result of another query, for example,
> SELECT pg_wait_for_lsn(lsn) FROM xxx. If this is valid use case,
> isn't the function better?
> 

I think that the main purpose of the feature is to achieve 
read-your-writes-consistency, while using async replica for reads. In 
that case lsn of last modification is stored inside application, so 
there is no need to do any query for that. Moreover, you cannot store 
this lsn inside database, since reads are distributed across all 
replicas (+ primary).

Thus, I could imagine that 'xxx' in your example states for some kind of 
stored procedure, that fetches lsn from the off-postgres storage, but it 
looks like very narrow case to count on it, doesn't it?

Anyway, I am not against implementing this as a function. That was just 
another option to consider.

Just realized that the last patch I have seen does not allow usage of 
wait on primary. It may be a problem if reads are pooled not only across 
replicas, but on primary as well, which should be quite usual I guess. 
In that case application does not know either request will be processed 
on replica, or on primary. I think it should be allowed without any 
warning, or just saying some LOG/DEBUG at most, that there was no 
waiting performed.


Regards
-- 
Alexey Kondratov

Postgres Professional https://www.postgrespro.com
Russian Postgres Company