Date: Wed, 20 May 2020 13:54:04 +0900 (JST)
Message-Id: <20200520.135404.64670166185539892.horikyota.ntt@gmail.com>
To: x4mmm@yandex-team.ru
Cc: pgsql-hackers@postgresql.org
Subject: Re: MultiXact\SLRU buffers configuration
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
In-Reply-To: <E29C4184-23B3-4AEF-8D82-34C0A7F7224B@yandex-team.ru>
References: <102F673B-C9C5-4823-A92F-96799FC61ED0@yandex-team.ru>
	<20200515.090333.24867479329066911.horikyota.ntt@gmail.com>
	<E29C4184-23B3-4AEF-8D82-34C0A7F7224B@yandex-team.ru>
User-Agent: Mew version 6.8 on Emacs 26.1
Mime-Version: 1.0
Content-Type: Text/Plain; charset=koi8-r
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

At Fri, 15 May 2020 14:01:46 +0500, "Andrey M. Borodin" <x4mmm@yandex-t=
eam.ru> wrote in =

> =

> =

> > 15 =CD=C1=D1 2020 =C7., =D7 05:03, Kyotaro Horiguchi <horikyota.ntt=
@gmail.com> =CE=C1=D0=C9=D3=C1=CC(=C1):
> > =

> > At Thu, 14 May 2020 11:44:01 +0500, "Andrey M. Borodin" <x4mmm@yand=
ex-team.ru> wrote in =

> >>> GetMultiXactIdMembers believes that 4 is successfully done if 2
> >>> returned valid offset, but actually that is not obvious.
> >>> =

> >>> If we add a single giant lock just to isolate ,say,
> >>> GetMultiXactIdMember and RecordNewMultiXact, it reduces concurren=
cy
> >>> unnecessarily.  Perhaps we need finer-grained locking-key for sta=
ndby
> >>> that works similary to buffer lock on primary, that doesn't cause=

> >>> confilicts between irrelevant mxids.
> >>> =

> >> We can just replay members before offsets. If offset is already th=
ere - members are there too.
> >> But I'd be happy if we could mitigate those 1000us too - with a hi=
nt about last maixd state in a shared MX state, for example.
> > =

> > Generally in such cases, condition variables would work.  In the
> > attached PoC, the reader side gets no penalty in the "likely" code
> > path.  The writer side always calls ConditionVariableBroadcast but =
the
> > waiter list is empty in almost all cases.  But I couldn't cause the=

> > situation where the sleep 1000u is reached.
> Thanks! That really looks like a good solution without magic timeouts=
. Beautiful!
> I think I can create temporary extension which calls MultiXact API an=
d tests edge-cases like this 1000us wait.
> This extension will also be also useful for me to assess impact of bi=
gger buffers, reduced read locking (as in my 2nd patch) and other tweak=
s.

Happy to hear that, It would need to use timeout just in case, though.

> >> Actually, if we read empty mxid array instead of something that is=
 replayed just yet - it's not a problem of inconsistency, because trans=
action in this mxid could not commit before we started. ISTM.
> >> So instead of fix, we, probably, can just add a comment. If this r=
easoning is correct.
> > =

> > The step 4 of the reader side reads the members of the target mxid.=
 It
> > is already written if the offset of the *next* mxid is filled-in.
> Most often - yes, but members are not guaranteed to be filled in orde=
r. Those who win MXMemberControlLock will write first.
> But nobody can read members of MXID before it is returned. And its me=
mbers will be written before returning MXID.

Yeah, right.  Otherwise assertion failure happens.


regards.

-- =

Kyotaro Horiguchi
NTT Open Source Software Center