Feedback-ID: i0fe9450f:Fastmail
Date: Wed, 16 Aug 2023 12:37:21 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>,
	Postgres hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: pg_logical_emit_message() misses a XLogFlush()
Message-ID: <ZNxEcXPj7BFvePzl@paquier.xyz>
References: <ZNsdThSe2qgsfs7R@paquier.xyz>
 <4e680b60-9bce-7c4a-0419-f478a30d30e3@enterprisedb.com>
 <ZNv1IEHhGxrUTtwI@paquier.xyz>
 <20230816003333.7hn2rx5m2l7una3d@awork3.anarazel.de>
 <619e219c-5894-cfd0-5634-0d465c41130b@enterprisedb.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="3j6mhD1o+ZXwR2OM"
Content-Disposition: inline
In-Reply-To: <619e219c-5894-cfd0-5634-0d465c41130b@enterprisedb.com>
Archived-At: <https://www.postgresql.org/message-id/ZNxEcXPj7BFvePzl%40paquier.xyz>
Precedence: bulk


--3j6mhD1o+ZXwR2OM
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Aug 16, 2023 at 03:20:53AM +0200, Tomas Vondra wrote:
> So are you objecting to adding the flush in general, or just to the
> backpatching part?
>=20
> IMHO we either guarantee durability of non-transactional messages, in
> which case this would be a clear bug - and I'd say a fairly serious one.
> I'm curious what the workload that'd see order of magnitude of slowdown
> does with logical messages, but even if such workload exists, would it
> really be enough to fix any other durability bug?

Yes, I also think that this is a pretty serious issue to not ensure
durability in the non-transactional case if you have solutions
designed around that.

> Or perhaps we don't want to guarantee durability for such messages, in
> which case we don't need to fix it at all (even in master).

I mean, just look at the first message of the thread I am mentioning
at the top of this thread: it lists three cases where something like
pg_logical_emit_message() was wanted.  And, it looks like a serious
issue to me for the first two ones at least (out-of-order messages and
inter-node communication), because an application may want to send
something from node1 to node2, and node1 may just forget about it
entirely if it crashes, finishing WAL redo at an earlier point than
the record inserted.

> The docs are not very clear on what to expect, unfortunately. It says
> that non-transactional messages are "written immediately" which I could
> interpret in either way.

Agreed.  The original thread never mentions "flush", "sync" or
"durab".

I won't fight much if there are objections to backpatching, but that's
not really wise (no idea how much EDB's close flavor of BDR relies on
that).
--
Michael

--3j6mhD1o+ZXwR2OM
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmTcRHEACgkQnvQgOdby
QH3hRQ//aFW4CyVDoFsW0AYEgeB+3RKe4mr6iZC0sK/vCoca6BZwpT0i2HM7EsA+
QEOVORXE12/ufR6MsK8TjDyGADqEOAgm2L/jegLl/FffWWidK2KG2zJSS6bloUzx
63j2zyd+VgzJN1vLOcsD0HZxSGm/kX/VQmhs3quL+hBNHDmXiKE10BEHf3pTuLRn
at/NjuSSESF9ESf0qaddusdFDahFeFF4UcH+Y56BTatDxf4KLhtoS7dsxfXn3eZl
HBDulyJys3hQ4Kc2prjqRNQtnLY24ztcTlN4MO7uhYD1RajtW5rsgorMSbHa5q6I
5KPeu9u7R/51D3lqJnhCLOVyR73z27ZdCjWjSjzRlUzOMDmE3Qy9oKU5cp3LXYYO
M4HzolGTnZqBg8lN+NiA1BweaNzi3tfeRnQmksSMGrwEY+j6qVaxTVfFZ2MCm9FC
JjWn3/M1ol4AGxr6fMxHFsx2ATH9JbcTt4UlNDuoF6QC1ZrqBLSxDhogXA3FUWbU
JCnaQdL1X90f1xKk3krI2YhbxWljfdecwOdesC8cSjs1AcLZiCMpjlQ4OAAMqx52
o2liEsO/Arb6i5F5ipwELq1ZhoEArMRyGYBo84mh+HBevkK7m2Go5HXXq3GKLPto
YA175YU8vms2aekVCpqICTFQ5+OTR6t5jYp6juhLmWaYDFF2OAg=
=vMkA
-----END PGP SIGNATURE-----

--3j6mhD1o+ZXwR2OM--