Date: Fri, 7 Jan 2022 17:46:38 -0500
From: Stephen Frost <sfrost@snowman.net>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Alexander Korotkov <aekorotkov@gmail.com>,
	Bruce Momjian <bruce@momjian.us>, Maxim Orlov <orlovmg@gmail.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Add 64-bit XIDs into PostgreSQL 15
Message-ID: <20220107224638.GR15820@tamriel.snowman.net>
References: 
 <CACG=ezZe1NQSCnfHOr78AtAZxJZeCvxrts0ygrxYwe=pyyjVWA@mail.gmail.com>
 <YdYxiA9X3iUJiGpt@momjian.us>
 <CAPpHfdt42uUWfFp3fc9J9rAozYuA_J7uQkdb8MNap3JOhCXT6w@mail.gmail.com>
 <CA+TgmobCrg7mwo=wcKgGvEUTCsF+yWH4=Vhr2XyVC_f+0JJwqg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="v15hXiddb3kq3Bam"
Content-Disposition: inline
In-Reply-To: 
 <CA+TgmobCrg7mwo=wcKgGvEUTCsF+yWH4=Vhr2XyVC_f+0JJwqg@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20220107224638.GR15820%40tamriel.snowman.net>
Precedence: bulk


--v15hXiddb3kq3Bam
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Robert Haas (robertmhaas@gmail.com) wrote:
> On Wed, Jan 5, 2022 at 9:53 PM Alexander Korotkov <aekorotkov@gmail.com> =
wrote:
> > 5) 32-bit limitation within the page mentioned upthread by Stephen
> > Frost should be also carefully considered.  Ideally, we should get rid
> > of it, but I don't have particular ideas in this field for now.  At
> > least, we should make sure we did our best at error reporting and
> > monitoring capabilities.
>=20
> I don't think I understand the thinking here. As long as we retain the
> existing limit that the oldest running XID can't be more than 2
> billion XIDs in the past, we can't ever need to throw an error. A new
> page modification that finds very old XIDs on the page can always
> escape trouble by pruning the page and freezing whatever old XIDs
> survive pruning.

So we'll just fail such an old transaction?  Or force a server restart?
or..?  What if we try to signal that transaction and it doesn't go away?

> I would argue that it's smarter not to change the in-memory
> representation of XIDs to 64-bit in places like the ProcArray. As you
> mention in (4), that might hurt performance. But also, the benefit is
> minimal. Nobody is really sad that they can't keep transactions open
> forever. They are sad because the system has severe bloat and/or shuts
> down entirely. Some kind of change along these lines can fix the
> second of those problems, and that's progress.

I brought up the concern that I did because I would be a bit sad if I
couldn't have a transaction open for a day on a very high rate system of
the type being discussed here.  Would be fantastic if we had a solution
to that issue, but I get that reducing the need to vacuum and such would
be a really nice improvement even if we can't make long running
transactions work.  Then again, if we do actually change the in-memory
bits- then maybe we could have such a long running transaction, provided
it didn't try to make an update to a page with really old xids on it,
which might be entirely reasonable in a lot of cases.  I do still worry
about how we explain what the limitation here is and how to avoid
hitting it.  Definitely seems like a 'gotcha' that people are going to
complain about, though hopefully not as much of one as the current cases
we hear about of vacuum falling behind and the system running out of
xids.

> > I think the realistic goal for PG 15 development cycle would be
> > agreement on a roadmap for all the items above (and probably some
> > initial implementations).
>=20
> +1. Trying to rush something through to commit is just going to result
> in a bunch of bugs. We need to work through the issues carefully and
> take the time to do it well.

+1.

Thanks,

Stephen

--v15hXiddb3kq3Bam
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJh2MLNAAoJEO1sijiDR2RVWtAQAMOMVPzL/e9QxJRwV1dOh1d2
iJbFW18/N3q3B4lScLWvJgCo4UoJ3GZoTZ06GLLQ29zdQTigrxw+C3vg6KQGzIJh
9ZYCWIwMBgALSqRt4kz5Sm0E84DJ1comviHJORM7bwCnDtQsMCfQqiLtzEldDbAC
HDuEyDNnRyI+1HkNco1D//iZcm2M1rfGl0RlVoXR1S2EOb1rKqHm/jcQgxxnxVKt
a15Dbi/6blLg48muyOvHVYBisa+6foK1orYMdVHodhVbPyLYg+qaWafXR30TmzZq
5lXgUSPxGjXYnSPuBwFjlA2ZwvczyKT8w7H6irMbP0aTjGAOy2SY/VdsJasrZAW5
i8BEqbevkPIeSLD2Aply2NCmoG/OiCm1/bHMeaaVLUtXKr2EtmJ4NymoAg79HZk0
Fyfj8QzWC26h6X/KWouXN8AJj/46LVR6N1WKmwZKCSrAr9zVoicFrgZIkUNflFsV
Bx0hho7zNw1iIVojNrpuK28PI/L10W4ZIHSIjtTeQxA9P8wo1pdqHvwkQ1J4sP1V
9sGLkJj8k+ze0RFDFx1fw36FU4qhVvcvXg5sCVtGlnxxcbFNDzgL+9bC3Qj2mKwc
AeKkSgr8HL6uNHf1tFwjHKlZtxN/FXineM6Vg99pQSSgIj4v7ELPi9KgYD9wu7cx
cd4uyFtS81MbtPdZBtjn
=09A2
-----END PGP SIGNATURE-----

--v15hXiddb3kq3Bam--