From: Stephen Frost <sfrost@snowman.net>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Maxim Orlov <orlovmg@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Add 64-bit XIDs into PostgreSQL 15
Date: Fri, 7 Jan 2022 17:46:38 -0500
Message-ID: <20220107224638.GR15820@tamriel.snowman.net> (raw)
In-Reply-To: <CA+TgmobCrg7mwo=wcKgGvEUTCsF+yWH4=Vhr2XyVC_f+0JJwqg@mail.gmail.com>
References: <CACG=ezZe1NQSCnfHOr78AtAZxJZeCvxrts0ygrxYwe=pyyjVWA@mail.gmail.com>
<YdYxiA9X3iUJiGpt@momjian.us>
<CAPpHfdt42uUWfFp3fc9J9rAozYuA_J7uQkdb8MNap3JOhCXT6w@mail.gmail.com>
<CA+TgmobCrg7mwo=wcKgGvEUTCsF+yWH4=Vhr2XyVC_f+0JJwqg@mail.gmail.com>
Greetings,
* Robert Haas (robertmhaas@gmail.com) wrote:
> On Wed, Jan 5, 2022 at 9:53 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > 5) 32-bit limitation within the page mentioned upthread by Stephen
> > Frost should be also carefully considered. Ideally, we should get rid
> > of it, but I don't have particular ideas in this field for now. At
> > least, we should make sure we did our best at error reporting and
> > monitoring capabilities.
>
> I don't think I understand the thinking here. As long as we retain the
> existing limit that the oldest running XID can't be more than 2
> billion XIDs in the past, we can't ever need to throw an error. A new
> page modification that finds very old XIDs on the page can always
> escape trouble by pruning the page and freezing whatever old XIDs
> survive pruning.
So we'll just fail such an old transaction? Or force a server restart?
or..? What if we try to signal that transaction and it doesn't go away?
> I would argue that it's smarter not to change the in-memory
> representation of XIDs to 64-bit in places like the ProcArray. As you
> mention in (4), that might hurt performance. But also, the benefit is
> minimal. Nobody is really sad that they can't keep transactions open
> forever. They are sad because the system has severe bloat and/or shuts
> down entirely. Some kind of change along these lines can fix the
> second of those problems, and that's progress.
I brought up the concern that I did because I would be a bit sad if I
couldn't have a transaction open for a day on a very high rate system of
the type being discussed here. Would be fantastic if we had a solution
to that issue, but I get that reducing the need to vacuum and such would
be a really nice improvement even if we can't make long running
transactions work. Then again, if we do actually change the in-memory
bits- then maybe we could have such a long running transaction, provided
it didn't try to make an update to a page with really old xids on it,
which might be entirely reasonable in a lot of cases. I do still worry
about how we explain what the limitation here is and how to avoid
hitting it. Definitely seems like a 'gotcha' that people are going to
complain about, though hopefully not as much of one as the current cases
we hear about of vacuum falling behind and the system running out of
xids.
> > I think the realistic goal for PG 15 development cycle would be
> > agreement on a roadmap for all the items above (and probably some
> > initial implementations).
>
> +1. Trying to rush something through to commit is just going to result
> in a bunch of bugs. We need to work through the issues carefully and
> take the time to do it well.
+1.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (818B, ../20220107224638.GR15820@tamriel.snowman.net/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: sfrost@snowman.net, robertmhaas@gmail.com, aekorotkov@gmail.com, bruce@momjian.us, orlovmg@gmail.com
Subject: Re: Add 64-bit XIDs into PostgreSQL 15
In-Reply-To: <20220107224638.GR15820@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox