MIME-Version: 1.0
References: 
 <CAAKRu_ZP-3=SaZykpwDBMJOdUKyW3Wm5JZfPFRR3L5Ac8ouq4w@mail.gmail.com>
 <CAAKRu_bgkOQqu3K5n4YLRsNBZqJ9Rjg80ROqgKSr2UGz4b5hUg@mail.gmail.com>
 <2wk7jo4m4qwh5sn33pfgerdjfujebbccsmmlownybddbh6nawl@mdyyqpqzxjek>
 <CAAKRu_YR-COJ9aGnMUQqt5yoWmUBjikqrd4jGNZYouHaXpis9g@mail.gmail.com>
 <CALdSSPhiCwJwWwgJP1NmqRmnp9RS2tGOBY0gQrfLCbB+OS5_KQ@mail.gmail.com>
 <CAAKRu_YS+Ocm=OzMaZnG4egFiE9v4VYfZ25DXd6jbwegqmGYbQ@mail.gmail.com>
 <CAAKRu_ZGZSqhGt-RcmmfiSheC+1fjQdxy6_+oM-1jMn8hyVptQ@mail.gmail.com>
 <CALdSSPg+B8RTzTXhJvCcKJBqgzhPZkq0E2oqxQdv74ZNZOMVzg@mail.gmail.com>
 <CAAKRu_Zha7HcdQBv8tTtQrcry5332J6kHnOc1X=TT03LzUXDow@mail.gmail.com>
 <CAAKRu_amF00f2T_H8N6pbbe75C22EeX1OqA=svpj8LNO1sdUuw@mail.gmail.com>
 <mhf4vkmh3j57zx7vuxp4jagtdzwhu3573pgfpmnjwqa6i6yj5y@sy4ymcdtdklo>
 <CAAKRu_agCLAQ9OjmrTdJe-X=Xr7QnU4d=cfxdQGwc9jNx9w31w@mail.gmail.com>
 <CAAKRu_ayWLg=WDGZZfSPWf0KjPM8u=LBb0D6XaEWyx2_YFFwAQ@mail.gmail.com>
 <CALdSSPhtPK36_sr9yFo3cN9TXQmjvSX3BZXCF8fVBLX-GETD0Q@mail.gmail.com>
 <CAAKRu_Yt6EH5aFSJBm-k7PrNM4bTt56fTRbyU7gqYXe4cW+F9g@mail.gmail.com>
 <CALdSSPhH7hi+EzYqq0=eMCthi7iNpY_YyECAC1qxPb7rd0TLrw@mail.gmail.com>
 <CAAKRu_baqEkdJ0_1vokvgbaN52ycR65HM2-DR-VsRmYVVQLV8w@mail.gmail.com>
 <CALdSSPgKH7TdW0zcDPjBxx+MX18bjc65K12KgoGtX6ZP8Qq63w@mail.gmail.com>
In-Reply-To: 
 <CALdSSPgKH7TdW0zcDPjBxx+MX18bjc65K12KgoGtX6ZP8Qq63w@mail.gmail.com>
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 18 Dec 2025 14:57:57 -0500
Message-ID: 
 <CAAKRu_YEpU2+Xu+BZ8wtCZ967Xj4NZ_MtU4mq7dw_FjyboVefw@mail.gmail.com>
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM
 on-access)
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>,
	Andrey Borodin <x4mmm@yandex-team.ru>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Heikki Linnakangas <hlinnaka@iki.fi>,
	Chao Li <li.evan.chao@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAAKRu_YEpU2%2BXu%2BBZ8wtCZ967Xj4NZ_MtU4mq7dw_FjyboVefw%40mail.gmail.com>
Precedence: bulk

On Thu, Dec 18, 2025 at 1:07=E2=80=AFPM Kirill Reshke <reshkekirill@gmail.c=
om> wrote:
>
> On Thu, 18 Dec 2025 at 20:18, Melanie Plageman
> <melanieplageman@gmail.com> wrote:
>
> > But you are right, I don't see any non-error code path where a heap
> > page would become empty (all line pointers set unused) and then not be
> > set all-visible. Only vacuum sets line pointers unused and if all the
> > line pointers are unused it will always set the page all-visible.
> >
> > I think, though, that if we error out in lazy_scan_prune() after
> > returning from heap_page_prune_and_freeze() such that we don't set the
> > empty page all-visible, we can end up with an empty page without
> > PD_ALL_VISIBLE set. You can see how this might work by patching the VM
> > set code in lazy_scan_prune() to skip empty pages.
>
> Thank you for your explanation!  I completely forgot that PD_ALL_VIS
> is a non-persistent change (hint bit). so its update can be trivially
> lost.
> The simplest real-life example is being killed just after returning
> from heap_page_prune_and_freeze, yes.
> PFA tap test covering lazy_scan_new_or_empty code path for
> empty-but-not-all-visible page

Cool test! I'm going to have to think more about whether or not it is
worth adding a whole new TAP test for this codepath. Is there an
existing TAP test we could add it to so we don't need to make a new
cluster, etc? How long does the test take to run? Obviously it will be
quite short, but every bit we add to the test suite counts. I don't
actually know how much overhead there is with injection points.

I was chatting with Andres and he mentioned there is one other case
where you can end up in this code path (empty page without
PD_ALL_VISIBLE set) and this case does actually trigger this code:

            if (RelationNeedsWAL(vacrel->rel) &&
                !XLogRecPtrIsValid(PageGetLSN(page)))
                log_newpage_buffer(buf, true);

If you are inserting to a new page and you successfully call
PageInit() (making the page no longer considered new by PageIsNew()
because pd_upper will be set) but you error out before actually
inserting the tuple, then you will have an empty page without
PD_ALL_VISIBLE set. And assuming you error out before emitting WAL,
the page will not have a valid LSN set. So you will hit that code
which calls log_newpage_buffer().

I would say this case is so narrow (the log_newpage_buffer() codepath
in lazy_scan_new_or_empty()), it's not worth the added test overhead,
but I just wanted to share what I learned about when this code could
be hit.

Previously it was more common in the bulk extension case to have empty
pages not set PD_ALL_VISIBLE because bulk extension would call
PageInit() on all of the pages it extended so all the pages except the
target page were empty (today they are not initialized so they go into
the PageIsNew() branch).

So, in both cases, it seems like the empty page not set PD_ALL_VISIBLE
mostly only hit if we previously errored out.

- Melanie