MIME-Version: 1.0
References: <20210423204306.5osfpkt2ggaedyvy@alap3.anarazel.de>
 <CAP4vRV5gEHFLB7NwOE6_dyHAeVfkvqF8Z_g5GaCQZNgBAE0Frw@mail.gmail.com>
 <20210611011850.detxnd7bnffg3daz@alap3.anarazel.de>
 <CAD21AoC-xMv1pModWvUQ1g1qByxMmu6yLghsKVb-R35rmShiwA@mail.gmail.com>
 <CAD21AoDVhkXp8HjpFO-gp3TgL6tCKcZQNxn04m01VAtcSi-5sA@mail.gmail.com>
In-Reply-To: 
 <CAD21AoDVhkXp8HjpFO-gp3TgL6tCKcZQNxn04m01VAtcSi-5sA@mail.gmail.com>
From: Ian Lawrence Barwick <barwick@gmail.com>
Date: Wed, 16 Nov 2022 13:38:10 +0900
Message-ID: 
 <CAB8KJ=j1b3kscX8Cg5G=Q39ZQsv2x4URXsuTueJLz=fcvJ3eoQ@mail.gmail.com>
Subject: Re: Testing autovacuum wraparound (including failsafe)
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Andres Freund <andres@anarazel.de>,
 Anastasia Lubennikova <lubennikovaav@gmail.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>, Peter Geoghegan <pg@bowt.ie>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAB8KJ%3Dj1b3kscX8Cg5G%3DQ39ZQsv2x4URXsuTueJLz%3DfcvJ3eoQ%40mail.gmail.com>
Precedence: bulk

2022=E5=B9=B46=E6=9C=8830=E6=97=A5(=E6=9C=A8) 10:40 Masahiko Sawada <sawada=
.mshk@gmail.com>:
>
> Hi,
>
> On Tue, Feb 1, 2022 at 11:58 AM Masahiko Sawada <sawada.mshk@gmail.com> w=
rote:
> >
> > On Fri, Jun 11, 2021 at 10:19 AM Andres Freund <andres@anarazel.de> wro=
te:
> > >
> > > Hi,
> > >
> > > On 2021-06-10 16:42:01 +0300, Anastasia Lubennikova wrote:
> > > > Cool. Thank you for working on that!
> > > > Could you please share a WIP patch for the $subj? I'd be happy to h=
elp with
> > > > it.
> > >
> > > I've attached the current WIP state, which hasn't evolved much since
> > > this message... I put the test in src/backend/access/heap/t/001_emerg=
ency_vacuum.pl
> > > but I'm not sure that's the best place. But I didn't think
> > > src/test/recovery is great either.
> > >
> >
> > Thank you for sharing the WIP patch.
> >
> > Regarding point (1) you mentioned (StartupSUBTRANS() takes a long time
> > for zeroing out all pages), how about using single-user mode instead
> > of preparing the transaction? That is, after pg_resetwal we check the
> > ages of datfrozenxid by executing a query in single-user mode. That
> > way, we don=E2=80=99t need to worry about autovacuum concurrently runni=
ng
> > while checking the ages of frozenxids. I=E2=80=99ve attached a PoC patc=
h that
> > does the scenario like:
> >
> > 1. start cluster with autovacuum=3Doff and create tables with a few dat=
a
> > and make garbage on them
> > 2. stop cluster and do pg_resetwal
> > 3. start cluster in single-user mode
> > 4. check age(datfrozenxid)
> > 5. stop cluster
> > 6. start cluster and wait for autovacuums to increase template0,
> > template1, and postgres datfrozenxids
>
> The above steps are wrong.
>
> I think we can expose a function in an extension used only by this
> test in order to set nextXid to a future value with zeroing out
> clog/subtrans pages. We don't need to fill all clog/subtrans pages
> between oldestActiveXID and nextXid. I've attached a PoC patch for
> adding this regression test and am going to register it to the next
> CF.
>
> BTW, while testing the emergency situation, I found there is a race
> condition where anti-wraparound vacuum isn't invoked with the settings
> autovacuum =3D off, autovacuum_max_workers =3D 1. AN autovacuum worker
> sends a signal to the postmaster after advancing datfrozenxid in
> SetTransactionIdLimit(). But with the settings, if the autovacuum
> launcher attempts to launch a worker before the autovacuum worker who
> has signaled to the postmaster finishes, the launcher exits without
> launching a worker due to no free workers. The new launcher won=E2=80=99t=
 be
> launched until new XID is generated (and only when new XID % 65536 =3D=3D
> 0). Although autovacuum_max_workers =3D 1 is not mandatory for this
> test, it's easier to verify the order of operations.

Hi

Thanks for the patch. While reviewing the patch backlog, we have determined=
 that
the latest version of this patch was submitted before meson support was
implemented, so it should have a "meson.build" file added for consideration=
 for
inclusion in PostgreSQL 16.

Regards

Ian Barwick