Message-ID: <64524348-b10e-9904-4858-8fe62d058b14@iki.fi>
Date: Fri, 3 Mar 2023 13:34:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: Testing autovacuum wraparound (including failsafe)
Content-Language: en-US
To: Ian Lawrence Barwick <barwick@gmail.com>,
 Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Andres Freund <andres@anarazel.de>,
 Anastasia Lubennikova <lubennikovaav@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>, Peter Geoghegan <pg@bowt.ie>
References: <20210423204306.5osfpkt2ggaedyvy@alap3.anarazel.de>
 <CAP4vRV5gEHFLB7NwOE6_dyHAeVfkvqF8Z_g5GaCQZNgBAE0Frw@mail.gmail.com>
 <20210611011850.detxnd7bnffg3daz@alap3.anarazel.de>
 <CAD21AoC-xMv1pModWvUQ1g1qByxMmu6yLghsKVb-R35rmShiwA@mail.gmail.com>
 <CAD21AoDVhkXp8HjpFO-gp3TgL6tCKcZQNxn04m01VAtcSi-5sA@mail.gmail.com>
 <CAB8KJ=j1b3kscX8Cg5G=Q39ZQsv2x4URXsuTueJLz=fcvJ3eoQ@mail.gmail.com>
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: <CAB8KJ=j1b3kscX8Cg5G=Q39ZQsv2x4URXsuTueJLz=fcvJ3eoQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/64524348-b10e-9904-4858-8fe62d058b14%40iki.fi>
Precedence: bulk

On 16/11/2022 06:38, Ian Lawrence Barwick wrote:
> Thanks for the patch. While reviewing the patch backlog, we have determined that
> the latest version of this patch was submitted before meson support was
> implemented, so it should have a "meson.build" file added for consideration for
> inclusion in PostgreSQL 16.

I wanted to do some XID wraparound testing again, to test the 64-bit 
SLRUs patches [1], and revived this.

I took a different approach to consuming the XIDs. Instead of setting 
nextXID directly, bypassing GetNewTransactionId(), this patch introduces 
a helper function to call GetNewTransactionId() repeatedly. But because 
that's slow, it does include a shortcut to skip over "uninteresting" 
XIDs. Whenever nextXid is close to an SLRU page boundary or XID 
wraparound, it calls GetNewTransactionId(), and otherwise it bumps up 
nextXid close to the next "interesting" value. That's still a lot slower 
than just setting nextXid, but exercises the code more realistically.

I've written some variant of this helper function many times over the 
years, for ad hoc testing. I'd love to have it permanently in the git tree.

In addition to Masahiko's test for emergency vacuum, this includes two 
other tests. 002_limits.pl tests the "warn limit" and "stop limit" in 
GetNewTransactionId(), and 003_wraparound.pl burns through 10 billion 
transactions in total, exercising XID wraparound in general. 
Unfortunately these tests are pretty slow; the tests run for about 4 
minutes on my laptop in total, and use about 20 GB of disk space. So 
perhaps these need to be put in a special test suite that's not run as 
part of "check-world". Or perhaps leave out the 003_wraparounds.pl test, 
that's the slowest of the tests. But I'd love to have these in the git 
tree in some form.

[1] 
https://www.postgresql.org/message-id/CAJ7c6TPKf0W3MfpP2vr=kq7-NM5G12vTBhi7miu_5m8AG3Cw-w@mail.gmail.com)

- Heikki