Content-Type: multipart/mixed; boundary="------------okLIZVff3gtGKJ1J6bSSCsVJ"
Message-ID: <b641be2a-945b-999c-2828-7bbd8a16a009@gmail.com>
Date: Mon, 15 Jan 2024 12:00:00 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Subject: Re: Test slots invalidations in 035_standby_logical_decoding.pl only
 if dead rows are removed
Content-Language: en-US
To: Michael Paquier <michael@paquier.xyz>,
 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>, Tom Lane <tgl@sss.pgh.pa.us>,
 Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>,
 Thomas Munro <thomas.munro@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <ZZ4cZkBLczQISHxq@paquier.xyz>
 <7c025095-5763-17a6-8c80-899b35bd0459@gmail.com>
 <ZZ7GpII4bAYN+jT5@ip-10-97-1-34.eu-west-3.compute.internal>
 <5d0e5bec-67f9-9164-36cb-c4ff5f95d1ed@gmail.com>
 <ZaACEPtVovKlRVUf@ip-10-97-1-34.eu-west-3.compute.internal>
 <6f85667e-5754-5d35-dbf1-c83fe08c1e48@gmail.com>
 <ZaBlU5FqOeDqyHtJ@paquier.xyz>
 <ZaDnIMTbT+0yR5LZ@ip-10-97-1-34.eu-west-3.compute.internal>
 <cc7925b8-30cc-c76d-b1b6-c9ec6bd36a03@gmail.com>
 <ZaFCoJ1FxBQ9jQkX@ip-10-97-1-34.eu-west-3.compute.internal>
 <ZaSthTtVmvKKuGNb@paquier.xyz>
From: Alexander Lakhin <exclusion@gmail.com>
In-Reply-To: <ZaSthTtVmvKKuGNb@paquier.xyz>
Archived-At: 
 <https://www.postgresql.org/message-id/b641be2a-945b-999c-2828-7bbd8a16a009%40gmail.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------okLIZVff3gtGKJ1J6bSSCsVJ
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hello Michael and Bertrand,

15.01.2024 06:59, Michael Paquier wrote:
> The WAL records related to standby snapshots are playing a lot with
> the randomness of the failures we are seeing.  Alexander has mentioned
> offlist something else: using SIGSTOP on the bgwriter to avoid these
> records and make the test more stable.  That would not be workable for
> Windows, but I could live with that knowing that logical decoding for
> standbys has no platform-speficic tweak for the code paths we're
> testing here, and that would put as limitation to skip the test for
> $windows_os.

I've found a way to implement pause/resume for Windows processed and it
looks acceptable to me if we can afford "use Win32::API;" on Windows
(maybe the test could be skipped only if this perl module is absent).
Please look at the PoC patch for the test 035_standby_logical_decoding.
(The patched test passes for me.)

If this approach looks promising to you, maybe we could add a submodule to
perl/PostgreSQL/Test/ and use this functionality in other tests (e.g., in
019_replslot_limit) as well.

Personally I think that having such a functionality for using in tests
might be useful not only to avoid some "problematic" behaviour but also to
test the opposite cases.

> While thinking about that, a second idea came into my mind: a
> superuser-settable developer GUC to disable such WAL records to be
> generated within certain areas of the test.  This requires a small
> implementation, but nothing really huge, while being portable
> everywhere.  And it is not the first time I've been annoyed with these
> records when wanting a predictible set of WAL records for some test
> case.

I see that the test in question exists in REL_16_STABLE, it means that a
new GUC would not help there?

Best regards,
Alexander
--------------okLIZVff3gtGKJ1J6bSSCsVJ
Content-Type: text/x-patch; charset=UTF-8; name="035-suspend-on-Windows.patch"
Content-Disposition: attachment; filename="035-suspend-on-Windows.patch"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL3NyYy90ZXN0L3JlY292ZXJ5L3QvMDM1X3N0YW5kYnlfbG9naWNhbF9k
ZWNvZGluZy5wbCBiL3NyYy90ZXN0L3JlY292ZXJ5L3QvMDM1X3N0YW5kYnlfbG9naWNhbF9k
ZWNvZGluZy5wbAppbmRleCA4YmMzOWE1ZjAzLi44YjA4YzdiNWM3IDEwMDY0NAotLS0gYS9z
cmMvdGVzdC9yZWNvdmVyeS90LzAzNV9zdGFuZGJ5X2xvZ2ljYWxfZGVjb2RpbmcucGwKKysr
IGIvc3JjL3Rlc3QvcmVjb3ZlcnkvdC8wMzVfc3RhbmRieV9sb2dpY2FsX2RlY29kaW5nLnBs
CkBAIC0xMCw2ICsxMCw4IEBAIHVzZSBQb3N0Z3JlU1FMOjpUZXN0OjpDbHVzdGVyOwogdXNl
IFBvc3RncmVTUUw6OlRlc3Q6OlV0aWxzOwogdXNlIFRlc3Q6Ok1vcmU7CiAKK3VzZSBXaW4z
Mjo6QVBJOworCiBteSAoJHN0ZGluLCAkc3Rkb3V0LCAkc3RkZXJyLAogCSRjYXNjYWRpbmdf
c3Rkb3V0LCAkY2FzY2FkaW5nX3N0ZGVyciwgJHN1YnNjcmliZXJfc3RkaW4sCiAJJHN1YnNj
cmliZXJfc3Rkb3V0LCAkc3Vic2NyaWJlcl9zdGRlcnIsICRyZXQsCkBAIC0yOCw2ICszMCwy
OCBAQCBteSAkcmVzOwogbXkgJHByaW1hcnlfc2xvdG5hbWUgPSAncHJpbWFyeV9waHlzaWNh
bCc7CiBteSAkc3RhbmRieV9waHlzaWNhbF9zbG90bmFtZSA9ICdzdGFuZGJ5X3BoeXNpY2Fs
JzsKIAorbXkgJE9wZW5Qcm9jZXNzID0gbmV3IFdpbjMyOjpBUEkoImtlcm5lbDMyIiwgIkhB
TkRMRSBPcGVuUHJvY2VzcyhEV09SRCBkd0Rlc2lyZWRBY2Nlc3MsIEJPT0wgYkluaGVyaXRI
YW5kbGUsIERXT1JEIGR3UHJvY2Vzc0lkICkiKTsKK215ICROdFN1c3BlbmRQcm9jZXNzID0g
bmV3IFdpbjMyOjpBUEkoIm50ZGxsIiwgJ0xPTkcgTnRTdXNwZW5kUHJvY2VzcyhIQU5ETEUg
aFByb2Nlc3MpJyk7CitteSAkTnRSZXN1bWVQcm9jZXNzID0gbmV3IFdpbjMyOjpBUEkoIm50
ZGxsIiwgJ0xPTkcgTnRSZXN1bWVQcm9jZXNzKEhBTkRMRSBoUHJvY2VzcyknKTsKK215ICRD
bG9zZUhhbmRsZSA9IG5ldyBXaW4zMjo6QVBJKCJrZXJuZWwzMiIsICJCT09MIENsb3NlSGFu
ZGxlKEhBTkRMRSBoT2JqZWN0KSIpOworbXkgJFBST0NFU1NfQUxMX0FDQ0VTUyA9IDB4MDAx
RjBGRkY7CisKK3N1YiBzdXNwZW5kX3Byb2Nlc3MKK3sKKwlteSAkcGlkID0gc2hpZnQ7CisJ
bXkgJGhQcm9jZXNzID0gJE9wZW5Qcm9jZXNzLT5DYWxsKCRQUk9DRVNTX0FMTF9BQ0NFU1Ms
IDAsICRwaWQpOworCSROdFN1c3BlbmRQcm9jZXNzLT5DYWxsKCRoUHJvY2Vzcyk7CisJJENs
b3NlSGFuZGxlLT5DYWxsKCRoUHJvY2Vzcyk7Cit9CisKK3N1YiByZXN1bWVfcHJvY2Vzcwor
eworCW15ICRwaWQgPSBzaGlmdDsKKwlteSAkaFByb2Nlc3MgPSAkT3BlblByb2Nlc3MtPkNh
bGwoJFBST0NFU1NfQUxMX0FDQ0VTUywgMCwgJHBpZCk7CisJJE50UmVzdW1lUHJvY2Vzcy0+
Q2FsbCgkaFByb2Nlc3MpOworCSRDbG9zZUhhbmRsZS0+Q2FsbCgkaFByb2Nlc3MpOworfQor
CiAjIEZldGNoIHhtaW4gY29sdW1ucyBmcm9tIHNsb3QncyBwZ19yZXBsaWNhdGlvbl9zbG90
cyByb3csIGFmdGVyIHdhaXRpbmcgZm9yCiAjIGdpdmVuIGJvb2xlYW4gY29uZGl0aW9uIHRv
IGJlIHRydWUgdG8gZW5zdXJlIHdlJ3ZlIHJlYWNoZWQgYSBxdWllc2NlbnQgc3RhdGUuCiBz
dWIgd2FpdF9mb3JfeG1pbnMKQEAgLTQ1Niw2ICs0ODAsOSBAQCBpcygkcmVzdWx0LCBxcSgx
MCksICdjaGVjayByZXBsaWNhdGVkIGluc2VydHMgYWZ0ZXIgc3Vic2NyaXB0aW9uIG9uIHN0
YW5kYnknKTsKICRub2RlX3N1YnNjcmliZXItPnNhZmVfcHNxbCgncG9zdGdyZXMnLCAiRFJP
UCBTVUJTQ1JJUFRJT04gdGFwX3N1YiIpOwogJG5vZGVfc3Vic2NyaWJlci0+c3RvcDsKIAor
bXkgJGJnd3JpdGVycGlkID0gJG5vZGVfcHJpbWFyeS0+c2FmZV9wc3FsKCdwb3N0Z3Jlcycs
ICJTRUxFQ1QgcGlkIEZST00gcGdfc3RhdF9hY3Rpdml0eSBXSEVSRSBiYWNrZW5kX3R5cGUg
PSAnYmFja2dyb3VuZCB3cml0ZXInIik7CitzdXNwZW5kX3Byb2Nlc3MoJGJnd3JpdGVycGlk
KTsKKwogIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj
IyMKICMgUmVjb3ZlcnkgY29uZmxpY3Q6IEludmFsaWRhdGUgY29uZmxpY3Rpbmcgc2xvdHMs
IGluY2x1ZGluZyBpbi11c2Ugc2xvdHMKICMgU2NlbmFyaW8gMTogaG90X3N0YW5kYnlfZmVl
ZGJhY2sgb2ZmIGFuZCB2YWN1dW0gRlVMTApAQCAtNjkwLDYgKzcxNyw3IEBAICRub2RlX3By
aW1hcnktPnNhZmVfcHNxbCgndGVzdGRiJywgcXFbSU5TRVJUIElOVE8gcHJ1biBWQUxVRVMg
KDEsICdBJyk7XSk7CiAkbm9kZV9wcmltYXJ5LT5zYWZlX3BzcWwoJ3Rlc3RkYicsIHFxW1VQ
REFURSBwcnVuIFNFVCBzID0gJ0InO10pOwogJG5vZGVfcHJpbWFyeS0+c2FmZV9wc3FsKCd0
ZXN0ZGInLCBxcVtVUERBVEUgcHJ1biBTRVQgcyA9ICdDJztdKTsKICRub2RlX3ByaW1hcnkt
PnNhZmVfcHNxbCgndGVzdGRiJywgcXFbVVBEQVRFIHBydW4gU0VUIHMgPSAnRCc7XSk7Cisk
bm9kZV9wcmltYXJ5LT5zYWZlX3BzcWwoJ3Rlc3RkYicsIHFxW1NFTEVDVCBwZ19zbGVlcCgy
NSk7XSk7CiAkbm9kZV9wcmltYXJ5LT5zYWZlX3BzcWwoJ3Rlc3RkYicsIHFxW1VQREFURSBw
cnVuIFNFVCBzID0gJ0UnO10pOwogCiAkbm9kZV9wcmltYXJ5LT53YWl0X2Zvcl9yZXBsYXlf
Y2F0Y2h1cCgkbm9kZV9zdGFuZGJ5KTsKQEAgLTcwOSw2ICs3MzcsOCBAQCBjaGVja19wZ19y
ZWN2bG9naWNhbF9zdGRlcnIoJGhhbmRsZSwKICMgVHVybiBob3Rfc3RhbmRieV9mZWVkYmFj
ayBiYWNrIG9uCiBjaGFuZ2VfaG90X3N0YW5kYnlfZmVlZGJhY2tfYW5kX3dhaXRfZm9yX3ht
aW5zKDEsIDEpOwogCityZXN1bWVfcHJvY2VzcygkYmd3cml0ZXJwaWQpOworCiAjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwogIyBSZWNvdmVy
eSBjb25mbGljdDogSW52YWxpZGF0ZSBjb25mbGljdGluZyBzbG90cywgaW5jbHVkaW5nIGlu
LXVzZSBzbG90cwogIyBTY2VuYXJpbyA1OiBpbmNvcnJlY3Qgd2FsX2xldmVsIG9uIHByaW1h
cnkuCg==

--------------okLIZVff3gtGKJ1J6bSSCsVJ--