Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wFr7M-005dOx-2S for pgsql-hackers@arkaria.postgresql.org; Thu, 23 Apr 2026 10:17:10 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wFr7L-001FI9-0Y for pgsql-hackers@arkaria.postgresql.org; Thu, 23 Apr 2026 10:17:07 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wFr7K-001FI0-0j for pgsql-hackers@lists.postgresql.org; Thu, 23 Apr 2026 10:17:06 +0000 Received: from smtp.outgoing.loopia.se ([93.188.3.37]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wFr7E-00000002OpS-06Ym for pgsql-hackers@lists.postgresql.org; Thu, 23 Apr 2026 10:17:05 +0000 Received: from s807.loopia.se (localhost [127.0.0.1]) by s807.loopia.se (Postfix) with ESMTP id BF91C590F68 for ; Thu, 23 Apr 2026 12:16:56 +0200 (CEST) Received: from s980.loopia.se (unknown [172.22.191.5]) by s807.loopia.se (Postfix) with ESMTP id A90985907EC; Thu, 23 Apr 2026 12:16:56 +0200 (CEST) Received: from localhost (unknown [172.22.191.5]) by s980.loopia.se (Postfix) with ESMTP id A3B512201605; Thu, 23 Apr 2026 12:16:56 +0200 (CEST) X-Virus-Scanned: amavis at amavis.loopia.se X-Spam-Flag: NO X-Spam-Score: -1.2 X-Spam-Level: X-Spam-Status: No, score=-1.2 tagged_above=-999 required=6.2 tests=[ALL_TRUSTED=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1] autolearn=disabled Authentication-Results: s471.loopia.se (amavis); dkim=pass (2048-bit key) header.d=yesql.se Received: from s979.loopia.se ([172.22.191.5]) by localhost (s471.loopia.se [172.22.190.35]) (amavis, port 10024) with UTF8LMTP id DAJmHh52lBps; Thu, 23 Apr 2026 12:16:55 +0200 (CEST) X-Loopia-Auth: user X-Loopia-User: daniel@yesql.se X-Loopia-Originating-IP: 89.255.232.236 Received: from smtpclient.apple (customer-89-255-232-236.stosn.net [89.255.232.236]) (Authenticated sender: daniel@yesql.se) by s979.loopia.se (Postfix) with ESMTPSA id 0EE5F10BC40E; Thu, 23 Apr 2026 12:16:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yesql.se; s=loopiadkim1707475645; t=1776939415; bh=+9OXWeblm7352ruhVyhFhbUeuo8P6Vf7tpmOhzUvU4w=; h=From:Subject:Date:In-Reply-To:Cc:To:References; b=qk8hh3eA1lADn1dqjm+7zkO5WnwMJjWgkKWL5I2WyDW5/em0XkHmnNw7VqoiKz1Vi VK2WcuwESd7pTR7LCRY6k8nIwDjJlvLEBVye3WyNt0rLVtUopszCXvVlsNmU4B0CTo SO6/BH+m7Tnu/VJb0Z/2UdQ5XJ5oMcwAFUfeOPJRuAqksVIrvMxw3O/kCpBgCVeTIB in65qlbikl3bHIAX8vtBot8lGr3BSpA8zV5MHHaE7Zva4xktWUhM7r2jpu8/7Fu2DB 6SDUyYSH9RjRlVOVZ9BMsz+6cU/sDquMNZnrZq+TqERhixfHKbjChBgKC4oRUA6P81 qqRkzNmzQ48kA== From: Daniel Gustafsson Message-Id: <9197F930-DDEB-4CAC-82A2-16FEC715CCE8@yesql.se> Content-Type: multipart/mixed; boundary="Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.2\)) Subject: Re: Changing the state of data checksums in a running cluster Date: Thu, 23 Apr 2026 12:16:44 +0200 In-Reply-To: <17BD06EB-3741-4921-AF60-6575FB4784ED@yesql.se> Cc: Heikki Linnakangas , Tomas Vondra , Andres Freund , Bernd Helmle , Michael Paquier , Michael Banck , SATYANARAYANA NARLAPURAM To: PostgreSQL Hackers References: <7357A519-3391-4F07-BC69-5C9C2B120792@yesql.se> <477897AE-1314-4724-9694-0BABC4F4ABDA@yesql.se> <5mv5vtbqj2xm2wmevsi22smyn32jtznva3ya3daih3ixh3onex@s5fyab63xt3c> <2535E72D-5979-4F58-A2D3-2AA4AA335C20@yesql.se> <775fddbc-4dec-413d-89bb-1a8186858ee6@vondra.me> <683618b7-0946-400e-8066-97067f32d448@vondra.me> <032619A2-D466-4A12-A524-98359D96AEA6@yesql.se> <300822bd-daeb-49e5-8af7-d7ce3af65adc@iki.fi> <46d52594-fa93-4fbe-ad97-dc3127a04dd8@iki.fi> <8B86AFB8-ACBC-44FC-A1AB-A58DB541A170@yesql.se> <452D5390-4F89-4DED-9835-8E9EA2A01E19@yesql.se> <4020B0AE-6286-462F-BD50-3AAD170F08C9@yesql.se> <30E2C103-2F50-4F9D-8868-E057250F55CA@yesql.se> <17BD06EB-3741-4921-AF60-6575FB4784ED@yesql.se> X-Mailer: Apple Mail (2.3776.700.51.11.2) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii While performing extensive post-commit testing using the built-in TAP = tests in checksum_extended mode, Tomas observed a couple issues. The testing was = done on multiple machines, ranging from a fast x86 machine to a much slower = rpi4, but they all performed a very large number of iterations. The = combination of hardware and the large number of tests executed is likely why some of = the issues went unnoticed. Below are the issues described in more detail: 1) race condition between checkpoint and checksum state changes The main issue is a race condition causing invalid state transitions. When analyzing the failure (which we couldn't synthetically reproduce, = only observe by running tests over and over for a long time) we realized that = the original coding was performing checksum state transitions while = replaying online checkpoints as well as redo checkpoints. Updating checksum state while replaying online checkpoints is incorrect. = After a crash we'll start with the REDO record, which initializes checksums to = the right state. And then later the checksum state is updated by the = regular XLOG_CHECKSUMS entries. Moreover, the checksum state in the XLOG_CHECKPOINT_ONLINE record can be = stale, because the value is determined at the checkpoint start, but the WAL = entry is added at the end. If there is a concurrent checksum change, the = value written to the WAL record will be stale. Replaying it will cause an = "invalid transition" error later, during the next checksum state update. The fix = is to remove the checksum update from the online checkpoint altogether. In fact, there's no need for CreateCheckPoint to update checksum state in the ControlFile. If the value is stale, it could make it permanent. But it's unnecessary - the ControlFile is updated by the process performing the checksum state change. So remove that. This, along with an re-ordering updating the controlfile and procsignal = barrier enission fixes the issue. The re-ordering makes sure that the = controlfile is always updated *before* a procsignalbarrier is emitted to avoid a race = like the one described below: 1. A barrier for off to inprogress-on is emitted 2. All active backends absorbs the barrier - All processes in the cluster are in state inprogress-on 3. A new backend b' forks, reads controlfile and sets state of off 4. The controlfile is updated 5. A new backend b'' forks, reads controlfile and sets state to = inprogress-on b' and b'' have different states, and b' has an incompatible state with = the rest of the cluster. Re-ordering as done in the attached makes this go = away. 2) race condition in launcher exit Another timing related issue was that reverting to the "off" state then launcher errors out had synchronization logic which was racy as it was = relying on the cached checksum state and not the value from XLogCtl. The logic = for determining if a launcher/worker was already active was also fragile as = it started another launcher which would overwrite certain data in shared = memory. The attached patch inspects shared memory instead and use that to signal = the running launcher to either abort (disable), or change cost parameters on = a running enable process. These fixes makes erroring out and going to off = state stable. 3) Concurrency issue with ProcSignalInit / InitLocalDataChecksumState The checksum barriers must not be consumed before the initial value gets properly set. On very slow systems, there could potentially be multiple checksum state transitions between a fork and = InitLocalDataChecksumState. In such cases we might get failures due to incorrect transitions. With the current code this is not a live issue, as there is no place = checking interrupts in between the two functions. But it's fragile, as it's = trivial to break this by adding an elog() somewhere. Which is what happened to us = while debugging the other issues. So better to explicitly hold interrupts for = a brief moment. To find the issues, and to validate their fix, Tomas developed a new = testsuite which is attached as a .txt. This is not proposed for adding to v19, it = is included to showcase what was done, and what will be further hacked on = for a new suite during the v20 cycle. It is gated behind PG_TEST_EXTRA and is intended to be executed by select members of the buildfarm. As part of this postcommit review we also found a few more cleanups and = smaller fixes which are included. The patchset also includes the patch = submitted upthread by Satyanarayana Narlapuram. More details can be found in the individual commit messages. Unless there are objections I would like to go ahead with this fixup = fairly soon. -- Daniel Gustafsson --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0008-Fix-data_checksum-GUC-show_hook.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0008-Fix-data_checksum-GUC-show_hook.patch" Content-Transfer-Encoding: quoted-printable =46rom=200a2b961f90427937796b5bca549b04e20958cd0f=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:47=20= +0200=0ASubject:=20[PATCH=208/8]=20Fix=20data_checksum=20GUC=20show_hook=0A= =0ACommit=20f19c0eccae=20erroneously=20omitted=20the=20show_hook=20for=20= the=0Adata_checksum=20GUC.=0A=0AAuthor:=20Daniel=20Gustafsson=20= =0ADiscussion:=20https://postgr.es/m/xxx=0A---=0A=20= src/backend/utils/misc/guc_parameters.dat=20|=201=20+=0A=201=20file=20= changed,=201=20insertion(+)=0A=0Adiff=20--git=20= a/src/backend/utils/misc/guc_parameters.dat=20= b/src/backend/utils/misc/guc_parameters.dat=0Aindex=20= 83af594d4af..afaa058b046=20100644=0A---=20= a/src/backend/utils/misc/guc_parameters.dat=0A+++=20= b/src/backend/utils/misc/guc_parameters.dat=0A@@=20-585,6=20+585,7=20@@=0A= =20=20=20variable=20=3D>=20'data_checksums',=0A=20=20=20boot_val=20=3D>=20= 'PG_DATA_CHECKSUM_OFF',=0A=20=20=20options=20=3D>=20= 'data_checksums_options',=0A+=20=20show_hook=20=3D>=20= 'show_data_checksums',=0A=20},=0A=20=0A=20#=20Can't=20be=20set=20by=20= ALTER=20SYSTEM=20as=20it=20can=20lead=20to=20recursive=20definition=0A--=20= =0A2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0007-Improve-database-detection-logic-in-datachecksumswor.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0007-Improve-database-detection-logic-in-datachecksumswor.patch" Content-Transfer-Encoding: quoted-printable =46rom=203e0fbc07bade81a423b668d975111bb02067442f=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:40=20= +0200=0ASubject:=20[PATCH=207/8]=20Improve=20database=20detection=20= logic=20in=20datachecksumsworker=0A=0AThe=20worker=20need=20to=20know=20= whether=20a=20database=20which=20failed=20checksum=0Aprocessing=20still=20= exists,=20or=20has=20been=20dropped.=20=20This=20improves=20the=0A= detection=20logic=20by=20checking=20for=20being=20partially=20dropped.=0A= =0AAuthor:=20Daniel=20Gustafsson=20=0ADiscussion:=20= https://postgr.es/m/xxx=0A---=0A=20= src/backend/postmaster/datachecksum_state.c=20|=2016=20+++++++++++++---=0A= =201=20file=20changed,=2013=20insertions(+),=203=20deletions(-)=0A=0A= diff=20--git=20a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Aindex=20= c382b954d91..53b66cbbcf6=20100644=0A---=20= a/src/backend/postmaster/datachecksum_state.c=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-843,8=20+843,7=20= @@=20ProcessDatabase(DataChecksumsWorkerDatabase=20*db)=0A=20=0A=20=09=09= /*=0A=20=09=09=20*=20Heuristic=20to=20see=20if=20the=20database=20was=20= dropped,=20and=20if=20it=20was=20we=20can=0A-=09=09=20*=20treat=20it=20= as=20not=20an=20error,=20else=20treat=20as=20fatal=20and=20error=20out.=20= TODO:=0A-=09=09=20*=20this=20could=20probably=20be=20improved=20with=20a=20= tighter=20check.=0A+=09=09=20*=20treat=20it=20as=20not=20an=20error,=20= else=20treat=20as=20fatal=20and=20error=20out.=0A=20=09=09=20*/=0A=20=09=09= if=20(DatabaseExists(db->dboid))=0A=20=09=09=09return=20= DATACHECKSUMSWORKER_FAILED;=0A@@=20-1305,7=20+1304,9=20@@=20= DataChecksumsShmemRequest(void=20*arg)=0A=20=20*=20DatabaseExists=0A=20=20= *=0A=20=20*=20Scans=20the=20system=20catalog=20to=20check=20if=20a=20= database=20with=20the=20given=20Oid=20exists=0A-=20*=20and=20returns=20= true=20if=20it=20is=20found,=20else=20false.=0A+=20*=20and=20returns=20= true=20if=20it=20is=20found=20and=20valid,=20else=20false.=20Note,=20we=20= cannot=20use=0A+=20*=20database_is_invalid_oid=20here=20as=20it=20will=20= ERROR=20out,=20and=20we=20want=20to=20gracefully=0A+=20*=20handle=20= errors.=0A=20=20*/=0A=20static=20bool=0A=20DatabaseExists(Oid=20dboid)=0A= @@=20-1315,6=20+1316,7=20@@=20DatabaseExists(Oid=20dboid)=0A=20=09= SysScanDesc=20scan;=0A=20=09bool=09=09found;=0A=20=09HeapTuple=09tuple;=0A= +=09Form_pg_database=20pg_database_tuple;=0A=20=0A=20=09= StartTransactionCommand();=0A=20=0A@@=20-1328,6=20+1330,14=20@@=20= DatabaseExists(Oid=20dboid)=0A=20=09tuple=20=3D=20= systable_getnext(scan);=0A=20=09found=20=3D=20HeapTupleIsValid(tuple);=0A= =20=0A+=09/*=20If=20the=20Oid=20exists,=20ensure=20that=20it's=20not=20= partially=20dropped=20*/=0A+=09if=20(found)=0A+=09{=0A+=09=09= pg_database_tuple=20=3D=20(Form_pg_database)=20GETSTRUCT(tuple);=0A+=09=09= if=20(database_is_invalid_form(pg_database_tuple))=0A+=09=09=09found=20=3D= =20false;=0A+=09}=0A+=0A=20=09systable_endscan(scan);=0A=20=09= table_close(rel,=20AccessShareLock);=0A=20=0A--=20=0A2.39.3=20(Apple=20= Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0006-Improve-handling-of-concurrent-checksum-requests.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0006-Improve-handling-of-concurrent-checksum-requests.patch" Content-Transfer-Encoding: quoted-printable =46rom=207517ce4a6c33be0f75780ade699afc1e807cf997=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:32=20= +0200=0ASubject:=20[PATCH=206/8]=20Improve=20handling=20of=20concurrent=20= checksum=20requests=0A=0AWhen=20pg_{enable|disable}_data_checksums=20is=20= called=20while=20checksums=20are=0Abeing=20enabled=20of=20disabled,=20= the=20already=20running=20launcher=20is=20detected=0Aand=20the=20new=20= desired=20state=20is=20recorded.=20=20Processing=20will=20then=20pick=20= up=0Athe=20new=20state=20and=20change=20it's=20operation=20to=20fulfill=20= the=20new=20request.=0A=0AIf=20the=20same=20state=20is=20requested=20but=20= updated=20cost=20values,=20the=20new=20cost=0Avalues=20will=20take=20= effect=20on=20the=20next=20relation=20processed.=20=20The=20previous=0A= coding=20had=20a=20complex=20logic=20of=20starting=20a=20new=20launcher=20= for=20this,=20which=0Ass=20now=20avoided=20with=20the=20shared=20mem=20= structure=20instead=20used=20to=20signal=0Acurrent=20processing.=0A=0A= This=20makes=20the=20logic=20more=20robust,=20and=20fixes=20a=20bug=20= where=20the=20launcher=0Awould=20erroneously=20revert=20back=20to=20the=20= "off"=20state.=0A=0AAuthor:=20Daniel=20Gustafsson=20=0A= Discussion:=20https://postgr.es/m/xxx=0A---=0A=20= src/backend/access/transam/xlog.c=20=20=20=20=20=20=20=20=20=20=20|=2033=20= +++++++-=0A=20src/backend/postmaster/datachecksum_state.c=20|=2085=20= +++++++++++++++------=0A=20src/include/access/xlog.h=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20|=20=202=20+=0A=203=20files=20= changed,=2096=20insertions(+),=2024=20deletions(-)=0A=0Adiff=20--git=20= a/src/backend/access/transam/xlog.c=20= b/src/backend/access/transam/xlog.c=0Aindex=209b1a10dbd1c..49772510855=20= 100644=0A---=20a/src/backend/access/transam/xlog.c=0A+++=20= b/src/backend/access/transam/xlog.c=0A@@=20-4682,10=20+4682,41=20@@=20= DataChecksumsNeedWrite(void)=0A=20=09=09=09LocalDataChecksumState=20=3D=3D= =20PG_DATA_CHECKSUM_INPROGRESS_OFF);=0A=20}=0A=20=0A+=0A+bool=0A= +DataChecksumsOff(void)=0A+{=0A+=09bool=09=09ret;=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09ret=20=3D=20= (XLogCtl->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_OFF);=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09return=20ret;=0A+}=0A+=0A= +bool=0A+DataChecksumsOn(void)=0A+{=0A+=09bool=09=09ret;=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09ret=20=3D=20= (XLogCtl->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_VERSION);=0A= +=09SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09return=20ret;=0A+}=0A= +=0A=20bool=0A=20DataChecksumsInProgressOn(void)=0A=20{=0A-=09return=20= LocalDataChecksumState=20=3D=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON;=0A+=09= bool=09=09ret;=0A+=0A+=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09ret=20= =3D=20(XLogCtl->data_checksum_version=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON);=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09return=20ret;=0A=20}=0A=20= =0A=20/*=0Adiff=20--git=20a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Aindex=20= 5e14803b0b0..c382b954d91=20100644=0A---=20= a/src/backend/postmaster/datachecksum_state.c=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-235,7=20+235,7=20= @@=20typedef=20struct=20ChecksumBarrierCondition=0A=20=09int=09=09=09to;=0A= =20}=20ChecksumBarrierCondition;=0A=20=0A-static=20const=20= ChecksumBarrierCondition=20checksum_barriers[7]=20=3D=0A+static=20const=20= ChecksumBarrierCondition=20checksum_barriers[9]=20=3D=0A=20{=0A=20=09/*=0A= =20=09=20*=20Disabling=20checksums:=20If=20checksums=20are=20currently=20= enabled,=20disabling=20must=0A@@=20-267,6=20+267,13=20@@=20static=20= const=20ChecksumBarrierCondition=20checksum_barriers[7]=20=3D=0A=20=09=20= *=20set=20to=20off=20since=20we=20cannot=20reach=20on=20at=20that=20= point.=0A=20=09=20*/=0A=20=09{PG_DATA_CHECKSUM_INPROGRESS_ON,=20= PG_DATA_CHECKSUM_INPROGRESS_OFF},=0A+=0A+=09/*=0A+=09=20*=20Transitions=20= that=20can=20happen=20when=20a=20new=20request=20is=20made=20while=20= another=20is=0A+=09=20*=20currently=20being=20processed.=0A+=09=20*/=0A+=09= {PG_DATA_CHECKSUM_INPROGRESS_OFF,=20PG_DATA_CHECKSUM_INPROGRESS_ON},=0A+=09= {PG_DATA_CHECKSUM_OFF,=20PG_DATA_CHECKSUM_INPROGRESS_OFF},=0A=20};=0A=20=0A= =20/*=0A@@=20-368,6=20+375,15=20@@=20const=20ShmemCallbacks=20= DataChecksumsShmemCallbacks=20=3D=20{=0A=20=09.request_fn=20=3D=20= DataChecksumsShmemRequest,=0A=20};=0A=20=0A+#define=20= CHECK_FOR_ABORT_REQUEST()=20\=0A+=09do=20{=09=09=09=09=09=09=09=09=09=09=09= =09=09=09=09\=0A+=09=09LWLockAcquire(DataChecksumsWorkerLock,=20= LW_SHARED);=09=09=09\=0A+=09=09if=20(DataChecksumState->launch_operation=20= !=3D=20operation)=09=09\=0A+=09=09=09abort_requested=20=3D=20true;=09=09=09= =09=09=09=09=09=09\=0A+=09=09LWLockRelease(DataChecksumsWorkerLock);=09=09= =09=09=09=09\=0A+=09}=20while=20(0)=0A+=0A+=0A=20= /*************************************************************************= ****=0A=20=20*=20Functionality=20for=20manipulating=20the=20data=20= checksum=20state=20in=20the=20cluster=0A=20=20*/=0A@@=20-557,7=20+573,6=20= @@=20StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A= =20=09BackgroundWorker=20bgw;=0A=20=09BackgroundWorkerHandle=20= *bgw_handle;=0A=20=09bool=09=09running;=0A-=09= DataChecksumsWorkerOperation=20launcher_running_op;=0A=20=0A=20#ifdef=20= USE_ASSERT_CHECKING=0A=20=09/*=20The=20cost=20delay=20settings=20have=20= no=20effect=20when=20disabling=20*/=0A@@=20-576,8=20+591,6=20@@=20= StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A=20= =0A=20=09/*=20Is=20the=20launcher=20already=20running?=20If=20so,=20what=20= is=20it=20doing?=20*/=0A=20=09running=20=3D=20= DataChecksumState->launcher_running;=0A-=09if=20(running)=0A-=09=09= launcher_running_op=20=3D=20DataChecksumState->operation;=0A=20=0A=20=09= LWLockRelease(DataChecksumsWorkerLock);=0A=20=0A@@=20-594,13=20+607,17=20= @@=20StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A= =20=09=20*=20the=20launcher=20has=20had=20a=20chance=20to=20start=20up,=20= we=20still=20end=20up=20launching=20it=0A=20=09=20*=20twice.=20=20That's=20= OK,=20the=20second=20invocation=20will=20see=20that=20a=20launcher=20is=0A= =20=09=20*=20already=20running=20and=20exit=20quickly.=0A-=09=20*=0A-=09=20= *=20TODO:=20We=20could=20optimize=20here=20and=20skip=20launching=20the=20= launcher,=20if=20we=20are=0A-=09=20*=20already=20in=20the=20desired=20= state,=20i.e.=20if=20the=20checksums=20are=20already=20enabled=0A-=09=20= *=20and=20you=20call=20pg_enable_data_checksums().=0A=20=09=20*/=0A=20=09= if=20(!running)=0A=20=09{=0A+=09=09if=20((op=20=3D=3D=20= ENABLE_DATACHECKSUMS=20&&=20DataChecksumsOn())=20||=0A+=09=09=09(op=20=3D=3D= =20DISABLE_DATACHECKSUMS=20&&=20DataChecksumsOff()))=0A+=09=09{=0A+=09=09= =09ereport(LOG,=0A+=09=09=09=09=09errmsg("data=20checksums=20already=20= in=20desired=20state,=20exiting"));=0A+=09=09=09return;=0A+=09=09}=0A+=0A= =20=09=09/*=0A=20=09=09=20*=20Prepare=20the=20BackgroundWorker=20and=20= launch=20it.=0A=20=09=09=20*/=0A@@=20-622,9=20+639,8=20@@=20= StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A=20= =09}=0A=20=09else=0A=20=09{=0A-=09=09if=20(launcher_running_op=20=3D=3D=20= op)=0A-=09=09=09ereport(ERROR,=0A-=09=09=09=09=09errmsg("data=20checksum=20= processing=20already=20running"));=0A+=09=09ereport(LOG,=0A+=09=09=09=09= errmsg("data=20checksum=20processing=20already=20running"));=0A=20=09}=0A= =20}=0A=20=0A@@=20-998,11=20+1014,8=20@@=20= WaitForAllTransactionsToFinish(void)=0A=20=09=09=09=09=09errhint("Data=20= checksums=20processing=20must=20be=20restarted=20manually=20after=20= cluster=20restart."));=0A=20=0A=20=09=09CHECK_FOR_INTERRUPTS();=0A+=09=09= CHECK_FOR_ABORT_REQUEST();=0A=20=0A-=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_SHARED);=0A-=09=09if=20= (DataChecksumState->launch_operation=20!=3D=20operation)=0A-=09=09=09= abort_requested=20=3D=20true;=0A-=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A=20=09=09if=20= (abort_requested)=0A=20=09=09=09break;=0A=20=09}=0A@@=20-1185,7=20= +1198,9=20@@=20ProcessAllDatabases(void)=0A=20=09int=09=09=09= cumulative_total=20=3D=200;=0A=20=0A=20=09/*=20Set=20up=20so=20first=20= run=20processes=20shared=20catalogs,=20not=20once=20in=20every=20db=20*/=0A= +=09LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A=20=09= DataChecksumState->process_shared_catalogs=20=3D=20true;=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A=20=0A=20=09/*=20Get=20a=20= list=20of=20all=20databases=20to=20process=20*/=0A=20=09= WaitForAllTransactionsToFinish();=0A@@=20-1261,7=20+1276,9=20@@=20= ProcessAllDatabases(void)=0A=20=09=09=20*=20When=20one=20database=20has=20= completed,=20it=20will=20have=20done=20shared=20catalogs=0A=20=09=09=20*=20= so=20we=20don't=20have=20to=20process=20them=20again.=0A=20=09=09=20*/=0A= +=09=09LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A=20=09=09= DataChecksumState->process_shared_catalogs=20=3D=20false;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A=20=09}=0A=20=0A=20=09= FreeDatabaseList(DatabaseList);=0A@@=20-1503,7=20+1520,6=20@@=20= DataChecksumsWorkerMain(Datum=20arg)=0A=20=09=20*=20implementation=20= detail=20and=20care=20should=20be=20taken=20to=20avoid=20it=20bleeding=0A= =20=09=20*=20through=20to=20the=20user=20to=20avoid=20confusion.=0A=20=09= =20*/=0A-=09Assert(DataChecksumState->operation=20=3D=3D=20= ENABLE_DATACHECKSUMS);=0A=20=09VacuumCostDelay=20=3D=20= DataChecksumState->cost_delay;=0A=20=09VacuumCostLimit=20=3D=20= DataChecksumState->cost_limit;=0A=20=09VacuumCostActive=20=3D=20= (VacuumCostDelay=20>=200);=0A@@=20-1539,8=20+1555,6=20@@=20= DataChecksumsWorkerMain(Datum=20arg)=0A=20=09rels_done=20=3D=200;=0A=20=09= foreach_oid(reloid,=20RelationList)=0A=20=09{=0A-=09=09= CHECK_FOR_INTERRUPTS();=0A-=0A=20=09=09if=20= (!ProcessSingleRelationByOid(reloid,=20strategy))=0A=20=09=09{=0A=20=09=09= =09aborted=20=3D=20true;=0A@@=20-1549,12=20+1563,38=20@@=20= DataChecksumsWorkerMain(Datum=20arg)=0A=20=0A=20=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_RELS_DONE,=0A=20=09=09= =09=09=09=09=09=09=09=20++rels_done);=0A+=09=09CHECK_FOR_INTERRUPTS();=0A= +=09=09CHECK_FOR_ABORT_REQUEST();=0A+=0A+=09=09if=20(abort_requested)=0A= +=09=09=09break;=0A+=0A+=09=09/*=20Check=20if=20the=20cost=20settings=20= changed=20during=20runtime=20*/=0A+=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09if=20= ((DataChecksumState->launch_cost_delay=20!=3D=20= DataChecksumState->cost_delay)=0A+=09=09=09||=20= (DataChecksumState->launch_cost_limit=20!=3D=20= DataChecksumState->cost_limit))=0A+=09=09{=0A+=09=09=09VacuumCostDelay=20= =3D=20DataChecksumState->launch_cost_delay;=0A+=09=09=09VacuumCostLimit=20= =3D=20DataChecksumState->launch_cost_limit;=0A+=09=09=09VacuumCostActive=20= =3D=20(VacuumCostDelay=20>=200);=0A+=0A+=09=09=09= FreeAccessStrategy(strategy);=0A+=09=09=09strategy=20=3D=20= GetAccessStrategy(BAS_VACUUM);=0A+=09=09=09DataChecksumState->cost_delay=20= =3D=20DataChecksumState->launch_cost_delay;=0A+=09=09=09= DataChecksumState->cost_limit=20=3D=20= DataChecksumState->launch_cost_limit;=0A+=09=09}=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A=20=09}=0A+=0A=20=09= list_free(RelationList);=0A+=09FreeAccessStrategy(strategy);=0A=20=0A-=09= if=20(aborted)=0A+=09if=20(aborted=20||=20abort_requested)=0A=20=09{=0A+=09= =09LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A=20=09=09= DataChecksumState->success=20=3D=20DATACHECKSUMSWORKER_ABORTED;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A=20=09=09ereport(DEBUG1,=0A=20=09= =09=09=09errmsg("data=20checksum=20processing=20aborted=20in=20database=20= OID=20%u",=0A=20=09=09=09=09=09=20=20=20dboid));=0A@@=20-1619,15=20= +1659,14=20@@=20DataChecksumsWorkerMain(Datum=20arg)=0A=20=09=09=09=09=09= =09=203000,=0A=20=09=09=09=09=09=09=20= WAIT_EVENT_CHECKSUM_ENABLE_TEMPTABLE_WAIT);=0A=20=0A-=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A-=09=09aborted=20= =3D=20DataChecksumState->launch_operation=20!=3D=20operation;=0A-=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A-=0A=20=09=09= CHECK_FOR_INTERRUPTS();=0A+=09=09CHECK_FOR_ABORT_REQUEST();=0A=20=0A=20=09= =09if=20(aborted=20||=20abort_requested)=0A=20=09=09{=0A+=09=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A=20=09=09=09= DataChecksumState->success=20=3D=20DATACHECKSUMSWORKER_ABORTED;=0A+=09=09= =09LWLockRelease(DataChecksumsWorkerLock);=0A=20=09=09=09ereport(LOG,=0A=20= =09=09=09=09=09errmsg("data=20checksum=20processing=20aborted=20in=20= database=20OID=20%u",=0A=20=09=09=09=09=09=09=20=20=20dboid));=0Adiff=20= --git=20a/src/include/access/xlog.h=20b/src/include/access/xlog.h=0A= index=20437b4f32349..4dd98624204=20100644=0A---=20= a/src/include/access/xlog.h=0A+++=20b/src/include/access/xlog.h=0A@@=20= -249,6=20+249,8=20@@=20extern=20uint64=20GetSystemIdentifier(void);=0A=20= extern=20char=20*GetMockAuthenticationNonce(void);=0A=20extern=20bool=20= DataChecksumsNeedWrite(void);=0A=20extern=20bool=20= DataChecksumsNeedVerify(void);=0A+extern=20bool=20DataChecksumsOn(void);=0A= +extern=20bool=20DataChecksumsOff(void);=0A=20extern=20bool=20= DataChecksumsInProgressOn(void);=0A=20extern=20void=20= SetDataChecksumsOnInProgress(void);=0A=20extern=20void=20= SetDataChecksumsOn(void);=0A--=20=0A2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0005-Typo-and-spelling-fixups-for-online-checksums.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0005-Typo-and-spelling-fixups-for-online-checksums.patch" Content-Transfer-Encoding: quoted-printable =46rom=20ae0f05e8134cde61895b37918f0e968ae0ef1e32=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:28=20= +0200=0ASubject:=20[PATCH=205/8]=20Typo=20and=20spelling=20fixups=20for=20= online=20checksums=0A=0AA=20collection=20of=20spelling,=20wording=20and=20= punctuation=20fixups=20for=20the=20code=0Adocumentation=20from=20= postcommit=20review.=0A=0AAuthor:=20Daniel=20Gustafsson=20= =0ADiscussion:=20https://postgr.es/m/xxx=0A---=0A=20= src/backend/postmaster/datachecksum_state.c=20|=2010=20+++++-----=0A=201=20= file=20changed,=205=20insertions(+),=205=20deletions(-)=0A=0Adiff=20= --git=20a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Aindex=20= 6d66322c625..5e14803b0b0=20100644=0A---=20= a/src/backend/postmaster/datachecksum_state.c=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-86,7=20+86,7=20@@=0A= =20=20*=20failing=20to=20validate=20the=20data=20checksum=20on=20the=20= page=20when=20reading=20it.=0A=20=20*=0A=20=20*=20When=20processing=20= starts=20all=20backends=20belong=20to=20one=20of=20the=20below=20sets,=20= with=0A-=20*=20one=20if=20Bd=20and=20Bi=20being=20empty:=0A+=20*=20one=20= of=20Bd=20and=20Bi=20being=20empty:=0A=20=20*=0A=20=20*=20Bg:=20Backend=20= updating=20the=20global=20state=20and=20emitting=20the=20= procsignalbarrier=0A=20=20*=20Bd:=20Backends=20in=20"off"=20state=0A@@=20= -286,7=20+286,7=20@@=20typedef=20struct=20DataChecksumsStateStruct=0A=20=09= int=09=09=09launch_cost_limit;=0A=20=0A=20=09/*=0A-=09=20*=20Is=20a=20= launcher=20process=20is=20currently=20running?=20=20This=20is=20set=20by=20= the=20main=0A+=09=20*=20Is=20a=20launcher=20process=20currently=20= running?=20=20This=20is=20set=20by=20the=20main=0A=20=09=20*=20launcher=20= process,=20after=20it=20has=20read=20the=20above=20launch_*=20= parameters.=0A=20=09=20*/=0A=20=09bool=09=09launcher_running;=0A@@=20= -809,7=20+809,7=20@@=20ProcessDatabase(DataChecksumsWorkerDatabase=20= *db)=0A=20=09{=0A=20=09=09/*=0A=20=09=09=20*=20If=20the=20worker=20= managed=20to=20start,=20and=20stop,=20before=20we=20got=20to=20waiting=0A= -=09=09=20*=20for=20it=20we=20can=20se=20a=20STOPPED=20status=20here=20= without=20it=20being=20a=20failure.=0A+=09=09=20*=20for=20it=20we=20can=20= see=20a=20STOPPED=20status=20here=20without=20it=20being=20a=20failure.=0A= =20=09=09=20*/=0A=20=09=09if=20(DataChecksumState->success=20=3D=3D=20= DATACHECKSUMSWORKER_SUCCESSFUL)=0A=20=09=09{=0A@@=20-1288,7=20+1288,7=20= @@=20DataChecksumsShmemRequest(void=20*arg)=0A=20=20*=20DatabaseExists=0A= =20=20*=0A=20=20*=20Scans=20the=20system=20catalog=20to=20check=20if=20a=20= database=20with=20the=20given=20Oid=20exists=0A-=20*=20and=20returns=20= true=20if=20it=20is=20found=20=20else=20false.=0A+=20*=20and=20returns=20= true=20if=20it=20is=20found,=20else=20false.=0A=20=20*/=0A=20static=20= bool=0A=20DatabaseExists(Oid=20dboid)=0A@@=20-1451,7=20+1451,7=20@@=20= BuildRelationList(bool=20temp_relations,=20bool=20include_shared)=0A=20= /*=0A=20=20*=20DataChecksumsWorkerMain=0A=20=20*=0A-=20*=20Main=20= function=20for=20enabling=20checksums=20in=20a=20single=20database,=20= This=20is=20the=0A+=20*=20Main=20function=20for=20enabling=20checksums=20= in=20a=20single=20database.=20This=20is=20the=0A=20=20*=20function=20set=20= as=20the=20bgw_function_name=20in=20the=20dynamic=20background=20worker=0A= =20=20*=20process=20initiated=20for=20each=20database=20by=20the=20= worker=20launcher.=20After=20enabling=0A=20=20*=20data=20checksums=20in=20= each=20applicable=20relation=20in=20the=20database,=20it=20will=20wait=20= for=0A--=20=0A2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0004-Fix-invalid-checksum-state-transition-in-checkpoints.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0004-Fix-invalid-checksum-state-transition-in-checkpoints.patch" Content-Transfer-Encoding: quoted-printable =46rom=20cf5de7abe443acc311f4880499be520f00a7bf5e=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:22=20= +0200=0ASubject:=20[PATCH=204/8]=20Fix=20invalid=20checksum=20state=20= transition=20in=20checkpoints=0A=0ACommit=2078e950cb8=20added=20checksum=20= state=20handling=20to=20all=20XLOG_CHECKPOINT=0Arecords=20which=20caused=20= unnecessary=20state=20transitions=20and=20emission=20of=0Aprocsignal=20= barriers.=20=20Remove=20as=20only=20the=20_REDO=20record=20need=20to=20= handle=0Achecksum=20state.=20=20Barrier=20emission=20is=20also=20= consistently=20made=20after=0Acontrolfile=20updates=20to=20avoid=20race=20= conditions.=0A=0AAdditionelly,=20interrupts=20are=20held=20between=20= calling=20ProcSignalInit=20and=0AInitLocalDataChecksumState=20to=20= remove=20a=20window=20where=20otherwise=20invalid=0Astate=20transitions=20= can=20happen.=0A=0AAlso=20remove=20a=20pointless=20assertion=20on=20= Controlfile=20which=20will=20never=20hit.=0A=0AAuthor:=20Tomas=20Vondra=20= =0AAuthor:=20Daniel=20Gustafsson=20=0A= Discussion:=20https://postgr.es/m/xxx=0A---=0A=20= src/backend/access/transam/xlog.c=20=20=20=20=20=20=20=20=20=20=20|=20= 106=20++++++--------------=0A=20src/backend/postmaster/auxprocess.c=20=20= =20=20=20=20=20=20=20|=20=2010=20++=0A=20= src/backend/postmaster/datachecksum_state.c=20|=20=20=208=20+-=0A=20= src/backend/utils/init/postinit.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= 10=20++=0A=204=20files=20changed,=2055=20insertions(+),=2079=20= deletions(-)=0A=0Adiff=20--git=20a/src/backend/access/transam/xlog.c=20= b/src/backend/access/transam/xlog.c=0Aindex=20f74d7a2ab1a..9b1a10dbd1c=20= 100644=0A---=20a/src/backend/access/transam/xlog.c=0A+++=20= b/src/backend/access/transam/xlog.c=0A@@=20-4723,8=20+4723,6=20@@=20= SetDataChecksumsOnInProgress(void)=0A=20{=0A=20=09uint64=09=09barrier;=0A= =20=0A-=09Assert(ControlFile=20!=3D=20NULL);=0A-=0A=20=09/*=0A=20=09=20*=20= The=20state=20transition=20is=20performed=20in=20a=20critical=20section=20= with=0A=20=09=20*=20checkpoints=20held=20off=20to=20provide=20crash=20= safety.=0A@@=20-4738,25=20+4736,16=20@@=20= SetDataChecksumsOnInProgress(void)=0A=20=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON;=0A= =20=09SpinLockRelease(&XLogCtl->info_lck);=0A=20=0A-=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON);=0A-=0A= -=09MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A-=09= END_CRIT_SECTION();=0A-=0A-=09/*=0A-=09=20*=20Update=20the=20controlfile=20= before=20waiting=20since=20if=20we=20have=20an=20immediate=0A-=09=20*=20= shutdown=20while=20waiting=20we=20want=20to=20come=20back=20up=20with=20= checksums=20enabled.=0A-=09=20*/=0A=20=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A=20=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON;=0A=20=09UpdateControlFile();=0A=20=09= LWLockRelease(ControlFileLock);=0A=20=0A-=09/*=0A-=09=20*=20Await=20= state=20change=20in=20all=20backends=20to=20ensure=20that=20all=20= backends=20are=20in=0A-=09=20*=20"inprogress-on".=20Once=20done=20we=20= know=20that=20all=20backends=20are=20writing=20data=0A-=09=20*=20= checksums.=0A-=09=20*/=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON);=0A+=0A= +=09MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09= END_CRIT_SECTION();=0A+=0A=20=09WaitForProcSignalBarrier(barrier);=0A=20= }=0A=20=0A@@=20-4787,8=20+4776,6=20@@=20SetDataChecksumsOn(void)=0A=20{=0A= =20=09uint64=09=09barrier;=0A=20=0A-=09Assert(ControlFile=20!=3D=20= NULL);=0A-=0A=20=09SpinLockAcquire(&XLogCtl->info_lck);=0A=20=0A=20=09/*=0A= @@=20-4818,11=20+4805,6=20@@=20SetDataChecksumsOn(void)=0A=20=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_VERSION;=0A=20=09= SpinLockRelease(&XLogCtl->info_lck);=0A=20=0A-=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_ON);=0A-=0A-=09= MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A-=09= END_CRIT_SECTION();=0A-=0A=20=09/*=0A=20=09=20*=20Update=20the=20= controlfile=20before=20waiting=20since=20if=20we=20have=20an=20immediate=0A= =20=09=20*=20shutdown=20while=20waiting=20we=20want=20to=20come=20back=20= up=20with=20checksums=20enabled.=0A@@=20-4832,12=20+4814,12=20@@=20= SetDataChecksumsOn(void)=0A=20=09UpdateControlFile();=0A=20=09= LWLockRelease(ControlFileLock);=0A=20=0A-=09= RequestCheckpoint(CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT=20|=20= CHECKPOINT_FAST);=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_ON);=0A=20=0A-=09/*=0A= -=09=20*=20Await=20state=20transition=20to=20"on"=20in=20all=20backends.=20= When=20done=20we=20know=20that=0A-=09=20*=20data=20checksums=20are=20= both=20written=20and=20verified=20in=20all=20backends.=0A-=09=20*/=0A+=09= MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09= END_CRIT_SECTION();=0A+=0A+=09RequestCheckpoint(CHECKPOINT_FORCE=20|=20= CHECKPOINT_WAIT=20|=20CHECKPOINT_FAST);=0A=20=09= WaitForProcSignalBarrier(barrier);=0A=20}=0A=20=0A@@=20-4859,8=20+4841,6=20= @@=20SetDataChecksumsOff(void)=0A=20{=0A=20=09uint64=09=09barrier;=0A=20=0A= -=09Assert(ControlFile=20!=3D=20NULL);=0A-=0A=20=09= SpinLockAcquire(&XLogCtl->info_lck);=0A=20=0A=20=09/*=20If=20data=20= checksums=20are=20already=20disabled=20there=20is=20nothing=20to=20do=20= */=0A@@=20-4891,22=20+4871,17=20@@=20SetDataChecksumsOff(void)=0A=20=09=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_INPROGRESS_OFF;=0A= =20=09=09SpinLockRelease(&XLogCtl->info_lck);=0A=20=0A+=09=09= LWLockAcquire(ControlFileLock,=20LW_EXCLUSIVE);=0A+=09=09= ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF;=0A+=09=09UpdateControlFile();=0A+=09=09= LWLockRelease(ControlFileLock);=0A+=0A=20=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF);=0A=20=0A= =20=09=09MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A=20=09=09= END_CRIT_SECTION();=0A=20=0A-=09=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A-=09=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_OFF;=0A-=09=09UpdateControlFile();=0A-=09=09= LWLockRelease(ControlFileLock);=0A-=0A=20=09=09= RequestCheckpoint(CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT=20|=20= CHECKPOINT_FAST);=0A-=0A-=09=09/*=0A-=09=09=20*=20Update=20local=20state=20= in=20all=20backends=20to=20ensure=20that=20any=20backend=20in=0A-=09=09=20= *=20"on"=20state=20is=20changed=20to=20"inprogress-off".=0A-=09=09=20*/=0A= =20=09=09WaitForProcSignalBarrier(barrier);=0A=20=0A=20=09=09/*=0A@@=20= -4935,18=20+4910,17=20@@=20SetDataChecksumsOff(void)=0A=20=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_OFF;=0A=20=09= SpinLockRelease(&XLogCtl->info_lck);=0A=20=0A-=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_OFF);=0A-=0A-=09= MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A-=09= END_CRIT_SECTION();=0A-=0A=20=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A=20=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_OFF;=0A=20=09UpdateControlFile();=0A=20=09= LWLockRelease(ControlFileLock);=0A=20=0A-=09= RequestCheckpoint(CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT=20|=20= CHECKPOINT_FAST);=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_OFF);=0A=20=0A+=09= MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09= END_CRIT_SECTION();=0A+=0A+=09RequestCheckpoint(CHECKPOINT_FORCE=20|=20= CHECKPOINT_WAIT=20|=20CHECKPOINT_FAST);=0A=20=09= WaitForProcSignalBarrier(barrier);=0A=20}=0A=20=0A@@=20-4961,6=20+4935,7=20= @@=20SetDataChecksumsOff(void)=0A=20void=0A=20= InitLocalDataChecksumState(void)=0A=20{=0A+=09= Assert(InterruptHoldoffCount=20>=200);=0A=20=09= SpinLockAcquire(&XLogCtl->info_lck);=0A=20=09= SetLocalDataChecksumState(XLogCtl->data_checksum_version);=0A=20=09= SpinLockRelease(&XLogCtl->info_lck);=0A@@=20-5427,7=20+5402,6=20@@=20= XLOGShmemInit(void=20*arg)=0A=20=0A=20=09/*=20Use=20the=20checksum=20= info=20from=20control=20file=20*/=0A=20=09XLogCtl->data_checksum_version=20= =3D=20ControlFile->data_checksum_version;=0A-=0A=20=09= SetLocalDataChecksumState(XLogCtl->data_checksum_version);=0A=20=0A=20=09= SpinLockInit(&XLogCtl->Insert.insertpos_lck);=0A@@=20-7518,7=20+7492,9=20= @@=20CreateCheckPoint(int=20flags)=0A=20=09=20*=20Get=20the=20current=20= data_checksum_version=20value=20from=20xlogctl,=20valid=20at=20the=0A=20=09= =20*=20time=20of=20the=20checkpoint.=0A=20=09=20*/=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A=20=09= checkPoint.dataChecksumState=20=3D=20XLogCtl->data_checksum_version;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A=20=0A=20=09if=20(shutdown)=0A=20=09= {=0A@@=20-7639,10=20+7615,6=20@@=20CreateCheckPoint(int=20flags)=0A=20=09= =09checkPoint.nextOid=20+=3D=20TransamVariables->oidCount;=0A=20=09= LWLockRelease(OidGenLock);=0A=20=0A-=09= SpinLockAcquire(&XLogCtl->info_lck);=0A-=09checkPoint.dataChecksumState=20= =3D=20XLogCtl->data_checksum_version;=0A-=09= SpinLockRelease(&XLogCtl->info_lck);=0A-=0A=20=09= checkPoint.logicalDecodingEnabled=20=3D=20IsLogicalDecodingEnabled();=0A=20= =0A=20=09MultiXactGetCheckptMulti(shutdown,=0A@@=20-7792,9=20+7764,6=20= @@=20CreateCheckPoint(int=20flags)=0A=20=09ControlFile->minRecoveryPoint=20= =3D=20InvalidXLogRecPtr;=0A=20=09ControlFile->minRecoveryPointTLI=20=3D=20= 0;=0A=20=0A-=09/*=20make=20sure=20we=20start=20with=20the=20checksum=20= version=20as=20of=20the=20checkpoint=20*/=0A-=09= ControlFile->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A= -=0A=20=09/*=0A=20=09=20*=20Persist=20unloggedLSN=20value.=20It's=20= reset=20on=20crash=20recovery,=20so=20this=20goes=0A=20=09=20*=20unused=20= on=20non-shutdown=20checkpoints,=20but=20seems=20useful=20to=20store=20= it=20always=0A@@=20-8871,11=20+8840,6=20@@=20xlog_redo(XLogReaderState=20= *record)=0A=20=09=09MultiXactAdvanceOldest(checkPoint.oldestMulti,=0A=20=09= =09=09=09=09=09=09=20=20=20checkPoint.oldestMultiDB);=0A=20=0A-=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A-=09=09= XLogCtl->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A-=09= =09SetLocalDataChecksumState(checkPoint.dataChecksumState);=0A-=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A-=0A=20=09=09/*=0A=20=09=09=20*=20= No=20need=20to=20set=20oldestClogXid=20here=20as=20well;=20it'll=20be=20= set=20when=20we=0A=20=09=09=20*=20redo=20an=20xl_clog_truncate=20if=20it=20= changed=20since=20initialization.=0A@@=20-8936,6=20+8900,8=20@@=20= xlog_redo(XLogReaderState=20*record)=0A=20=09=09= LWLockAcquire(ControlFileLock,=20LW_EXCLUSIVE);=0A=20=09=09= ControlFile->checkPointCopy.nextXid=20=3D=20checkPoint.nextXid;=0A=20=09=09= ControlFile->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A= +=0A+=09=09UpdateControlFile();=0A=20=09=09= LWLockRelease(ControlFileLock);=0A=20=0A=20=09=09/*=0A@@=20-8962,8=20= +8928,6=20@@=20xlog_redo(XLogReaderState=20*record)=0A=20=09{=0A=20=09=09= CheckPoint=09checkPoint;=0A=20=09=09TimeLineID=09replayTLI;=0A-=09=09= bool=09=09new_state=20=3D=20false;=0A-=09=09int=09=09=09old_state;=0A=20=0A= =20=09=09memcpy(&checkPoint,=20XLogRecGetData(record),=20= sizeof(CheckPoint));=0A=20=09=09/*=20In=20an=20ONLINE=20checkpoint,=20= treat=20the=20XID=20counter=20as=20a=20minimum=20*/=0A@@=20-9002,8=20= +8966,6=20@@=20xlog_redo(XLogReaderState=20*record)=0A=20=09=09/*=20= ControlFile->checkPointCopy=20always=20tracks=20the=20latest=20ckpt=20= XID=20*/=0A=20=09=09LWLockAcquire(ControlFileLock,=20LW_EXCLUSIVE);=0A=20= =09=09ControlFile->checkPointCopy.nextXid=20=3D=20checkPoint.nextXid;=0A= -=09=09old_state=20=3D=20ControlFile->data_checksum_version;=0A-=09=09= ControlFile->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A= =20=09=09LWLockRelease(ControlFileLock);=0A=20=0A=20=09=09/*=20TLI=20= should=20not=20change=20in=20an=20on-line=20checkpoint=20*/=0A@@=20= -9015,18=20+8977,6=20@@=20xlog_redo(XLogReaderState=20*record)=0A=20=0A=20= =09=09RecoveryRestartPoint(&checkPoint,=20record);=0A=20=0A-=09=09/*=0A-=09= =09=20*=20If=20the=20data=20checksum=20state=20change=20we=20need=20to=20= emit=20a=20barrier.=0A-=09=09=20*/=0A-=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A-=09=09= XLogCtl->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A-=09= =09if=20(checkPoint.dataChecksumState=20!=3D=20old_state)=0A-=09=09=09= new_state=20=3D=20true;=0A-=09=09SpinLockRelease(&XLogCtl->info_lck);=0A= -=0A-=09=09if=20(new_state)=0A-=09=09=09= EmitAndWaitDataChecksumsBarrier(checkPoint.dataChecksumState);=0A-=0A=20=09= =09/*=0A=20=09=09=20*=20After=20replaying=20a=20checkpoint=20record,=20= free=20all=20smgr=20objects.=0A=20=09=09=20*=20Otherwise=20we=20would=20= never=20do=20so=20for=20dropped=20relations,=20as=20the=0A@@=20-9195,6=20= +9145,7=20@@=20xlog_redo(XLogReaderState=20*record)=0A=20=0A=20=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A=20=09=09= XLogCtl->data_checksum_version=20=3D=20redo_rec.data_checksum_version;=0A= +=09=09SetLocalDataChecksumState(redo_rec.data_checksum_version);=0A=20=09= =09if=20(redo_rec.data_checksum_version=20!=3D=20= ControlFile->data_checksum_version)=0A=20=09=09=09new_state=20=3D=20= true;=0A=20=09=09SpinLockRelease(&XLogCtl->info_lck);=0A@@=20-9268,6=20= +9219,11=20@@=20xlog2_redo(XLogReaderState=20*record)=0A=20=09=09= XLogCtl->data_checksum_version=20=3D=20state.new_checksum_state;=0A=20=09= =09SpinLockRelease(&XLogCtl->info_lck);=0A=20=0A+=09=09= LWLockAcquire(ControlFileLock,=20LW_EXCLUSIVE);=0A+=09=09= ControlFile->data_checksum_version=20=3D=20state.new_checksum_state;=0A+=09= =09UpdateControlFile();=0A+=09=09LWLockRelease(ControlFileLock);=0A+=0A=20= =09=09/*=0A=20=09=09=20*=20Block=20on=20a=20procsignalbarrier=20to=20= await=20all=20processes=20having=20seen=20the=0A=20=09=09=20*=20change=20= to=20checksum=20status.=20Once=20the=20barrier=20has=20been=20passed=20= we=20can=0Adiff=20--git=20a/src/backend/postmaster/auxprocess.c=20= b/src/backend/postmaster/auxprocess.c=0Aindex=208fdc518b3a1..421502fa1cb=20= 100644=0A---=20a/src/backend/postmaster/auxprocess.c=0A+++=20= b/src/backend/postmaster/auxprocess.c=0A@@=20-68,6=20+68,14=20@@=20= AuxiliaryProcessMainCommon(void)=0A=20=0A=20=09BaseInit();=0A=20=0A+=09= /*=0A+=09=20*=20Prevent=20consuming=20interrups=20between=20setting=20= ProcSignalInit=20and=20setting=0A+=09=20*=20the=20initial=20local=20data=20= checksum=20value.=20=20If=20a=20barrier=20is=20emitted,=20and=0A+=09=20*=20= absorbed,=20before=20local=20cached=20state=20is=20initialized=20the=20= state=20transition=0A+=09=20*=20can=20be=20invalid.=0A+=09=20*/=0A+=09= HOLD_INTERRUPTS();=0A+=0A=20=09ProcSignalInit(NULL,=200);=0A=20=0A=20=09= /*=0A@@=20-88,6=20+96,8=20@@=20AuxiliaryProcessMainCommon(void)=0A=20=09=20= */=0A=20=09InitLocalDataChecksumState();=0A=20=0A+=09= RESUME_INTERRUPTS();=0A+=0A=20=09/*=0A=20=09=20*=20Auxiliary=20processes=20= don't=20run=20transactions,=20but=20they=20may=20need=20a=0A=20=09=20*=20= resource=20owner=20anyway=20to=20manage=20buffer=20pins=20acquired=20= outside=0Adiff=20--git=20a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Aindex=20= e62a36b0af5..6d66322c625=20100644=0A---=20= a/src/backend/postmaster/datachecksum_state.c=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-263,8=20+263,8=20= @@=20static=20const=20ChecksumBarrierCondition=20checksum_barriers[7]=20= =3D=0A=20=09{PG_DATA_CHECKSUM_INPROGRESS_OFF,=20= PG_DATA_CHECKSUM_VERSION},=0A=20=0A=20=09/*=0A-=09=20*=20If=20checksums=20= are=20being=20enabled=20when=20launcher_exit=20is=20executed,=20state=0A= -=09=20*=20is=20set=20to=20off=20since=20we=20cannot=20reach=20on=20at=20= that=20point.=0A+=09=20*=20If=20checksums=20are=20being=20enabled=20when=20= launcher_exit=20is=20executed,=20state=20is=0A+=09=20*=20set=20to=20off=20= since=20we=20cannot=20reach=20on=20at=20that=20point.=0A=20=09=20*/=0A=20= =09{PG_DATA_CHECKSUM_INPROGRESS_ON,=20PG_DATA_CHECKSUM_INPROGRESS_OFF},=0A= =20};=0A@@=20-1287,8=20+1287,8=20@@=20DataChecksumsShmemRequest(void=20= *arg)=0A=20/*=0A=20=20*=20DatabaseExists=0A=20=20*=0A-=20*=20Scans=20the=20= system=20catalog=20to=20check=20if=20a=20database=20with=20the=20given=20= Oid=20exist=0A-=20*=20and=20returns=20true=20if=20it=20is=20found,=20= else=20false.=0A+=20*=20Scans=20the=20system=20catalog=20to=20check=20if=20= a=20database=20with=20the=20given=20Oid=20exists=0A+=20*=20and=20returns=20= true=20if=20it=20is=20found=20=20else=20false.=0A=20=20*/=0A=20static=20= bool=0A=20DatabaseExists(Oid=20dboid)=0Adiff=20--git=20= a/src/backend/utils/init/postinit.c=20= b/src/backend/utils/init/postinit.c=0Aindex=206f074013aa9..b2c80b0bd1e=20= 100644=0A---=20a/src/backend/utils/init/postinit.c=0A+++=20= b/src/backend/utils/init/postinit.c=0A@@=20-756,6=20+756,14=20@@=20= InitPostgres(const=20char=20*in_dbname,=20Oid=20dboid,=0A=20=09=20*/=0A=20= =09SharedInvalBackendInit(false);=0A=20=0A+=09/*=0A+=09=20*=20Prevent=20= consuming=20interrups=20between=20setting=20ProcSignalInit=20and=20= setting=0A+=09=20*=20the=20initial=20local=20data=20checksum=20value.=20=20= If=20a=20barrier=20is=20emitted,=20and=0A+=09=20*=20absorbed,=20before=20= local=20cached=20state=20is=20initialized=20the=20state=20transition=0A+=09= =20*=20can=20be=20invalid.=0A+=09=20*/=0A+=09HOLD_INTERRUPTS();=0A+=0A=20= =09ProcSignalInit(MyCancelKey,=20MyCancelKeyLength);=0A=20=0A=20=09/*=0A= @@=20-776,6=20+784,8=20@@=20InitPostgres(const=20char=20*in_dbname,=20= Oid=20dboid,=0A=20=09=20*/=0A=20=09InitLocalDataChecksumState();=0A=20=0A= +=09RESUME_INTERRUPTS();=0A+=0A=20=09/*=0A=20=09=20*=20Also=20set=20up=20= timeout=20handlers=20needed=20for=20backend=20operation.=20=20We=20need=0A= =20=09=20*=20these=20in=20every=20case=20except=20bootstrap.=0A--=20=0A= 2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0003-Handle-data_checksum-state-changes-during-launcher_e.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0003-Handle-data_checksum-state-changes-during-launcher_e.patch" Content-Transfer-Encoding: quoted-printable =46rom=20b24179efcd960c15a1fa2309bb5107c5625e33e8=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:16=20= +0200=0ASubject:=20[PATCH=203/8]=20Handle=20data_checksum=20state=20= changes=20during=20launcher_exit=0A=0AWhen=20erroring=20out=20from=20the=20= datachecksums=20launcher=20during=20data=20checksum=0Aenabling,=20before=20= state=20has=20transitioned=20to=20"on",=20we=20revert=20back=20to=20the=0A= "off"=20state.=20=20Since=20checksums=20weren't=20enabled,=20there=20is=20= no=20use=20staying=0Ain=20an=20inprogress=20state=20since=20the=20= checksum=20launcher=20currently=20doesn't=0Asupport=20restarting=20from=20= where=20it=20left=20off.=20=20Should=20restartability=20get=0Aadded=20in=20= the=20future,=20this=20would=20need=20to=20be=20revisited.=20=20This=20= state=0Atransition=20was=20however=20missing=20from=20the=20allowed=20= transitions=20in=20the=0Astatemachine=20causing=20an=20error.=0A=0A= Author:=20Daniel=20Gustafsson=20=0ADiscussion:=20= https://postgr.es/m/xxx=0A---=0A=20src/backend/access/transam/xlog.c=20=20= =20=20=20=20=20=20=20=20=20|=2015=20++++++++-------=0A=20= src/backend/postmaster/datachecksum_state.c=20|=20=208=20+++++++-=0A=202=20= files=20changed,=2015=20insertions(+),=208=20deletions(-)=0A=0Adiff=20= --git=20a/src/backend/access/transam/xlog.c=20= b/src/backend/access/transam/xlog.c=0Aindex=20e39af79c03b..f74d7a2ab1a=20= 100644=0A---=20a/src/backend/access/transam/xlog.c=0A+++=20= b/src/backend/access/transam/xlog.c=0A@@=20-4871,13=20+4871,14=20@@=20= SetDataChecksumsOff(void)=0A=20=09}=0A=20=0A=20=09/*=0A-=09=20*=20If=20= data=20checksums=20are=20currently=20enabled=20we=20first=20transition=20= to=20the=0A-=09=20*=20"inprogress-off"=20state=20during=20which=20= backends=20continue=20to=20write=0A-=09=20*=20checksums=20without=20= verifying=20them.=20When=20all=20backends=20are=20in=0A-=09=20*=20= "inprogress-off"=20the=20next=20transition=20to=20"off"=20can=20be=20= performed,=20after=0A-=09=20*=20which=20all=20data=20checksum=20= processing=20is=20disabled.=0A-=09=20*/=0A-=09if=20= (XLogCtl->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_VERSION)=0A+=09= =20*=20If=20data=20checksums=20are=20currently=20enabled,=20or=20in=20= the=20process=20of=20being=0A+=09=20*=20enabled,=20we=20first=20= transition=20to=20the=20"inprogress-off"=20state=20during=20which=0A+=09=20= *=20backends=20continue=20to=20write=20checksums=20without=20verifying=20= them.=20When=20all=0A+=09=20*=20backends=20are=20in=20"inprogress-off"=20= the=20next=20transition=20to=20"off"=20can=20be=0A+=09=20*=20performed,=20= after=20which=20all=20data=20checksum=20processing=20is=20disabled.=0A+=09= =20*/=0A+=09if=20(XLogCtl->data_checksum_version=20=3D=3D=20= PG_DATA_CHECKSUM_VERSION=20||=0A+=09=09XLogCtl->data_checksum_version=20= =3D=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON)=0A=20=09{=0A=20=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A=20=0Adiff=20--git=20= a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Aindex=20= ea102086144..e62a36b0af5=20100644=0A---=20= a/src/backend/postmaster/datachecksum_state.c=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-235,7=20+235,7=20= @@=20typedef=20struct=20ChecksumBarrierCondition=0A=20=09int=09=09=09to;=0A= =20}=20ChecksumBarrierCondition;=0A=20=0A-static=20const=20= ChecksumBarrierCondition=20checksum_barriers[6]=20=3D=0A+static=20const=20= ChecksumBarrierCondition=20checksum_barriers[7]=20=3D=0A=20{=0A=20=09/*=0A= =20=09=20*=20Disabling=20checksums:=20If=20checksums=20are=20currently=20= enabled,=20disabling=20must=0A@@=20-261,6=20+261,12=20@@=20static=20= const=20ChecksumBarrierCondition=20checksum_barriers[6]=20=3D=0A=20=09=20= *=20checksums,=20we=20can=20go=20straight=20back=20to=20'on'=0A=20=09=20= */=0A=20=09{PG_DATA_CHECKSUM_INPROGRESS_OFF,=20= PG_DATA_CHECKSUM_VERSION},=0A+=0A+=09/*=0A+=09=20*=20If=20checksums=20= are=20being=20enabled=20when=20launcher_exit=20is=20executed,=20state=0A= +=09=20*=20is=20set=20to=20off=20since=20we=20cannot=20reach=20on=20at=20= that=20point.=0A+=09=20*/=0A+=09{PG_DATA_CHECKSUM_INPROGRESS_ON,=20= PG_DATA_CHECKSUM_INPROGRESS_OFF},=0A=20};=0A=20=0A=20/*=0A--=20=0A2.39.3=20= (Apple=20Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0002-Test-improvements-for-online-checksums.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0002-Test-improvements-for-online-checksums.patch" Content-Transfer-Encoding: quoted-printable =46rom=20a24b6fc17b633514de2ca5064c65029910be06ac=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:44:09=20= +0200=0ASubject:=20[PATCH=202/8]=20Test=20improvements=20for=20online=20= checksums=0A=0AThis=20includes=20a=20number=20of=20smaller=20fixups=20to=20= the=20onine=20checksums=20test=0Amodule=20which=20were=20found=20during=20= postcommit=20review=20and=20stabilization=0Awork.=0A=0A=20*=20Fix=20= scope=20increase=20for=20PG_TEST_EXTRA:=20The=20online=20checksums=20= tests=0A=20=20=20have=20two=20levels=20of=20PG_TEST_EXTRA,=20checksum=20= and=20checksums_extended=0A=20=20=20for=20extra=20test=20runs=20and=20= test=20runs=20with=20increased=20randomization.=0A=20=20=20The=20logic=20= for=20increasing=20the=20number=20of=20test=20iterations=20was=20however=0A= =20=20=20backwards.=0A=20*=20Change=20stopmode=20for=20PITR=20test:=20= The=20pitr=20suite=20used=20immediate=20stop=0A=20=20=20mode=20which=20= caused=20problems=20on=20slower=20machines=20where=20the=20sigquit=0A=20=20= =20would=20interrupt=20archive=20commands=20leaving=20partial=20WAL=20= files=20behind.=0A=20=20=20This=20would=20then=20prevent=20restart.=20=20= Fix=20by=20using=20fast=20mode=20which=20is=0A=20=20=20the=20appropriate=20= mode=20for=20the=20test=20at=20hand.=20=20Also=20increase=20timeouts=0A=20= =20=20to=20help=20slower=20test=20systems=20since=20an=20expired=20= timeout=20will=20incur=0A=20=20=20the=20same=20effect=20as=20an=20= immediate=20standby=20with=20a=20partial=20WAL=20left=0A=20=20=20behind.=20= =20This=20issue=20was=20observed=20when=20running=20the=20test=20suites=20= on=0A=20=20=20a=20Raspberry=20Pi=204=20machine.=0A=20*=20Improve=20= logging:=20The=20test=20suite=20for=20data=20checksums=20use=20a=20set=20= of=0A=20=20=20helper=20functions=20in=20a=20Perl=20module=20to=20avoid=20= repeating=20code,=20this=0A=20=20=20makes=20sure=20that=20the=20helper=20= functions=20do=20a=20better=20job=20of=20logging=0A=20=20=20their=20test=20= output=20to=20make=20debug=20easier.=0A=20*=20Remove=20unused=20code:=20= wait_for_cluster_crash=20was=20used=20during=20the=0A=20=20=20= development=20of=20online=20checksums=20but=20was=20never=20used=20in=20= any=20test=0A=20=20=20which=20shipped,=20so=20remove=20the=20function.=0A= =20*=20Standby=20fixes:=20Ensure=20no=20vacuum=20on=20pgbench=20init=20= on=20standby=20with=0A=20=20=20-n=20to=20avoid=20bogus=20error=20message=20= in=20the=20log,=20and=20enable=0A=20=20=20hot_standby_feedback=20to=20= prevent=20queries=20from=20getting=20cancelled=0A=20=20=20due=20to=20= recovery=20on=20slower=20systems.=0A=0AAuthor:=20Daniel=20Gustafsson=20= =0AAuthor:=20Tomas=20Vondra=20=0A= Discussion:=20https://postgr.es/m/xxx=0A---=0A=20= .../test_checksums/t/007_pgbench_standby.pl=20=20=20|=2012=20+++--=0A=20= src/test/modules/test_checksums/t/008_pitr.pl=20|=20=205=20+-=0A=20= .../test_checksums/t/DataChecksums/Utils.pm=20=20=20|=2053=20= +++----------------=0A=203=20files=20changed,=2020=20insertions(+),=2050=20= deletions(-)=0A=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/007_pgbench_standby.pl=20= b/src/test/modules/test_checksums/t/007_pgbench_standby.pl=0Aindex=20= f3611e7ce25..0b3996f1d69=20100644=0A---=20= a/src/test/modules/test_checksums/t/007_pgbench_standby.pl=0A+++=20= b/src/test/modules/test_checksums/t/007_pgbench_standby.pl=0A@@=20-49,8=20= +49,8=20@@=20my=20$node_standby_loglocation=20=3D=200;=0A=20#=20of=20= tests=20performed=20and=20the=20wall=20time=20taken=20is=20= non-deterministic=20as=20the=20test=0A=20#=20performs=20a=20lot=20of=20= randomized=20actions,=20but=205=20iterations=20will=20be=20a=20long=20= test=0A=20#=20run=20regardless.=0A-my=20$TEST_ITERATIONS=20=3D=205;=0A= -$TEST_ITERATIONS=20=3D=201=20if=20($extended);=0A+my=20$TEST_ITERATIONS=20= =3D=201;=0A+$TEST_ITERATIONS=20=3D=205=20if=20($extended);=0A=20=0A=20#=20= Variables=20which=20record=20the=20current=20state=20of=20the=20cluster=0A= =20my=20$data_checksum_state=20=3D=20'off';=0A@@=20-83,6=20+83,7=20@@=20= sub=20background_pgbench=0A=20=09push(@cmd,=20'-C')=20if=20($extended=20= &&=20cointoss());=0A=20=09#=20If=20we=20run=20on=20a=20standby=20it=20= needs=20to=20be=20a=20read-only=20benchmark=0A=20=09push(@cmd,=20'-S')=20= if=20($standby);=0A+=09push(@cmd,=20'-n')=20if=20($standby);=0A=20=09#=20= Finally=20add=20the=20database=20name=20to=20use=0A=20=09push(@cmd,=20= 'postgres');=0A=20=0A@@=20-146,8=20+147,10=20@@=20sub=20= flip_data_checksums=0A=20=09=09=09=20=20.=20"FROM=20= pg_catalog.pg_settings=20"=0A=20=09=09=09=20=20.=20"WHERE=20name=20=3D=20= 'data_checksums';");=0A=20=0A-=09=09is(($result=20eq=20'inprogress-on'=20= ||=20$result=20eq=20'on'),=0A-=09=09=091,=20'ensure=20checksums=20are=20= on,=20or=20in=20progress,=20on=20standby_1');=0A+=09=09is(=20($result=20= eq=20'inprogress-on'=20||=20$result=20eq=20'on'),=0A+=09=09=091,=0A+=09=09= =09'ensure=20checksums=20are=20on,=20or=20in=20progress,=20on=20= standby_1,=20got:=20'=0A+=09=09=09=20=20.=20$result);=0A=20=0A=20=09=09#=20= Wait=20for=20checksums=20enabled=20on=20the=20primary=20and=20standby=0A=20= =09=09wait_for_checksum_state($node_primary,=20'on');=0A@@=20-210,6=20= +213,7=20@@=20$node_primary->append_conf(=0A=20=09qq[=0A=20= max_connections=20=3D=2030=0A=20log_statement=20=3D=20none=0A= +hot_standby_feedback=20=3D=20on=0A=20]);=0A=20$node_primary->start;=0A=20= $node_primary->safe_psql('postgres',=20'CREATE=20EXTENSION=20= test_checksums;');=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/008_pitr.pl=20= b/src/test/modules/test_checksums/t/008_pitr.pl=0Aindex=20= e8cb2b0ed96..1f8176686fd=20100644=0A---=20= a/src/test/modules/test_checksums/t/008_pitr.pl=0A+++=20= b/src/test/modules/test_checksums/t/008_pitr.pl=0A@@=20-124,11=20+124,14=20= @@=20$node_primary->init(=0A=20=09has_archiving=20=3D>=201,=0A=20=09= allows_streaming=20=3D>=201,=0A=20=09no_data_checksums=20=3D>=201);=0A= +my=20$timeout_unit=20=3D=20's';=0A=20$node_primary->append_conf(=0A=20=09= 'postgresql.conf',=0A=20=09qq[=0A=20max_connections=20=3D=20100=0A=20= log_statement=20=3D=20none=0A+wal_sender_timeout=20=3D=20= $PostgreSQL::Test::Utils::timeout_default$timeout_unit=0A= +wal_receiver_timeout=20=3D=20= $PostgreSQL::Test::Utils::timeout_default$timeout_unit=0A=20]);=0A=20= $node_primary->start;=0A=20=0A@@=20-154,7=20+157,7=20@@=20my=20= ($pre_lsn,=20$post_lsn)=20=3D=20flip_data_checksums();=0A=20= $node_primary->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20= +=201;");=0A=20$node_primary->safe_psql('postgres',=20"SELECT=20= pg_create_restore_point('a');");=0A=20= $node_primary->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20= +=201;");=0A-$node_primary->stop('immediate');=0A= +$node_primary->stop('fast');=0A=20=0A=20my=20$node_pitr=20=3D=20= PostgreSQL::Test::Cluster->new('pitr_backup');=0A=20= $node_pitr->init_from_backup(=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=20= b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0Aindex=20= fb704623a60..cb78dd6ecfb=20100644=0A---=20= a/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0A+++=20= b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0A@@=20-43,7=20= +43,6=20@@=20our=20@EXPORT=20=3D=20qw(=0A=20=20=20stopmode=0A=20=20=20= test_checksum_state=0A=20=20=20wait_for_checksum_state=0A-=20=20= wait_for_cluster_crash=0A=20);=0A=20=0A=20=3Dpod=0A@@=20-67,7=20+66,10=20= @@=20sub=20test_checksum_state=0A=20=09my=20$result=20=3D=20= $postgresnode->safe_psql('postgres',=0A=20=09=09"SELECT=20setting=20FROM=20= pg_catalog.pg_settings=20WHERE=20name=20=3D=20'data_checksums';"=0A=20=09= );=0A-=09is($result,=20$state,=20'ensure=20checksums=20are=20set=20to=20= '=20.=20$state);=0A+=09is($result,=20$state,=0A+=09=09=09'ensure=20= checksums=20are=20set=20to=20'=0A+=09=09=20=20.=20$state=20.=20'=20on=20= '=0A+=09=09=20=20.=20$postgresnode->name());=0A=20=09return=20$result=20= eq=20$state;=0A=20}=0A=20=0A@@=20-89,52=20+91,13=20@@=20sub=20= wait_for_checksum_state=0A=20=09=09'postgres',=0A=20=09=09"SELECT=20= setting=20FROM=20pg_catalog.pg_settings=20WHERE=20name=20=3D=20= 'data_checksums';",=0A=20=09=09$state);=0A-=09is($res,=201,=20'ensure=20= data=20checksums=20are=20transitioned=20to=20'=20.=20$state);=0A+=09= is($res,=201,=0A+=09=09=09'ensure=20data=20checksums=20are=20= transitioned=20to=20'=0A+=09=09=20=20.=20$state=20.=20'=20on=20'=0A+=09=09= =20=20.=20$postgresnode->name());=0A=20=09return=20$res=20=3D=3D=201;=0A=20= }=0A=20=0A-=3Ditem=20wait_for_cluster_crash(node,=20params)=0A-=0A= -Repeatedly=20test=20if=20the=20cluster=20running=20at=20B=20= responds=20to=20connections=0A-and=20return=20when=20it=20no=20longer=20= does=20so,=20or=20when=20it=20times=20out.=20=20Processing=20will=0A-run=20= for=20$PostgreSQL::Test::Utils::timeout_default=20seconds=20unless=20a=20= timeout=0A-value=20is=20specified=20as=20a=20parameter.=20=20Returns=20= True=20if=20the=20cluster=20crashed,=20else=0A-False=20if=20the=20= process=20timed=20out.=0A-=0A-=3Dover=0A-=0A-=3Ditem=20timeout=0A-=0A= -Approximate=20number=20of=20seconds=20to=20wait=20for=20cluster=20to=20= crash,=20default=20is=0A-$PostgreSQL::Test::Utils::timeout_default.=20=20= There=20are=20no=20real-time=20guarantees=0A-that=20the=20total=20= process=20time=20won't=20exceed=20the=20timeout.=0A-=0A-=3Dback=0A-=0A= -=3Dcut=0A-=0A-sub=20wait_for_cluster_crash=0A-{=0A-=09my=20= $postgresnode=20=3D=20shift;=0A-=09my=20%params=20=3D=20@_;=0A-=09my=20= $crash=20=3D=200;=0A-=0A-=09$params{timeout}=20=3D=20= $PostgreSQL::Test::Utils::timeout_default=0A-=09=20=20unless=20= (defined($params{timeout}));=0A-=0A-=09for=20(my=20$naps=20=3D=200;=20= $naps=20<=20$params{timeout};=20$naps++)=0A-=09{=0A-=09=09if=20= (!$postgresnode->is_alive)=0A-=09=09{=0A-=09=09=09$crash=20=3D=201;=0A-=09= =09=09last;=0A-=09=09}=0A-=09=09sleep(1);=0A-=09}=0A-=0A-=09return=20= $crash=20=3D=3D=201;=0A-}=0A-=0A=20=3Ditem=20= enable_data_checksums($node,=20%params)=0A=20=0A=20Function=20for=20= enabling=20data=20checksums=20in=20the=20cluster=20running=20at=20= B.=0A--=20=0A2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=0001-Prevent-pg_enable-disable_data_checksums-on-standby.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0001-Prevent-pg_enable-disable_data_checksums-on-standby.patch" Content-Transfer-Encoding: quoted-printable =46rom=2096f0fc24add90a7c0af0e3700c24b835f4497a7b=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Thu,=2023=20Apr=202026=2010:43:58=20= +0200=0ASubject:=20[PATCH=201/8]=20Prevent=20= pg_enable/disable_data_checksums()=20on=20standby=0A=0AThese=20functions=20= missed=20a=20RecoveryInProgress()=20check,=20allowing=20them=20to=0Abe=20= called=20on=20a=20hot=20standby.=20=20Enabling,=20or=20disabling,=20= checksums=20on=20the=0Astandby=20only=20would=20cause=20the=20cluster=20= to=20get=20out=20of=20sync=20and=20replaying=0Achecksum=20transitions=20= to=20fail.=0A=0AAuthor:=20Satyanarayana=20Narlapuram=20= =0ADiscussion:=20= https://postgr.es/m/CAHg+QDfRk4-S7DMmdbXJnQ-xF=3DsUpMAKuh8b83ObLqYVKx5QLA@= mail.gmail.com=0A---=0A=20src/backend/postmaster/datachecksum_state.c=20= |=204=20++++=0A=201=20file=20changed,=204=20insertions(+)=0A=0Adiff=20= --git=20a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Aindex=20= 5556a9ca893..ea102086144=20100644=0A---=20= a/src/backend/postmaster/datachecksum_state.c=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-487,6=20+487,8=20= @@=20AbsorbDataChecksumsBarrier(ProcSignalBarrierType=20barrier)=0A=20= Datum=0A=20disable_data_checksums(PG_FUNCTION_ARGS)=0A=20{=0A+=09= PreventCommandDuringRecovery("pg_disable_data_checksums()");=0A+=0A=20=09= if=20(!superuser())=0A=20=09=09ereport(ERROR,=0A=20=09=09=09=09= errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),=0A@@=20-507,6=20+509,8=20@@=20= enable_data_checksums(PG_FUNCTION_ARGS)=0A=20=09int=09=09=09cost_delay=20= =3D=20PG_GETARG_INT32(0);=0A=20=09int=09=09=09cost_limit=20=3D=20= PG_GETARG_INT32(1);=0A=20=0A+=09= PreventCommandDuringRecovery("pg_enable_data_checksums()");=0A+=0A=20=09= if=20(!superuser())=0A=20=09=09ereport(ERROR,=0A=20=09=09=09=09= errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),=0A--=20=0A2.39.3=20(Apple=20= Git-146)=0A=0A= --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F Content-Disposition: attachment; filename=POC_test.diff.txt Content-Type: text/plain; x-unix-mode=0644; name="POC_test.diff.txt" Content-Transfer-Encoding: quoted-printable commit f502fa990e877c6e4d1d03518e524519d8e88806 Author: Tomas Vondra Date: Wed Apr 15 12:50:06 2026 +0200 POC: Test checksum state transitions using step through injection = points diff --git a/src/backend/access/transam/xlog.c = b/src/backend/access/transam/xlog.c index be92b6af20f..5b481784543 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4762,6 +4762,9 @@ SetDataChecksumsOnInProgress(void) uint32 data_checksum_version; =20 elog(LOG, "SetDataChecksumsOnInProgress / start"); + + = INJECTION_POINT("datachecksums-enable-inprogress-checksums-delay", = NULL); + /* * The state transition is performed in a critical section with * checkpoints held off to provide crash safety. @@ -4782,6 +4785,8 @@ SetDataChecksumsOnInProgress(void) MyProc->delayChkptFlags &=3D ~DELAY_CHKPT_START; END_CRIT_SECTION(); =20 + = INJECTION_POINT("datachecksums-enable-inprogress-checksums-after-xlogctl",= NULL); + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); =20 elog(LOG, "SetDataChecksumsOnInProgress = ControlFile->data_checksum_version %u =3D> %u", @@ -4791,6 +4796,8 @@ SetDataChecksumsOnInProgress(void) UpdateControlFile(); LWLockRelease(ControlFileLock); =20 + = INJECTION_POINT("datachecksums-enable-inprogress-checksums-after-controlfi= le", NULL); + elog(LOG, "SetDataChecksumsOnInProgress / = EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_INPROGRESS_ON)"); EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_INPROGRESS_ON); =20 @@ -4862,6 +4869,8 @@ SetDataChecksumsOn(void) MyProc->delayChkptFlags &=3D ~DELAY_CHKPT_START; END_CRIT_SECTION(); =20 + INJECTION_POINT("datachecksums-enable-checksums-after-xlogctl", = NULL); + /* * Update the controlfile before waiting since if we have an = immediate * shutdown while waiting we want to come back up with checksums = enabled. @@ -4875,9 +4884,13 @@ SetDataChecksumsOn(void) UpdateControlFile(); LWLockRelease(ControlFileLock); =20 + = INJECTION_POINT("datachecksums-enable-checksums-after-controlfile", = NULL); + elog(LOG, "SetDataChecksumsOn / = RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | = CHECKPOINT_FAST)"); RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | = CHECKPOINT_FAST); =20 + = INJECTION_POINT("datachecksums-enable-checksums-after-checkpoint", = NULL); + elog(LOG, "SetDataChecksumsOn / = EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_VERSION)"); EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_VERSION); =20 @@ -4924,6 +4937,8 @@ SetDataChecksumsOff(void) { SpinLockRelease(&XLogCtl->info_lck); =20 + = INJECTION_POINT("datachecksums-disable-inprogress-checksums-delay", = NULL); + START_CRIT_SECTION(); MyProc->delayChkptFlags |=3D DELAY_CHKPT_START; =20 @@ -4934,24 +4949,30 @@ SetDataChecksumsOff(void) XLogCtl->data_checksum_version =3D = PG_DATA_CHECKSUM_INPROGRESS_OFF; SpinLockRelease(&XLogCtl->info_lck); =20 - elog(LOG, "SetDataChecksumsOff / = XLogCtl->data_checksum_version %u =3D %u", + elog(LOG, "SetDataChecksumsOff / = XLogCtl->data_checksum_version %u =3D> %u", data_checksum_version, = PG_DATA_CHECKSUM_INPROGRESS_OFF); =20 MyProc->delayChkptFlags &=3D ~DELAY_CHKPT_START; END_CRIT_SECTION(); =20 + = INJECTION_POINT("datachecksums-disable-inprogress-checksums-after-xlogctl"= , NULL); + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); =20 - elog(LOG, "SetDataChecksumsOff / = ControlFile->data_checksum_version %u =3D %u", + elog(LOG, "SetDataChecksumsOff / = ControlFile->data_checksum_version %u =3D> %u", ControlFile->data_checksum_version, = PG_DATA_CHECKSUM_INPROGRESS_OFF); =20 ControlFile->data_checksum_version =3D = PG_DATA_CHECKSUM_INPROGRESS_OFF; UpdateControlFile(); LWLockRelease(ControlFileLock); =20 + = INJECTION_POINT("datachecksums-disable-inprogress-checksums-after-controlf= ile", NULL); + elog(LOG, "SetDataChecksumsOff / = RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | = CHECKPOINT_FAST)"); RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | = CHECKPOINT_FAST); =20 + = INJECTION_POINT("datachecksums-disable-inprogress-checksums-after-checkpoi= nt", NULL); + elog(LOG, "SetDataChecksumsOff / = EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_INPROGRESS_OFF)"); = EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_INPROGRESS_OFF); =20 @@ -4971,6 +4992,8 @@ SetDataChecksumsOff(void) SpinLockRelease(&XLogCtl->info_lck); } =20 + INJECTION_POINT("datachecksums-disable-checksums-delay", NULL); + START_CRIT_SECTION(); /* Ensure that we don't incur a checkpoint during disabling = checksums */ MyProc->delayChkptFlags |=3D DELAY_CHKPT_START; @@ -4988,6 +5011,8 @@ SetDataChecksumsOff(void) MyProc->delayChkptFlags &=3D ~DELAY_CHKPT_START; END_CRIT_SECTION(); =20 + INJECTION_POINT("datachecksums-disable-checksums-after-xlogctl", = NULL); + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); =20 elog(LOG, "SetDataChecksumsOff / = ControlFile->data_checksum_version %u =3D> %u", @@ -4997,9 +5022,13 @@ SetDataChecksumsOff(void) UpdateControlFile(); LWLockRelease(ControlFileLock); =20 + = INJECTION_POINT("datachecksums-disable-checksums-after-controlfile", = NULL); + elog(LOG, "SetDataChecksumsOff / = RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | = CHECKPOINT_FAST)"); RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | = CHECKPOINT_FAST); =20 + = INJECTION_POINT("datachecksums-disable-checksums-after-checkpoint", = NULL); + elog(LOG, "SetDataChecksumsOff / = EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_OFF)"); EmitAndWaitDataChecksumsBarrier(PG_DATA_CHECKSUM_OFF); =20 diff --git a/src/test/modules/test_checksums/t/010_injection_2.pl = b/src/test/modules/test_checksums/t/010_injection_2.pl new file mode 100644 index 00000000000..a8ae9ffd151 --- /dev/null +++ b/src/test/modules/test_checksums/t/010_injection_2.pl @@ -0,0 +1,209 @@ + +# Copyright (c) 2026, PostgreSQL Global Development Group + +# Test suite for testing enabling data checksums in an online cluster = with +# injection point tests injecting failures into the processing + +use strict; +use warnings FATAL =3D> 'all'; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +use FindBin; +use lib $FindBin::RealBin; + +use DataChecksums::Utils; + +# This test suite is expensive, or very expensive, to execute. There = are two +# PG_TEST_EXTRA options for running it, "checksum" for a pared-down = test suite +# an "checksum_extended" for the full suite. The full suite can run = for hours +# on slow or constrained systems. +my $extended =3D undef; +if ($ENV{PG_TEST_EXTRA}) +{ + $extended =3D 1 if ($ENV{PG_TEST_EXTRA} =3D~ = /\bchecksum_extended\b/); + plan skip_all =3D> 'Expensive data checksums test disabled' + unless ($ENV{PG_TEST_EXTRA} =3D~ /\bchecksum(_extended)?\b/); +} + +if ($ENV{enable_injection_points} ne 'yes') +{ + plan skip_all =3D> 'Injection points not supported by this = build'; +} + +# = --------------------------------------------------------------------------= - +# Test cluster setup +# + +# Initiate testcluster +my $node =3D PostgreSQL::Test::Cluster->new('injection_node'); +$node->init(no_data_checksums =3D> 1); +$node->start; + +# Set up test environment +$node->safe_psql('postgres', 'CREATE EXTENSION test_checksums;'); +$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;'); + +my $pgbench =3D undef; +my $scalefactor =3D ($extended ? 10 : 1); +my $node_loglocation =3D 0; + +$node->command_ok( + [ + 'pgbench', '-p', $node->port, '-i', + '-s', $scalefactor, '-q', 'postgres' + ]); + +# Start a pgbench run in the background against the server specified = via the +# port passed as parameter. +sub background_rw_pgbench +{ + my $port =3D shift; + + # If a previous pgbench is still running, start by shutting it = down. + $pgbench->finish if $pgbench; + + my $clients =3D 1; + my $runtime =3D 2; + + if ($extended) + { + # Randomize the number of pgbench clients a bit (range = 1-16) + $clients =3D 1 + int(rand(15)); + $runtime =3D 600; + } + my @cmd =3D ('pgbench', '-p', $port, '-T', $runtime, '-c', = $clients); + + # Randomize whether we spawn connections or not + push(@cmd, '-C') if ($extended && cointoss); + # Finally add the database name to use + push(@cmd, 'postgres'); + + $pgbench =3D IPC::Run::start( + \@cmd, + '<' =3D> '/dev/null', + '>' =3D> '/dev/null', + '2>' =3D> '/dev/null', + = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default)); +} + +# Test checksum transition. The function has these arguments: +# +# - start checksum state (enabled/disabled) +# - first - first checksum change +# - second - second checksum change +# - point - injection point the first change should wait on +# - final - expected checksum state at the end +# +# The test puts the instance into the initial checksum state, triggers = two +# checksum changes, and verifies the final state is as expected. The = first +# state change is paused on a selected injection point, and unpaused = after +# the second change gets initiated. +# +# The injection point is triggered only by the datachecksum launcher, = and +# there can be only one such process. So there's no risk of hitting the +# injection point by both changes. +sub test_checksum_transition +{ + my ($start, $first, $second, $point, $final) =3D @_; + + $node->safe_psql('postgres', + "SELECT '=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D " . $start . " / = " . $first . " / " . $second . " / " . $point . " / " . $final . " = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D'"); + + note($start . " / " . $first . " / " . $second . " / " . $point = . " / " . $final); + + note('changing checksums into initial state: ' . $start); + + enable_data_checksums($node, wait =3D> 'on') if ($start eq = 'enabled'); + disable_data_checksums($node, wait =3D> 'off') if ($start eq = 'disabled'); + + note('attaching injection point: ' . $point); + $node->safe_psql('postgres', + "SELECT injection_points_attach('" . $point . = "','wait');" + ); + + note("triggering first checksum change: " . $first); + + enable_data_checksums($node) if ($first eq 'enable'); + disable_data_checksums($node) if ($first eq 'disable'); + + note("waiting for the injection point to be hit"); + $node->poll_query_until( + 'postgres', + "SELECT COUNT(*) FROM pg_catalog.pg_stat_activity WHERE = wait_event =3D '" . $point . "'", + '1'); + + note("triggering second checksum change: " . $second); + + enable_data_checksums($node) if ($second eq 'enable'); + disable_data_checksums($node) if ($second eq 'disable'); + + note("waking and detaching injection point"); + $node->safe_psql('postgres', + "SELECT injection_points_wakeup('" . $point . "');"); + + note("detaching injection point"); + $node->safe_psql('postgres', + "SELECT injection_points_detach('" . $point . "');"); + + note('wait for the checksum launcher to exit'); + $node->poll_query_until('postgres', + "SELECT count(*) =3D 0 " + . "FROM pg_catalog.pg_stat_activity " + . "WHERE backend_type =3D 'datachecksum launcher';"); + + test_checksum_state($node, $final); + + # Since the log isn't being written to now, parse the log and = check + # for instances of checksum verification failures. + my $log =3D PostgreSQL::Test::Utils::slurp_file($node->logfile, + $node_loglocation); + unlike( + $log, + qr/page verification failed,.+\d$/, + "no checksum validation errors in primary log (during = WAL recovery)" + ); + $node_loglocation =3D -s $node->logfile; +} + +# Start the test suite with pgbench running. +background_rw_pgbench($node->port); + +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-inprogress-checksums-delay', 'off'); +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-inprogress-checksums-after-xlogctl', 'off'); +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-inprogress-checksums-after-controlfile', 'off'); +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-checksums-delay', 'off'); +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-checksums-after-xlogctl', 'off'); +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-checksums-after-controlfile', 'off'); +test_checksum_transition('disabled', 'enable', 'disable', = 'datachecksums-enable-checksums-after-checkpoint', 'off'); + +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-inprogress-checksums-delay', 'on'); +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-inprogress-checksums-after-xlogctl', 'on'); +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-inprogress-checksums-after-controlfile', 'on'); +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-checksums-delay', 'on'); +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-checksums-after-xlogctl', 'on'); +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-checksums-after-controlfile', 'on'); +test_checksum_transition('disabled', 'enable', 'enable', = 'datachecksums-enable-checksums-after-checkpoint', 'on'); + +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-inprogress-checksums-delay', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-inprogress-checksums-after-xlogctl', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-inprogress-checksums-after-controlfile', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-inprogress-checksums-after-checkpoint', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-checksums-delay', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-checksums-after-xlogctl', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-checksums-after-controlfile', 'off'); +test_checksum_transition('enabled', 'disable', 'disable', = 'datachecksums-disable-checksums-after-checkpoint', 'off'); + +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-inprogress-checksums-delay', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-inprogress-checksums-after-xlogctl', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-inprogress-checksums-after-controlfile', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-inprogress-checksums-after-checkpoint', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-checksums-delay', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-checksums-after-xlogctl', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-checksums-after-controlfile', 'on'); +test_checksum_transition('enabled', 'disable', 'enable', = 'datachecksums-disable-checksums-after-checkpoint', 'on'); + +$node->stop; +done_testing(); diff --git = a/src/test/modules/test_checksums/t/011_injection_checkpoint.pl = b/src/test/modules/test_checksums/t/011_injection_checkpoint.pl new file mode 100644 index 00000000000..ce1553e8ecd --- /dev/null +++ b/src/test/modules/test_checksums/t/011_injection_checkpoint.pl @@ -0,0 +1,196 @@ + +# Copyright (c) 2026, PostgreSQL Global Development Group + +# Test suite for testing enabling data checksums in an online cluster = with +# injection point tests injecting failures into the processing + +use strict; +use warnings FATAL =3D> 'all'; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +use FindBin; +use lib $FindBin::RealBin; + +use DataChecksums::Utils; + +# This test suite is expensive, or very expensive, to execute. There = are two +# PG_TEST_EXTRA options for running it, "checksum" for a pared-down = test suite +# an "checksum_extended" for the full suite. The full suite can run = for hours +# on slow or constrained systems. +my $extended =3D undef; +if ($ENV{PG_TEST_EXTRA}) +{ + $extended =3D 1 if ($ENV{PG_TEST_EXTRA} =3D~ = /\bchecksum_extended\b/); + plan skip_all =3D> 'Expensive data checksums test disabled' + unless ($ENV{PG_TEST_EXTRA} =3D~ /\bchecksum(_extended)?\b/); +} + +if ($ENV{enable_injection_points} ne 'yes') +{ + plan skip_all =3D> 'Injection points not supported by this = build'; +} + +# = --------------------------------------------------------------------------= - +# Test cluster setup +# + +# Initiate testcluster +my $node =3D PostgreSQL::Test::Cluster->new('injection_node'); +$node->init(no_data_checksums =3D> 1); +$node->start; + +# Set up test environment +$node->safe_psql('postgres', 'CREATE EXTENSION test_checksums;'); +$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;'); + +my $pgbench =3D undef; +my $scalefactor =3D ($extended ? 10 : 1); +my $node_loglocation =3D 0; + +$node->command_ok( + [ + 'pgbench', '-p', $node->port, '-i', + '-s', $scalefactor, '-q', 'postgres' + ]); + +# Start a pgbench run in the background against the server specified = via the +# port passed as parameter. +sub background_rw_pgbench +{ + my $port =3D shift; + + # If a previous pgbench is still running, start by shutting it = down. + $pgbench->finish if $pgbench; + + my $clients =3D 1; + my $runtime =3D 2; + + if ($extended) + { + # Randomize the number of pgbench clients a bit (range = 1-16) + $clients =3D 1 + int(rand(15)); + $runtime =3D 600; + } + my @cmd =3D ('pgbench', '-p', $port, '-T', $runtime, '-c', = $clients); + + # Randomize whether we spawn connections or not + push(@cmd, '-C') if ($extended && cointoss); + # Finally add the database name to use + push(@cmd, 'postgres'); + + $pgbench =3D IPC::Run::start( + \@cmd, + '<' =3D> '/dev/null', + '>' =3D> '/dev/null', + '2>' =3D> '/dev/null', + = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default)); +} + +# Test checksum transition concurrent with a checkpoint. +# +# The function has these arguments: +# +# - start checksum state (enabled/disabled) +# - change - checksum change to initiate +# - point - injection point the first change should wait on +# - final - expected checksum state at the end +# +# The test puts the instance into the initial checksum state, triggers = a +# checksum change concurrent with a checkpoint, and verifies the final = state +# is as expected. The state change is paused on a selected injection +# point, and unpaused after performing a checkpoint. +# +# Finally, the instance is restarted (in either fast ot immediate = mode), +# the final checksum state is validated against the expected value, and +# the server log is checked for checksum failures. +sub test_checksum_transition +{ + my ($start, $change, $point, $final) =3D @_; + + # Start the test suite with pgbench running. + background_rw_pgbench($node->port); + + $node->safe_psql('postgres', + "SELECT '=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D " . $start . " / = " . $change . " / " . $point . " / " . $final . " =3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D'"); + + note($start . " / " . $change . " / " . $point . " / " . = $final); + + note('changing checksums into initial state: ' . $start); + + enable_data_checksums($node, wait =3D> 'on') if ($start eq = 'enabled'); + disable_data_checksums($node, wait =3D> 'off') if ($start eq = 'disabled'); + + note('attaching injection point: ' . $point); + $node->safe_psql('postgres', + "SELECT injection_points_attach('" . $point . = "','wait');" + ); + + note("triggering checksum change: " . $change); + + enable_data_checksums($node) if ($change eq 'enable'); + disable_data_checksums($node) if ($change eq 'disable'); + + note("waiting for the injection point to be hit"); + $node->poll_query_until( + 'postgres', + "SELECT COUNT(*) FROM pg_catalog.pg_stat_activity WHERE = wait_event =3D '" . $point . "'", + '1'); + + note('checkpoint'); + $node->safe_psql('postgres', "CHECKPOINT"); + + note("waking and detaching injection point"); + $node->safe_psql('postgres', + "SELECT injection_points_wakeup('" . $point . "');"); + + note("detaching injection point"); + $node->safe_psql('postgres', + "SELECT injection_points_detach('" . $point . "');"); + + note('wait for the checksum launcher to exit'); + $node->poll_query_until('postgres', + "SELECT count(*) =3D 0 " + . "FROM pg_catalog.pg_stat_activity " + . "WHERE backend_type =3D 'datachecksum launcher';"); + + test_checksum_state($node, $final); + + $node->stop(stopmode()); + $node->start; + + test_checksum_state($node, $final); + + # Since the log isn't being written to now, parse the log and = check + # for instances of checksum verification failures. + my $log =3D PostgreSQL::Test::Utils::slurp_file($node->logfile, + $node_loglocation); + unlike( + $log, + qr/page verification failed,.+\d$/, + "no checksum validation errors in primary log (during = WAL recovery)" + ); + $node_loglocation =3D -s $node->logfile; +} + +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-inprogress-checksums-delay', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-inprogress-checksums-after-xlogctl', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-inprogress-checksums-after-controlfile', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-delay', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-after-xlogctl', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-after-controlfile', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-after-checkpoint', 'on'); + +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-delay', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-after-xlogctl', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-after-controlfile', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-after-checkpoint', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-delay', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-after-xlogctl', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-after-controlfile', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-after-checkpoint', 'off'); + +$node->stop; +done_testing(); diff --git = a/src/test/modules/test_checksums/t/012_injection_checkpoint_crash.pl = b/src/test/modules/test_checksums/t/012_injection_checkpoint_crash.pl new file mode 100644 index 00000000000..98e9c805c41 --- /dev/null +++ = b/src/test/modules/test_checksums/t/012_injection_checkpoint_crash.pl @@ -0,0 +1,214 @@ + +# Copyright (c) 2026, PostgreSQL Global Development Group + +# Test suite for testing enabling data checksums in an online cluster = with +# injection point tests injecting failures into the processing + +use strict; +use warnings FATAL =3D> 'all'; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +use FindBin; +use lib $FindBin::RealBin; + +use DataChecksums::Utils; + +# This test suite is expensive, or very expensive, to execute. There = are two +# PG_TEST_EXTRA options for running it, "checksum" for a pared-down = test suite +# an "checksum_extended" for the full suite. The full suite can run = for hours +# on slow or constrained systems. +my $extended =3D undef; +if ($ENV{PG_TEST_EXTRA}) +{ + $extended =3D 1 if ($ENV{PG_TEST_EXTRA} =3D~ = /\bchecksum_extended\b/); + plan skip_all =3D> 'Expensive data checksums test disabled' + unless ($ENV{PG_TEST_EXTRA} =3D~ /\bchecksum(_extended)?\b/); +} + +if ($ENV{enable_injection_points} ne 'yes') +{ + plan skip_all =3D> 'Injection points not supported by this = build'; +} + +# = --------------------------------------------------------------------------= - +# Test cluster setup +# + +# Initiate testcluster +my $node =3D PostgreSQL::Test::Cluster->new('injection_node'); +$node->init(no_data_checksums =3D> 1); +$node->start; + +# Set up test environment +$node->safe_psql('postgres', 'CREATE EXTENSION test_checksums;'); +$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;'); + +my $pgbench =3D undef; +my $scalefactor =3D ($extended ? 10 : 1); +my $node_loglocation =3D 0; + +$node->command_ok( + [ + 'pgbench', '-p', $node->port, '-i', + '-s', $scalefactor, '-q', 'postgres' + ]); + +# Start a pgbench run in the background against the server specified = via the +# port passed as parameter. +sub background_rw_pgbench +{ + my $port =3D shift; + + # If a previous pgbench is still running, start by shutting it = down. + $pgbench->finish if $pgbench; + + my $clients =3D 1; + my $runtime =3D 2; + + if ($extended) + { + # Randomize the number of pgbench clients a bit (range = 1-16) + $clients =3D 1 + int(rand(15)); + $runtime =3D 600; + } + my @cmd =3D ('pgbench', '-p', $port, '-T', $runtime, '-c', = $clients); + + # Randomize whether we spawn connections or not + push(@cmd, '-C') if ($extended && cointoss); + # Finally add the database name to use + push(@cmd, 'postgres'); + + $pgbench =3D IPC::Run::start( + \@cmd, + '<' =3D> '/dev/null', + '>' =3D> '/dev/null', + '2>' =3D> '/dev/null', + = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default)); +} + +# Test checksum transition concurrent with a checkpoint. +# +# The function has these arguments: +# +# - start checksum state (enabled/disabled) +# - change - checksum change to initiate +# - point1 - injection point before checkpoint +# - point2 - injection point after checkpoint +# - final - expected checksum state at the end +# +# The test puts the instance into the initial checksum state, triggers = a +# checksum change that pauses on a selected injection point. Then = performs +# a checkpoint, unpauses the change so that it proceeds to a second +# injection point. +# +# Then the instance is restarted in immediate mode to simulate failure, +# and the final checksum state is validated against the expected value. +# The server log is checked for checksum failures. +sub test_checksum_transition +{ + my ($start, $change, $point1, $point2, $final) =3D @_; + + # Start the test suite with pgbench running. + background_rw_pgbench($node->port); + + $node->safe_psql('postgres', + "SELECT '=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D " . $start . " / = " . $change . " / " . $point1 . " / " . $final . " =3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D'"); + + note($start . " / " . $change . " / " . $point1 . " / " . = $final); + + note('changing checksums into initial state: ' . $start); + + enable_data_checksums($node, wait =3D> 'on') if ($start eq = 'enabled'); + disable_data_checksums($node, wait =3D> 'off') if ($start eq = 'disabled'); + + note('attaching injection point: ' . $point1); + $node->safe_psql('postgres', + "SELECT injection_points_attach('" . $point1 . = "','wait');" + ); + + if (defined($point2)) + { + note('attaching injection point: ' . $point2); + $node->safe_psql('postgres', + "SELECT injection_points_attach('" . $point2 . = "','wait');" + ); + } + + note("triggering checksum change: " . $change); + + enable_data_checksums($node) if ($change eq 'enable'); + disable_data_checksums($node) if ($change eq 'disable'); + + note("waiting for the injection point to be hit"); + $node->poll_query_until( + 'postgres', + "SELECT COUNT(*) FROM pg_catalog.pg_stat_activity WHERE = wait_event =3D '" . $point1 . "'", + '1'); + + note('checkpoint'); + $node->safe_psql('postgres', "CHECKPOINT"); + + note("waking and detaching injection point"); + $node->safe_psql('postgres', + "SELECT injection_points_wakeup('" . $point1 . "');"); + + note("detaching injection point"); + $node->safe_psql('postgres', + "SELECT injection_points_detach('" . $point1 . "');"); + + if (defined($point2)) + { + note("waiting for the injection point to be hit"); + $node->poll_query_until( + 'postgres', + "SELECT COUNT(*) FROM = pg_catalog.pg_stat_activity WHERE wait_event =3D '" . $point2 . "'", + '1'); + } + else + { + note('wait for the checksum launcher to exit'); + $node->poll_query_until('postgres', + "SELECT count(*) =3D 0 " + . "FROM pg_catalog.pg_stat_activity " + . "WHERE backend_type =3D 'datachecksum = launcher';"); + } + + $node->stop('immediate'); + $node->start; + + test_checksum_state($node, $final); + + # Since the log isn't being written to now, parse the log and = check + # for instances of checksum verification failures. + my $log =3D PostgreSQL::Test::Utils::slurp_file($node->logfile, + $node_loglocation); + unlike( + $log, + qr/page verification failed,.+\d$/, + "no checksum validation errors in primary log (during = WAL recovery)" + ); + $node_loglocation =3D -s $node->logfile; +} + +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-inprogress-checksums-delay', = 'datachecksums-enable-inprogress-checksums-after-xlogctl', 'off'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-inprogress-checksums-after-xlogctl', = 'datachecksums-enable-inprogress-checksums-after-controlfile', 'off'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-inprogress-checksums-after-controlfile', = 'datachecksums-enable-checksums-delay', 'off'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-delay', = 'datachecksums-enable-checksums-after-xlogctl', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-after-xlogctl', = 'datachecksums-enable-checksums-after-controlfile', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-after-controlfile', = 'datachecksums-enable-checksums-after-checkpoint', 'on'); +test_checksum_transition('disabled', 'enable', = 'datachecksums-enable-checksums-after-checkpoint', undef, 'on'); + +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-delay', = 'datachecksums-disable-inprogress-checksums-after-xlogctl', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-after-xlogctl', = 'datachecksums-disable-inprogress-checksums-after-controlfile', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-after-controlfile', = 'datachecksums-disable-inprogress-checksums-after-checkpoint', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-inprogress-checksums-after-checkpoint', = 'datachecksums-disable-checksums-delay', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-delay', = 'datachecksums-disable-checksums-after-xlogctl', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-after-xlogctl', = 'datachecksums-disable-checksums-after-controlfile', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-after-controlfile', = 'datachecksums-disable-checksums-after-checkpoint', 'off'); +test_checksum_transition('enabled', 'disable', = 'datachecksums-disable-checksums-after-checkpoint', undef, 'off'); + +$node->stop; +done_testing(); --Apple-Mail=_51E97F6A-2C64-41FA-B26A-C447572F466F--