Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1w8gWT-000mJl-0B for pgsql-hackers@arkaria.postgresql.org; Fri, 03 Apr 2026 15:33:26 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1w8gWR-00CGU5-2x for pgsql-hackers@arkaria.postgresql.org; Fri, 03 Apr 2026 15:33:24 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1w8gWQ-00CGTw-2Z for pgsql-hackers@lists.postgresql.org; Fri, 03 Apr 2026 15:33:23 +0000 Received: from smtp.outgoing.loopia.se ([93.188.3.37]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1w8gWM-00000000NF2-1uTD for pgsql-hackers@lists.postgresql.org; Fri, 03 Apr 2026 15:33:22 +0000 Received: from s807.loopia.se (localhost [127.0.0.1]) by s807.loopia.se (Postfix) with ESMTP id C19CA5A2A34 for ; Fri, 03 Apr 2026 17:33:14 +0200 (CEST) Received: from s980.loopia.se (unknown [172.22.191.6]) by s807.loopia.se (Postfix) with ESMTP id B07A95A2348; Fri, 03 Apr 2026 17:33:14 +0200 (CEST) Received: from localhost (unknown [172.22.191.6]) by s980.loopia.se (Postfix) with ESMTP id AC1D72201565; Fri, 03 Apr 2026 17:33:14 +0200 (CEST) X-Virus-Scanned: amavis at amavis.loopia.se X-Spam-Flag: NO X-Spam-Score: -1.2 X-Spam-Level: X-Spam-Status: No, score=-1.2 tagged_above=-999 required=6.2 tests=[ALL_TRUSTED=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1] autolearn=disabled Authentication-Results: s474.loopia.se (amavis); dkim=pass (2048-bit key) header.d=yesql.se Received: from s980.loopia.se ([172.22.191.6]) by localhost (s474.loopia.se [172.22.190.14]) (amavis, port 10024) with LMTP id aG9B38J4PeZ0; Fri, 3 Apr 2026 17:33:14 +0200 (CEST) X-Loopia-Auth: user X-Loopia-User: daniel@yesql.se X-Loopia-Originating-IP: 89.255.232.236 Received: from smtpclient.apple (customer-89-255-232-236.stosn.net [89.255.232.236]) (Authenticated sender: daniel@yesql.se) by s980.loopia.se (Postfix) with ESMTPSA id C457B220155F; Fri, 03 Apr 2026 17:33:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yesql.se; s=loopiadkim1707475645; t=1775230393; bh=RSuf3TiFLM7USpqcFK0TdOIzlK8i5I0FRPCq6k5xQL4=; h=From:Subject:Date:In-Reply-To:Cc:To:References; b=pH+isF4t4YjlGD2aiJhGM6D15ty7wc2zrctA0ObJclmGQzVbTkqT78lgybg1BbOmH zYPUP/7e9fvYgsmR52tqMItzVT3HXfrfeWiw0ts734cPPbKhznksMrIz2v5IDCnnZ9 nRangdH8hCh25vjmCdpQ6q90cO6PuJms+Ekf28qeQvFSaZMhiSyR17+Te63gerWcvg O7YV5GyON/HN0W6qeHQdcC289bEb3dZ7MhPY3sO+AVtyKbo0lhATA7DYuxNs4ii7vz rwbIkq2gijjp/YzqDeBolT8a8Aif86eWqCCibobDHXjHvwUIaOU4acxFwz5DtHAYMG dpGfiwcKtBzmA== From: Daniel Gustafsson Message-Id: Content-Type: multipart/mixed; boundary="Apple-Mail=_E4DE5431-D9EB-4BD5-9E42-507D14F7721A" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.2\)) Subject: Re: Changing the state of data checksums in a running cluster Date: Fri, 3 Apr 2026 17:33:03 +0200 In-Reply-To: Cc: Tomas Vondra , Andres Freund , Bernd Helmle , Michael Paquier , Michael Banck , PostgreSQL Hackers To: Heikki Linnakangas References: <7357A519-3391-4F07-BC69-5C9C2B120792@yesql.se> <477897AE-1314-4724-9694-0BABC4F4ABDA@yesql.se> <5mv5vtbqj2xm2wmevsi22smyn32jtznva3ya3daih3ixh3onex@s5fyab63xt3c> <2535E72D-5979-4F58-A2D3-2AA4AA335C20@yesql.se> <775fddbc-4dec-413d-89bb-1a8186858ee6@vondra.me> <683618b7-0946-400e-8066-97067f32d448@vondra.me> <032619A2-D466-4A12-A524-98359D96AEA6@yesql.se> <300822bd-daeb-49e5-8af7-d7ce3af65adc@iki.fi> <46d52594-fa93-4fbe-ad97-dc3127a04dd8@iki.fi> <8B86AFB8-ACBC-44FC-A1AB-A58DB541A170@yesql.se> <452D5390-4F89-4DED-9835-8E9EA2A01E19@yesql.se> <4020B0AE-6286-462F-BD50-3AAD170F08C9@yesql.se> <30E2C103-2F50-4F9D-8868-E057250F55CA@yesql.se> X-Mailer: Apple Mail (2.3776.700.51.11.2) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_E4DE5431-D9EB-4BD5-9E42-507D14F7721A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii The attached rebase with a PG_CONTROL_VERSION bump is what I have staged = for later tonight, submitting here to have the (hopefully) final patch = archived as well as another CFBot run. -- Daniel Gustafsson --Apple-Mail=_E4DE5431-D9EB-4BD5-9E42-507D14F7721A Content-Disposition: attachment; filename=v20260403-0001-Online-enabling-and-disabling-of-data-chec.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v20260403-0001-Online-enabling-and-disabling-of-data-chec.patch" Content-Transfer-Encoding: quoted-printable =46rom=20b327a450f0e116853dee027adcb98a748f49f23c=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Fri,=203=20Apr=202026=2017:00:32=20= +0200=0ASubject:=20[PATCH=20v20260403]=20Online=20enabling=20and=20= disabling=20of=20data=20checksums=0A=0AThis=20allows=20data=20checksums=20= to=20be=20enabled,=20or=20disabled,=20in=20a=20running=0Acluster=20= without=20restricting=20access=20to=20the=20cluster=20during=20= processing.=0A=0AData=20checksums=20could=20prior=20to=20this=20only=20= be=20enabled=20during=20initdb=20or=0Awhen=20the=20cluster=20is=20= offline=20using=20the=20pg_checksums=20app.=20This=20commit=0Aintroduce=20= functionality=20to=20enable,=20or=20disable,=20data=20checksums=20while=0A= the=20cluster=20is=20running=20regardless=20of=20how=20it=20was=20= initialized.=0A=0AA=20background=20worker=20launcher=20process=20is=20= responsible=20for=20launching=20a=0Adynamic=20per-database=20background=20= worker=20which=20will=20mark=20all=20buffers=0Adirty=20for=20all=20= relation=20with=20storage=20in=20order=20for=20them=20to=20have=20data=0A= checksums=20calculated=20on=20write.=20=20Once=20all=20relations=20in=20= all=20databases=0Ahave=20been=20processed,=20the=20data_checksums=20= state=20will=20be=20set=20to=20on=20and=0Athe=20cluster=20will=20at=20= that=20point=20be=20identical=20to=20one=20which=20had=20data=0A= checksums=20enabled=20during=20initialization=20or=20via=20offline=20= processing.=0A=0AWhen=20data=20checksums=20are=20being=20enabled,=20= concurrent=20I/O=20operations=0Afrom=20backends=20other=20than=20the=20= data=20checksums=20worker=20will=20write=20the=0Achecksums=20but=20not=20= verify=20them=20on=20reading.=20=20Only=20when=20all=20backends=0Ahave=20= absorbed=20the=20procsignalbarrier=20for=20setting=20data_checksums=20to=0A= on=20will=20they=20also=20start=20verifying=20checksums=20on=20reading.=20= =20The=20same=0Aprocess=20is=20repeated=20during=20disabling;=20all=20= backends=20write=20checksums=0Abut=20do=20not=20verify=20them=20until=20= the=20barrier=20for=20setting=20the=20state=20to=0Aoff=20has=20been=20= absorbed=20by=20all.=20=20This=20in-progress=20state=20is=20used=20to=0A= ensure=20there=20are=20no=20false=20negatives=20(or=20positives)=20due=20= to=20reading=0Aa=20checksum=20which=20is=20not=20in=20sync=20with=20the=20= page.=0A=0AA=20new=20testmodule,=20test_checksums,=20is=20introduced=20= with=20an=20extensive=0Aset=20of=20tests=20covering=20both=20online=20= and=20offline=20data=20checksum=20mode=0Achanges.=20=20The=20tests=20= which=20run=20concurrent=20pgbdench=20during=20online=0Aprocessing=20are=20= gated=20behind=20the=20PG_TEST_EXTRA=20flag=20due=20to=20being=0Avery=20= expensive=20to=20run.=20=20Two=20levels=20of=20PG_TEST_EXTRA=20flags=20= exist=0Ato=20turn=20on=20a=20subset=20of=20the=20expensive=20tests,=20or=20= the=20full=20suite=20of=0Amultiple=20runs.=0A=0AThis=20work=20is=20based=20= on=20an=20earlier=20version=20of=20this=20patch=20which=20was=0Areviewed=20= by=20among=20others=20Heikki=20Linnakangas,=20Robert=20Haas,=20Andres=0A= Freund,=20Tomas=20Vondra,=20Michael=20Banck=20and=20Andrey=20Borodin.=20=20= During=0Athe=20work=20on=20this=20new=20version,=20Tomas=20Vondra=20has=20= given=20invaluable=0Aassistance=20with=20not=20only=20coding=20and=20= reviewing=20but=20very=20in-depth=0Atesting.=0A=0AAuthor:=20Daniel=20= Gustafsson=20=0AAuthor:=20Magnus=20Hagander=20= =0ACo-authored-by:=20Tomas=20Vondra=20= =0AReviewed-by:=20Tomas=20Vondra=20=0A= Reviewed-by:=20Andres=20Freund=20=0AReviewed-by:=20= Heikki=20Linnakangas=20=0ADiscussion:=20= https://postgr.es/m/CABUevExz9hUUOLnJVr2kpw9Cx=3Do4MCr1SVKwbupzuxP7ckNutA@= mail.gmail.com=0ADiscussion:=20= https://postgr.es/m/20181030051643.elbxjww5jjgnjaxg@alap3.anarazel.de=0A= Discussion:=20= https://postgr.es/m/CABUevEwE3urLtwxxqdgd5O2oQz9J717ZzMbh+ziCSa5YLLU_BA@ma= il.gmail.com=0A---=0A=20doc/src/sgml/config.sgml=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= doc/src/sgml/func/func-admin.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=2078=20+=0A=20doc/src/sgml/glossary.sgml=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20|=20=20=2024=20+=0A=20= doc/src/sgml/images/Makefile=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20|=20=20=20=201=20+=0A=20doc/src/sgml/images/datachecksums.gv=20=20= =20=20=20=20=20=20=20=20|=20=20=2014=20+=0A=20= doc/src/sgml/images/datachecksums.svg=20=20=20=20=20=20=20=20=20|=20=20=20= 81=20+=0A=20doc/src/sgml/monitoring.sgml=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20|=20=20228=20++-=0A=20= doc/src/sgml/ref/pg_checksums.sgml=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=20=206=20+=0A=20doc/src/sgml/regress.sgml=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20|=20=20=2014=20+=0A=20= doc/src/sgml/wal.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20|=20=20126=20+-=0A=20= src/backend/access/rmgrdesc/xlogdesc.c=20=20=20=20=20=20=20=20|=20=20=20= 58=20+-=0A=20src/backend/access/transam/xlog.c=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20502=20++++-=0A=20src/backend/backup/basebackup.c=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=2031=20+-=0A=20= src/backend/catalog/system_views.sql=20=20=20=20=20=20=20=20=20=20|=20=20= =2019=20+=0A=20src/backend/commands/dbcommands.c=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20=20=207=20+=0A=20src/backend/postmaster/Makefile=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/backend/postmaster/auxprocess.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =2019=20+=0A=20src/backend/postmaster/bgworker.c=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20=2010=20+-=0A=20= src/backend/postmaster/datachecksum_state.c=20=20=20|=201610=20= +++++++++++++++++=0A=20src/backend/postmaster/meson.build=20=20=20=20=20=20= =20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/backend/postmaster/postmaster.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =20=205=20+=0A=20src/backend/replication/logical/decode.c=20=20=20=20=20=20= |=20=20=2016=20+=0A=20src/backend/storage/buffer/bufmgr.c=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=207=20+=0A=20src/backend/storage/ipc/ipci.c=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=203=20+=0A=20= src/backend/storage/ipc/procsignal.c=20=20=20=20=20=20=20=20=20=20|=20=20= =20=208=20+=0A=20src/backend/storage/page/README=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=20=20=204=20+-=0A=20= src/backend/storage/page/bufpage.c=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=2023=20+-=0A=20src/backend/utils/activity/pgstat_backend.c=20=20=20|=20= =20=20=202=20+=0A=20src/backend/utils/activity/pgstat_io.c=20=20=20=20=20= =20=20=20|=20=20=20=202=20+=0A=20.../utils/activity/wait_event_names.txt=20= =20=20=20=20=20=20|=20=20=20=203=20+=0A=20= src/backend/utils/adt/pgstatfuncs.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =20=208=20+-=0A=20src/backend/utils/init/miscinit.c=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=203=20+-=0A=20= src/backend/utils/init/postinit.c=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=2020=20+-=0A=20src/backend/utils/misc/guc_parameters.dat=20=20=20= =20=20|=20=20=20=205=20+-=0A=20src/backend/utils/misc/guc_tables.c=20=20=20= =20=20=20=20=20=20=20=20|=20=20=20=209=20+-=0A=20= src/backend/utils/misc/postgresql.conf.sample=20|=20=20=2010=20+-=0A=20= src/bin/pg_checksums/pg_checksums.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =20=204=20+-=0A=20src/bin/pg_controldata/pg_controldata.c=20=20=20=20=20=20= =20|=20=20=20=202=20+=0A=20src/bin/pg_upgrade/controldata.c=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=20=20=20=209=20+=0A=20= src/bin/pg_waldump/t/001_basic.pl=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=203=20+-=0A=20src/include/access/rmgrlist.h=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/include/access/xlog.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=2017=20+-=0A=20src/include/access/xlog_internal.h=20= =20=20=20=20=20=20=20=20=20=20=20|=20=20=20=208=20+=0A=20= src/include/catalog/pg_control.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=208=20+-=0A=20src/include/catalog/pg_proc.dat=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=20=20=2014=20+=0A=20= src/include/commands/progress.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20|=20=20=2016=20+=0A=20src/include/miscadmin.h=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=206=20+=0A=20= src/include/postmaster/datachecksum_state.h=20=20=20|=20=20=2058=20+=0A=20= src/include/postmaster/proctypelist.h=20=20=20=20=20=20=20=20=20|=20=20=20= =202=20+=0A=20src/include/replication/decode.h=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=201=20+=0A=20src/include/storage/bufpage.h=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=203=20+-=0A=20= src/include/storage/checksum.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20|=20=20=2014=20+=0A=20src/include/storage/lwlocklist.h=20=20=20=20=20=20= =20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/include/storage/procsignal.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=204=20+=0A=20src/include/utils/backend_progress.h=20=20=20=20=20= =20=20=20=20=20|=20=20=20=201=20+=0A=20src/test/modules/Makefile=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A= =20src/test/modules/meson.build=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20=20=201=20+=0A=20= src/test/modules/test_checksums/.gitignore=20=20=20=20|=20=20=20=202=20+=0A= =20src/test/modules/test_checksums/Makefile=20=20=20=20=20=20|=20=20=20= 40=20+=0A=20src/test/modules/test_checksums/README=20=20=20=20=20=20=20=20= |=20=20=2030=20+=0A=20src/test/modules/test_checksums/meson.build=20=20=20= |=20=20=2038=20+=0A=20.../modules/test_checksums/t/001_basic.pl=20=20=20=20= =20|=20=20=2063=20+=0A=20.../modules/test_checksums/t/002_restarts.pl=20=20= |=20=20110=20++=0A=20.../test_checksums/t/003_standby_restarts.pl=20=20|=20= =20114=20++=0A=20.../modules/test_checksums/t/004_offline.pl=20=20=20|=20= =20=2082=20+=0A=20.../modules/test_checksums/t/005_injection.pl=20|=20=20= =2074=20+=0A=20.../test_checksums/t/006_pgbench_single.pl=20=20=20=20|=20= =20275=20+++=0A=20.../test_checksums/t/007_pgbench_standby.pl=20=20=20|=20= =20400=20++++=0A=20src/test/modules/test_checksums/t/008_pitr.pl=20|=20=20= 189=20++=0A=20src/test/modules/test_checksums/t/009_fpi.pl=20=20|=20=20=20= 64=20+=0A=20.../test_checksums/t/DataChecksums/Utils.pm=20=20=20|=20=20= 262=20+++=0A=20.../test_checksums/test_checksums--1.0.sql=20=20=20=20|=20= =20=2024=20+=0A=20.../modules/test_checksums/test_checksums.c=20=20=20|=20= =20184=20++=0A=20.../test_checksums/test_checksums.control=20=20=20=20=20= |=20=20=20=204=20+=0A=20src/test/perl/PostgreSQL/Test/Cluster.pm=20=20=20= =20=20=20|=20=20=2036=20+=0A=20src/test/regress/expected/rules.out=20=20=20= =20=20=20=20=20=20=20=20|=20=20=2035=20+=0A=20= src/test/regress/expected/stats.out=20=20=20=20=20=20=20=20=20=20=20|=20=20= =2018=20+-=0A=20src/tools/pgindent/typedefs.list=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=207=20+=0A=2078=20files=20changed,=205127=20= insertions(+),=2057=20deletions(-)=0A=20create=20mode=20100644=20= doc/src/sgml/images/datachecksums.gv=0A=20create=20mode=20100644=20= doc/src/sgml/images/datachecksums.svg=0A=20create=20mode=20100644=20= src/backend/postmaster/datachecksum_state.c=0A=20create=20mode=20100644=20= src/include/postmaster/datachecksum_state.h=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/.gitignore=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/Makefile=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/README=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/meson.build=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/t/001_basic.pl=0A=20create=20mode=20= 100644=20src/test/modules/test_checksums/t/002_restarts.pl=0A=20create=20= mode=20100644=20= src/test/modules/test_checksums/t/003_standby_restarts.pl=0A=20create=20= mode=20100644=20src/test/modules/test_checksums/t/004_offline.pl=0A=20= create=20mode=20100644=20= src/test/modules/test_checksums/t/005_injection.pl=0A=20create=20mode=20= 100644=20src/test/modules/test_checksums/t/006_pgbench_single.pl=0A=20= create=20mode=20100644=20= src/test/modules/test_checksums/t/007_pgbench_standby.pl=0A=20create=20= mode=20100644=20src/test/modules/test_checksums/t/008_pitr.pl=0A=20= create=20mode=20100644=20src/test/modules/test_checksums/t/009_fpi.pl=0A=20= create=20mode=20100644=20= src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0A=20create=20= mode=20100644=20src/test/modules/test_checksums/test_checksums--1.0.sql=0A= =20create=20mode=20100644=20= src/test/modules/test_checksums/test_checksums.c=0A=20create=20mode=20= 100644=20src/test/modules/test_checksums/test_checksums.control=0A=0A= diff=20--git=20a/doc/src/sgml/config.sgml=20b/doc/src/sgml/config.sgml=0A= index=20fdb77df0fdb..a5b16625c3c=20100644=0A---=20= a/doc/src/sgml/config.sgml=0A+++=20b/doc/src/sgml/config.sgml=0A@@=20= -7187,6=20+7187,7=20@@=20local0.*=20=20=20=20/var/log/postgresql=0A=20=20= =20=20=20=20=20=20=20=20bgworker=0A=20= =20=20=20=20=20=20=20=20=20bgwriter=0A= =20=20=20=20=20=20=20=20=20=20= checkpointer=0A+=20=20=20=20=20=20=20= =20=20checksums=0A=20=20=20=20=20=20=20= =20=20=20ioworker=0A=20=20=20=20=20=20= =20=20=20=20postmaster=0A=20=20=20=20= =20=20=20=20=20=20slotsyncworker=0A= diff=20--git=20a/doc/src/sgml/func/func-admin.sgml=20= b/doc/src/sgml/func/func-admin.sgml=0Aindex=20210b1118bdf..24ecb46542e=20= 100644=0A---=20a/doc/src/sgml/func/func-admin.sgml=0A+++=20= b/doc/src/sgml/func/func-admin.sgml=0A@@=20-3123,4=20+3123,82=20@@=20= SELECT=20convert_from(pg_read_binary_file('file_in_utf8.txt'),=20= 'UTF8');=0A=20=0A=20=20=20=0A=20=0A+=20=20=0A+=20=20=20Data=20Checksum=20= Functions=0A+=0A+=20=20=20=0A+=20=20=20=20The=20functions=20= shown=20in=20=20can=0A= +=20=20=20=20be=20used=20to=20enable=20or=20disable=20data=20checksums=20= in=20a=20running=20cluster.=0A+=20=20=20=0A+=20=20=20=0A+=20= =20=20=20Changing=20data=20checksums=20can=20be=20done=20in=20a=20= cluster=20with=20concurrent=20activity=0A+=20=20=20=20without=20blocking=20= queries,=20but=20overall=20system=20performance=20will=20be=20affected.=0A= +=20=20=20=20See=20=20for=20further=20= details=20on=20how=20changing=20the=0A+=20=20=20=20data=20checksums=20= state=20can=20affect=20a=20system=20and=20possible=20mitigations=20for=20= how=0A+=20=20=20=20to=20reduce=20the=20impact.=0A+=20=20=20=0A+=0A= +=20=20=20=0A+=20=20=20=20= Data=20Checksum=20Functions=0A+=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20Function=0A+=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20Description=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20= pg_enable_data_checksums=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20= pg_enable_data_checksums=20(=20= cost_delay=20int,=20= cost_limit=20int=20)=0A+=20= =20=20=20=20=20=20=20void=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Initiates=20the=20process=20of=20enabling=20data=20checksums=20for=20the=20= cluster.=20This=0A+=20=20=20=20=20=20=20=20will=20set=20the=20data=20= checksums=20state=20to=20inprogress-on=0A+=20=20=20=20= =20=20=20=20as=20well=20as=20start=20a=20background=20worker=20that=20= will=20process=20all=20pages=20in=20all=0A+=20=20=20=20=20=20=20=20= databases=20and=20enable=20data=20checksums=20on=20them.=20=20When=20all=20= pages=20have=0A+=20=20=20=20=20=20=20=20been=20processed,=20the=20= cluster=20will=20automatically=20set=20data=20checksums=20state=0A+=20=20= =20=20=20=20=20=20to=20on.=20=20This=20operation=20is=20= WAL=20logged=20and=20replicated=0A+=20=20=20=20=20=20=20=20to=20all=20= standby=20nodes.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20If=20cost_delay=20= and=20cost_limit=20are=0A+=20=20=20=20=20=20=20=20= specified,=20the=20process=20is=20throttled=20using=20the=20same=20= principles=20as=0A+=20=20=20=20=20=20=20=20Cost-based=20Vacuum=20= Delay.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=20=20pg_disable_data_checksums=0A+=20=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= pg_disable_data_checksums=20()=0A+=20=20=20=20=20=20= =20=20void=0A+=20=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20Disables=20data=20= checksum=20calculation=20and=20validation=20for=20the=20cluster.=20This=0A= +=20=20=20=20=20=20=20=20will=20set=20the=20data=20checksum=20state=20to=20= inprogress-off=0A+=20=20=20=20=20=20=20=20while=20= data=20checksums=20are=20being=20disabled.=20=20When=20all=20active=20= backends=20have=0A+=20=20=20=20=20=20=20=20stopped=20validating=20data=20= checksums,=20the=20data=20checksum=20state=20will=20be=0A+=20=20=20=20=20= =20=20=20set=20to=20off.=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=0A+=20=20=20=0A+=0A= +=20=20=0A+=0A=20=20=20=0Adiff=20--git=20= a/doc/src/sgml/glossary.sgml=20b/doc/src/sgml/glossary.sgml=0Aindex=20= 113d7640626..b881ae71198=20100644=0A---=20a/doc/src/sgml/glossary.sgml=0A= +++=20b/doc/src/sgml/glossary.sgml=0A@@=20-199,6=20+199,8=20@@=0A=20=20=20= =20=20=20(but=20not=20the=20autovacuum=20workers),=0A=20=20=20=20=20=20= the=20background=20= writer,=0A=20=20=20=20=20=20the=20checkpointer,=0A+=20=20=20=20= =20the=20data=20= checksums=20worker,=0A+=20=20=20=20=20the=20data=20checksums=20= worker=20launcher,=0A=20=20=20=20=20=20the=20logger,=0A=20=20=20=20=20=20the=20= startup=20= process,=0A=20=20=20=20=20=20the=20WAL=20archiver,=0A@@=20= -574,6=20+576,28=20@@=0A=20=20=20=20=0A=20=20=20=0A=20= =0A+=20=20=0A+=20=20=20= Data=20Checksums=20Worker=0A+=20=20=20=0A= +=20=20=20=20=0A+=20=20=20=20=20A=20background=20worker=0A= +=20=20=20=20=20which=20enables=20data=20checksums=20in=20a=20specific=20= database.=0A+=20=20=20=20=0A+=20=20=20=0A+=20=20= =0A+=0A+=20=20=0A+=20=20=20= Data=20Checksums=20Worker=20Launcher=0A+=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=20A=20background=20worker=0A= +=20=20=20=20=20which=20starts=20=20data=0A+=20=20=20=20=20= checksum=20worker=20processes=20for=20enabling=20data=20= checksums=20in=20each=0A+=20=20=20=20=20database,=20or=20disables=20data=20= checksums=20cluster-wide.=0A+=20=20=20=20=0A+=20=20=20=0A= +=20=20=0A+=0A=20=20=20=0A=20=20=20=20Database=0A= =20=20=20=20=0Adiff=20--git=20a/doc/src/sgml/images/Makefile=20= b/doc/src/sgml/images/Makefile=0Aindex=2038f8869d78d..7b8ac0fbb32=20= 100644=0A---=20a/doc/src/sgml/images/Makefile=0A+++=20= b/doc/src/sgml/images/Makefile=0A@@=20-3,6=20+3,7=20@@=0A=20#=20see=20= README=20in=20this=20directory=20about=20image=20handling=0A=20=0A=20= ALL_IMAGES=20=3D=20\=0A+=09datachecksums.svg=20\=0A=20=09= genetic-algorithm.svg=20\=0A=20=09gin.svg=20\=0A=20=09pagelayout.svg=20\=0A= diff=20--git=20a/doc/src/sgml/images/datachecksums.gv=20= b/doc/src/sgml/images/datachecksums.gv=0Anew=20file=20mode=20100644=0A= index=2000000000000..dff3ff7340a=0A---=20/dev/null=0A+++=20= b/doc/src/sgml/images/datachecksums.gv=0A@@=20-0,0=20+1,14=20@@=0A= +digraph=20G=20{=0A+=20=20=20=20A=20->=20B=20[label=3D"SELECT=20= pg_enable_data_checksums()"];=0A+=20=20=20=20B=20->=20C;=0A+=20=20=20=20= D=20->=20A;=0A+=20=20=20=20C=20->=20D=20[label=3D"SELECT=20= pg_disable_data_checksums()"];=0A+=20=20=20=20E=20->=20A=20[label=3D"=20= --no-data-checksums"];=0A+=20=20=20=20E=20->=20C=20[label=3D"=20= --data-checksums"];=0A+=0A+=20=20=20=20A=20[label=3D"off"];=0A+=20=20=20=20= B=20[label=3D"inprogress-on"];=0A+=20=20=20=20C=20[label=3D"on"];=0A+=20=20= =20=20D=20[label=3D"inprogress-off"];=0A+=20=20=20=20E=20= [label=3D"initdb"];=0A+}=0Adiff=20--git=20= a/doc/src/sgml/images/datachecksums.svg=20= b/doc/src/sgml/images/datachecksums.svg=0Anew=20file=20mode=20100644=0A= index=2000000000000..8c58f42922e=0A---=20/dev/null=0A+++=20= b/doc/src/sgml/images/datachecksums.svg=0A@@=20-0,0=20+1,81=20@@=0A= +=0A= +=0A+=0A+=0A+=0A+G=0A+=0A+=0A+=0A= +A=0A+=0A+off=0A+=0A+=0A+=0A+B=0A+=0A+inprogress-on= =0A+=0A+=0A+=0A+A->B=0A+=0A= +=0A+SELECT=20= pg_enable_data_checksums()=0A+=0A+=0A+=0A+C=0A+=0A+on=0A+=0A= +=0A+=0A= +B->C=0A+=0A= +=0A+=0A+=0A+=0A+D=0A= +=0A+inprogress-off=0A+=0A+=0A+=0A= +C->D=0A+=0A= +=0A+SELECT=20= pg_disable_data_checksums()=0A+=0A+=0A= +=0A+D->A=0A= +=0A+=0A+=0A+=0A= +=0A+E=0A+=0A+initdb=0A+=0A= +=0A+=0A= +E->A=0A+=0A= +=0A+=20= --no-data-checksums=0A+=0A+=0A+=0A+E->C=0A= +=0A+=0A+=20= --data-checksums=0A+=0A+=0A+=0Adiff=20= --git=20a/doc/src/sgml/monitoring.sgml=20b/doc/src/sgml/monitoring.sgml=0A= index=20bb75ed1069b..312374da5e0=20100644=0A---=20= a/doc/src/sgml/monitoring.sgml=0A+++=20b/doc/src/sgml/monitoring.sgml=0A= @@=20-3885,9=20+3885,14=20@@=20description=20|=20Waiting=20for=20a=20= newly=20initialized=20WAL=20file=20to=20reach=20durable=20storage=0A=20=20= =20=20=20=20=20=0A=20=20=20=20=20=20=20=0A=20=20=20=20=20=20= =20=20Number=20of=20data=20page=20checksum=20failures=20detected=20in=20= this=0A-=20=20=20=20=20=20=20database=20(or=20on=20a=20shared=20object),=20= or=20NULL=20if=20data=20checksums=20are=0A-=20=20=20=20=20=20=20= disabled.=0A-=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= database=20(or=20on=20a=20shared=20object).=20=20Detected=20failures=20= are=20not=20reset=20if=0A+=20=20=20=20=20=20=20the=20=20setting=20changes.=20=20Clusters=0A+=20= =20=20=20=20=20=20which=20are=20initialized=20without=20data=20checksums=20= will=20show=20this=20as=0A+=20=20=20=20=20=20=200.=20= In=20PostgreSQL=20version=0A+=20=20=20=20=20=20= =2018=20and=20earlier,=20this=20was=20set=20to=20NULL=20= for=20clusters=0A+=20=20=20=20=20=20=20with=20data=20checksums=20= disabled.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A=20=20= =20=20=20=20=0A=20=0A=20=20=20=20=20=20=0A@@=20-3896,8=20= +3901,8=20@@=20description=20|=20Waiting=20for=20a=20newly=20initialized=20= WAL=20file=20to=20reach=20durable=20storage=0A=20=20=20=20=20=20=20= =0A=20=20=20=20=20=20=20=0A=20=20=20=20=20=20=20=20Time=20= at=20which=20the=20last=20data=20page=20checksum=20failure=20was=20= detected=20in=0A-=20=20=20=20=20=20=20this=20database=20(or=20on=20a=20= shared=20object),=20or=20NULL=20if=20data=20checksums=20are=0A-=20=20=20=20= =20=20=20disabled.=0A+=20=20=20=20=20=20=20this=20database=20(or=20on=20= a=20shared=20object).=20Last=20failure=20is=20reported=0A+=20=20=20=20=20= =20=20regardless=20of=20the=20=20= setting.=0A=20=20=20=20=20=20=20=0A=20=20=20=20=20=20= =0A=20=0A@@=20-7634,6=20+7639,219=20@@=20FROM=20= pg_stat_get_backend_idset()=20AS=20backendid;=0A=20=0A=20=20=0A=20= =0A+=20=0A+=20=20= Data=20Checksum=20Progress=20Reporting=0A+=0A+=20=20= =0A+=20=20=20= pg_stat_progress_data_checksums=0A+=20=20=0A= +=0A+=20=20=0A+=20=20=20When=20data=20checksums=20are=20being=20= enabled=20on=20a=20running=20cluster,=20the=0A+=20=20=20= pg_stat_progress_data_checksums=20view=20will=20= contain=0A+=20=20=20a=20row=20for=20the=20launcher=20process,=20and=20= one=20row=20for=20each=20worker=20process=20which=0A+=20=20=20is=20= currently=20calculating=20and=20writing=20checksums=20for=20the=20data=20= pages=20in=20a=20database.=0A+=20=20=20The=20launcher=20provides=20= overview=20of=20the=20overall=20progress=20(how=20many=20databases=0A+=20= =20=20have=20been=20processed,=20how=20many=20remain),=20while=20the=20= workers=20track=20progress=20for=0A+=20=20=20currently=20processed=20= databases.=0A+=20=20=0A+=0A+=20=20=0A+=20=20=20= <structname>pg_stat_progress_data_checksums</structname>=20= View=0A+=20=20=20=0A+=20=20=20=20=0A= +=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20Column=20Type=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=20Description=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= pid=20integer=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Process=20ID=20of=20the=20data=20checksum=20process,=20launcher=20or=20= worker.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= datid=20oid=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20OID=20= of=20this=20database,=20or=200=20for=20the=20launcher=20= process.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= datname=20name=0A+=20=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Name=20of=20this=20database,=20or=20NULL=20for=20the=0A= +=20=20=20=20=20=20=20=20launcher=20process.=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= phase=20text=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Current=20processing=20phase,=20see=20=0A+=20=20=20=20=20=20=20=20for=20= description=20of=20the=20phases.=0A+=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20databases_total=20integer=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=20The=20total=20number=20of=20databases=20which=20will=20be=20= processed.=20Only=20the=0A+=20=20=20=20=20=20=20=20launcher=20process=20= has=20this=20value=20set,=20the=20worker=20processes=20have=20this=0A+=20= =20=20=20=20=20=20=20set=20to=20NULL.=0A+=20=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=0A= +=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= databases_done=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20number=20of=20databases=20which=20have=20been=20processed.=20= Only=20the=20launcher=0A+=20=20=20=20=20=20=20=20process=20has=20this=20= value=20set,=20the=20worker=20processes=20have=20this=20set=20to=0A+=20=20= =20=20=20=20=20=20NULL.=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= relations_total=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20total=20number=20of=20relations=20which=20will=20be=20= processed,=20or=0A+=20=20=20=20=20=20=20=20NULL=20if=20= the=20worker=20process=20hasn't=0A+=20=20=20=20=20=20=20=20calculated=20= the=20number=20of=20relations=20yet.=20The=20launcher=20process=20has=0A= +=20=20=20=20=20=20=20=20this=20set=20to=20NULL=20= since=20it=20isn't=20responsible=20for=0A+=20=20=20=20=20=20=20=20= processing=20relations,=20only=20launching=20worker=20processes.=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= relations_done=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20number=20of=20relations=20which=20have=20been=20processed.=20= The=20launcher=0A+=20=20=20=20=20=20=20=20process=20has=20this=20set=20= to=20NULL.=0A+=20=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=20=20= blocks_total=20integer=0A+=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20The=20number=20of=20blocks=20in=20the=20current=20relation=20which=20= will=20be=20processed,=0A+=20=20=20=20=20=20=20=20or=20= NULL=20if=20the=20worker=20process=20hasn't=0A+=20=20=20= =20=20=20=20=20calculated=20the=20number=20of=20blocks=20yet.=20The=20= launcher=20process=20has=0A+=20=20=20=20=20=20=20=20this=20set=20to=20= NULL.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20= blocks_done=20integer=0A+=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20The=20number=20of=20blocks=20in=20the=20current=20relation=20which=20= have=20been=20processed.=0A+=20=20=20=20=20=20=20=20The=20launcher=20= process=20has=20this=20set=20to=20NULL.=0A+=20=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=0A+=20=20=20=20=0A+=20=20=20=0A+=20=20=0A+=0A= +=20=20=0A+=20=20=20Data=20= Checksum=20Phases=0A+=20=20=20=0A+=20=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=0A+=20=20=20=20= =20=0A+=20=20=20=20=20=20Phase=0A+=20=20=20=20=20=20= Description=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20= =20enabling=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20The=20command=20is=20currently=20= enabling=20data=20checksums=20on=20the=20cluster.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =20=20disabling=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20The=20command=20is=20currently=20= disabling=20data=20checksums=20on=20the=20cluster.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =20=20done=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20The=20command=20is=20done=20and=20the=20= data=20checksum=20state=20in=20the=20cluster=20has=0A+=20=20=20=20=20=20=20= changed.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20=20waiting=20on=20= barrier=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20The=20command=20is=20currently=20waiting=20for=20the=20current=20= active=20backends=20to=0A+=20=20=20=20=20=20=20acknowledge=20the=20= change=20in=20data=20checksum=20state.=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= waiting=20on=20temporary=20tables=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20The=20command=20is=20= currently=20waiting=20for=20all=20temporary=20tables=20which=20existed=0A= +=20=20=20=20=20=20=20at=20the=20time=20the=20command=20was=20started=20= to=20be=20removed.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=0A+=20=20=0A= +=20=0A+=0A=20=20=0A=20=0A=20=20=0Adiff=20--git=20= a/doc/src/sgml/ref/pg_checksums.sgml=20= b/doc/src/sgml/ref/pg_checksums.sgml=0Aindex=20b64393c813f..45890324075=20= 100644=0A---=20a/doc/src/sgml/ref/pg_checksums.sgml=0A+++=20= b/doc/src/sgml/ref/pg_checksums.sgml=0A@@=20-45,6=20+45,12=20@@=20= PostgreSQL=20documentation=0A=20=20=20=20exit=20status=20is=20nonzero=20= if=20the=20operation=20failed.=0A=20=20=20=0A=20=0A+=20=20=0A= +=20=20=20When=20enabling=20checksums,=20if=20checksums=20were=20in=20= the=20process=20of=20being=20enabled=0A+=20=20=20when=20the=20cluster=20= was=20shut=20down,=20pg_checksums=0A+=20=20=20= will=20still=20process=20all=20relations=20regardless=20of=20the=20= online=20processing.=0A+=20=20=0A+=0A=20=20=20=0A=20=20=20=20= When=20verifying=20checksums,=20every=20file=20in=20the=20cluster=20is=20= scanned.=20When=0A=20=20=20=20enabling=20checksums,=20each=20relation=20= file=20block=20with=20a=20changed=20checksum=20is=0Adiff=20--git=20= a/doc/src/sgml/regress.sgml=20b/doc/src/sgml/regress.sgml=0Aindex=20= 873387ec168..c74941bfbf2=20100644=0A---=20a/doc/src/sgml/regress.sgml=0A= +++=20b/doc/src/sgml/regress.sgml=0A@@=20-275,6=20+275,20=20@@=20make=20= check-world=20PG_TEST_EXTRA=3D'kerberos=20ldap=20ssl=20load_balance=20= libpq_encryption'=0A=20=0A=20=20=20=20The=20following=20= values=20are=20currently=20supported:=0A=20=20=20=20=0A+=20= =20=20=20=0A+=20=20=20=20=20= checksum,=20= checksum_extended=0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Runs=20additional=20= tests=20for=20enabling=20data=20checksums=20which=20inject=20faults=0A+=20= =20=20=20=20=20=20to=20cause=20re-tries=20in=20the=20processing,=20as=20= well=20as=20tests=20that=20run=20pgbench=0A+=20=20=20=20=20=20=20= concurrently=20and=20randomly=20restarts=20the=20cluster.=20=20Some=20of=20= these=20test=0A+=20=20=20=20=20=20=20suites=20require=20injection=20= points=20enabled=20in=20the=20installation.=0A+=20=20=20=20=20=20=20= checksum_extended=20is=20an=20extended=20version=20= with=0A+=20=20=20=20=20=20=20longer=20runtime,=20injected=20random=20= delays=20and=20larger=20datasets.=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=0A+=0A=20=20=20=20=20= =0A=20=20=20=20=20=20= kerberos=0A=20=20=20=20=20=20=0A= diff=20--git=20a/doc/src/sgml/wal.sgml=20b/doc/src/sgml/wal.sgml=0Aindex=20= f3b86b26be9..165af8a0cf2=20100644=0A---=20a/doc/src/sgml/wal.sgml=0A+++=20= b/doc/src/sgml/wal.sgml=0A@@=20-246,9=20+246,10=20@@=0A=20=20=20=0A= =20=20=20=20Checksums=20can=20be=20disabled=20when=20the=20cluster=20is=20= initialized=20using=20initdb.=0A-=20=20=20They=20can=20also=20be=20enabled=20or=20disabled=20at=20= a=20later=20time=20as=20an=20offline=0A-=20=20=20operation.=20Data=20= checksums=20are=20enabled=20or=20disabled=20at=20the=20full=20cluster=0A= -=20=20=20level,=20and=20cannot=20be=20specified=20individually=20for=20= databases=20or=20tables.=0A+=20=20=20They=20can=20also=20be=20enabled=20= or=20disabled=20at=20a=20later=20time=20either=20as=20an=20offline=0A+=20= =20=20operation=20or=20online=20in=20a=20running=20cluster=20allowing=20= concurrent=20access.=20Data=0A+=20=20=20checksums=20are=20enabled=20or=20= disabled=20at=20the=20full=20cluster=20level,=20and=20cannot=20be=0A+=20=20= =20specified=20individually=20for=20databases,=20tables=20or=20= replicated=20cluster=20members.=0A=20=20=20=0A=20=0A=20=20=20= =0A@@=20-265,7=20+266,7=20@@=0A=20=20=20=0A=20=0A=20=20=20= =0A-=20=20=20= Off-line=20Enabling=20of=20Checksums=0A+=20=20=20= Offline=20Enabling=20of=20Checksums=0A=20=0A=20=20=20=20= =0A=20=20=20=20=20The=20pg_checksums= =0A@@=20-274,6=20+275,123=20@@=0A=20=20=20=20=0A=20=0A=20=20=20= =0A+=0A+=20=20=0A= +=20=20=20Online=20Enabling=20of=20Checksums=0A+=0A+=20=20= =20=0A+=20=20=20=20Checksums=20can=20be=20enabled=20or=20disabled=20= online,=20by=20calling=20the=20appropriate=0A+=20=20=20=20functions.=0A+=20=20=20= =0A+=0A+=20=20=20=0A+=20=20=20=20Both=20enabling=20and=20= disabling=20data=20checksums=20happens=20in=20two=20phases,=20separated=0A= +=20=20=20=20by=20a=20checkpoint=20to=20ensure=20durability.=20=20The=20= different=20states,=20and=20their=0A+=20=20=20=20transitions,=20are=20= illustrated=20in=20=0A= +=20=20=20=20and=20discussed=20in=20further=20detail=20in=20this=20= section.=0A+=20=20=20=0A+=0A+=20=20=20=0A+=20=20=20=20= =0A+=20=20=20=20=20= data=20checksums=20states=0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20= =20=20=20=0A+=20=20=20=0A+=0A+=20=20=20=0A+=20=20=20= =20Enabling=20checksums=20will=20set=20the=20cluster=20checksum=20state=20= to=0A+=20=20=20=20inprogress-on.=20=20During=20this=20= time,=20checksums=20will=20be=0A+=20=20=20=20written=20but=20not=20= verified.=20In=20addition=20to=20this,=20a=20background=20worker=20= process=0A+=20=20=20=20is=20started=20that=20enables=20checksums=20on=20= all=20existing=20data=20in=20the=20cluster.=20Once=0A+=20=20=20=20this=20= worker=20has=20completed=20processing=20all=20databases=20in=20the=20= cluster,=20the=0A+=20=20=20=20checksum=20state=20will=20automatically=20= switch=20to=20on.=20The=0A+=20=20=20=20processing=20= will=20consume=20two=20background=20worker=20processes,=20make=20sure=20= that=0A+=20=20=20=20max_worker_processes=20allows=20= for=20at=20least=20two=20more=0A+=20=20=20=20additional=20processes.=0A+=20= =20=20=0A+=0A+=20=20=20=0A+=20=20=20=20The=20process=20will=20= initially=20wait=20for=20all=20open=20transactions=20to=20finish=20= before=0A+=20=20=20=20it=20starts,=20so=20that=20it=20can=20be=20certain=20= that=20there=20are=20no=20tables=20that=20have=20been=0A+=20=20=20=20= created=20inside=20a=20transaction=20that=20has=20not=20committed=20yet=20= and=20thus=20would=20not=0A+=20=20=20=20be=20visible=20to=20the=20= process=20enabling=20checksums.=20It=20will=20also,=20for=20each=20= database,=0A+=20=20=20=20wait=20for=20all=20pre-existing=20temporary=20= tables=20to=20get=20removed=20before=20it=20finishes.=0A+=20=20=20=20If=20= long-lived=20temporary=20tables=20are=20used=20in=20an=20application=20= it=20may=20be=20necessary=0A+=20=20=20=20to=20terminate=20these=20= application=20connections=20to=20allow=20the=20process=20to=20complete.=0A= +=20=20=20=0A+=0A+=20=20=20=0A+=20=20=20=20If=20the=20= cluster=20is=20stopped=20while=20in=20inprogress-on=20= state,=0A+=20=20=20=20for=20any=20reason,=20or=20processing=20was=20= interrupted,=20then=20the=20checksum=20enable=0A+=20=20=20=20process=20= must=20be=20restarted=20manually.=20=20To=20do=20this,=20re-execute=20= the=20function=0A+=20=20=20=20= pg_enable_data_checksums()=20once=20the=20cluster=20= has=20been=0A+=20=20=20=20restarted.=20=20The=20process=20will=20start=20= over,=20there=20is=20no=20support=20for=20resuming=0A+=20=20=20=20work=20= from=20where=20it=20was=20interrupted.=20=20If=20the=20cluster=20is=20= stopped=20while=20in=0A+=20=20=20=20inprogress-off,=20= then=20the=20checksum=20state=20will=20be=20set=20to=0A+=20=20=20=20= off=20when=20the=20cluster=20is=20restarted.=0A+=20=20= =20=0A+=0A+=20=20=20=0A+=20=20=20=20Disabling=20data=20= checksums=20will=20set=20the=20data=20checksum=20state=20to=0A+=20=20=20=20= inprogress-off.=20=20During=20this=20time,=20= checksums=20will=20be=0A+=20=20=20=20written=20but=20not=20verified.=20=20= After=20all=20processes=20acknowledge=20the=20change,=0A+=20=20=20=20the=20= state=20will=20automatically=20be=20set=20to=20off.=0A= +=20=20=20=0A+=0A+=20=20=20=0A+=20=20=20=20Disabling=20data=20= checksums=20while=20data=20checksums=20are=20actively=20being=20enabled=0A= +=20=20=20=20will=20abort=20the=20current=20processing.=0A+=20=20=20= =0A+=0A+=20=20=20=0A= +=20=20=20=20Impact=20on=20system=20of=20online=20= operations=0A+=20=20=20=20=0A+=20=20=20=20=20Enabling=20= data=20checksums=20can=20cause=20significant=20I/O=20to=20the=20system,=20= as=20all=20of=20the=0A+=20=20=20=20=20database=20pages=20will=20need=20= to=20be=20rewritten,=20and=20will=20be=20written=20both=20to=20the=0A+=20= =20=20=20=20data=20files=20and=20the=20WAL.=20The=20impact=20may=20be=20= limited=20by=20throttling=20using=20the=0A+=20=20=20=20=20= cost_delay=20and=20= cost_limit=0A+=20=20=20=20=20parameters=20of=20= the=20pg_enable_data_checksums()=20function.=0A+=20=20= =20=20=0A+=0A+=20=20=20=20=0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20I/O:=20all=20= pages=20need=20to=20have=20data=20checksums=20calculated=20and=20written=20= which=0A+=20=20=20=20=20=20=20will=20generate=20a=20lot=20of=20dirty=20= pages=20that=20will=20need=20to=20be=20flushed=20to=20disk,=0A+=20=20=20=20= =20=20=20as=20well=20as=20WAL=20logged.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20Replication:=20When=20the=20standby=20receives=20the=20data=20= checksum=20state=20change=0A+=20=20=20=20=20=20=20in=20the=20WAL=20= stream=20it=20will=20issue=20a=20=0A+=20=20=20=20=20=20=20= restartpoint=20in=20order=20to=20flush=20the=20current=20= state=20into=20the=0A+=20=20=20=20=20=20=20= pg_control=20file.=20=20The=20restartpoint=20will=20= flush=20the=0A+=20=20=20=20=20=20=20current=20state=20to=20disk=20and=20= will=20block=20redo=20until=20finished.=20=20This=20in=20turn=0A+=20=20=20= =20=20=20=20will=20induce=20replication=20lag,=20which=20on=20= synchronous=20standbys=20also=20blocks=0A+=20=20=20=20=20=20=20the=20= primary.=20=20Reducing=20=20before=20= the=0A+=20=20=20=20=20=20=20process=20is=20started=20can=20help=20with=20= reducing=20the=20time=20it=20takes=20for=20the=0A+=20=20=20=20=20=20=20= restartpoint=20to=20finish.=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20= Shutdown/Restart:=20If=20the=20server=20is=20shut=20down=20or=20= restarted=20when=20data=0A+=20=20=20=20=20=20=20checksums=20are=20being=20= enabled,=20the=20process=20will=20not=20resume=20and=20all=20pages=0A+=20= =20=20=20=20=20=20need=20to=20be=20recalculated=20and=20rewritten.=20=20= Enabling=20data=20checksums=20should=0A+=20=20=20=20=20=20=20be=20done=20= when=20there=20is=20no=20need=20for=20regular=20maintenance=20or=20= during=20a=0A+=20=20=20=20=20=20=20service=20window.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20No=20I/O=20is=20= incurred=20when=20disabling=20data=20checksums,=20but=20checkpoints=20= are=0A+=20=20=20=20=20still=20required.=0A+=20=20=20=20=0A+=20=20=20= =0A+=0A+=20=20=0A=20=20=0A=20=0A=20=20=20=0Adiff=20--git=20= a/src/backend/access/rmgrdesc/xlogdesc.c=20= b/src/backend/access/rmgrdesc/xlogdesc.c=0Aindex=20= 44194d3ea17..50a9a84c865=20100644=0A---=20= a/src/backend/access/rmgrdesc/xlogdesc.c=0A+++=20= b/src/backend/access/rmgrdesc/xlogdesc.c=0A@@=20-18,6=20+18,7=20@@=0A=20= #include=20"access/xlog.h"=0A=20#include=20"access/xlog_internal.h"=0A=20= #include=20"catalog/pg_control.h"=0A+#include=20"storage/bufpage.h"=0A=20= #include=20"utils/guc.h"=0A=20#include=20"utils/timestamp.h"=0A=20=0A@@=20= -54,6=20+55,40=20@@=20get_wal_level_string(int=20wal_level)=0A=20=09= return=20wal_level_str;=0A=20}=0A=20=0A+const=20char=20*=0A= +get_checksum_state_string(uint32=20state)=0A+{=0A+=09switch=20(state)=0A= +=09{=0A+=09=09case=20PG_DATA_CHECKSUM_VERSION:=0A+=09=09=09return=20= "on";=0A+=09=09case=20PG_DATA_CHECKSUM_INPROGRESS_OFF:=0A+=09=09=09= return=20"inprogress-off";=0A+=09=09case=20= PG_DATA_CHECKSUM_INPROGRESS_ON:=0A+=09=09=09return=20"inprogress-on";=0A= +=09=09case=20PG_DATA_CHECKSUM_OFF:=0A+=09=09=09return=20"off";=0A+=09}=0A= +=0A+=09Assert(false);=0A+=09return=20"?";=0A+}=0A+=0A+void=0A= +xlog2_desc(StringInfo=20buf,=20XLogReaderState=20*record)=0A+{=0A+=09= char=09=20=20=20*rec=20=3D=20XLogRecGetData(record);=0A+=09uint8=09=09= info=20=3D=20XLogRecGetInfo(record)=20&=20~XLR_INFO_MASK;=0A+=0A+=09if=20= (info=20=3D=3D=20XLOG2_CHECKSUMS)=0A+=09{=0A+=09=09xl_checksum_state=20= xlrec;=0A+=0A+=09=09memcpy(&xlrec,=20rec,=20sizeof(xl_checksum_state));=0A= +=09=09appendStringInfoString(buf,=20= get_checksum_state_string(xlrec.new_checksum_state));=0A+=09}=0A+}=0A+=0A= =20void=0A=20xlog_desc(StringInfo=20buf,=20XLogReaderState=20*record)=0A=20= {=0A@@=20-69,7=20+104,8=20@@=20xlog_desc(StringInfo=20buf,=20= XLogReaderState=20*record)=0A=20=09=09=09=09=09=09=20"tli=20%u;=20prev=20= tli=20%u;=20fpw=20%s;=20wal_level=20%s;=20logical=20decoding=20%s;=20xid=20= %u:%u;=20oid=20%u;=20multi=20%u;=20offset=20%"=20PRIu64=20";=20"=0A=20=09= =09=09=09=09=09=20"oldest=20xid=20%u=20in=20DB=20%u;=20oldest=20multi=20= %u=20in=20DB=20%u;=20"=0A=20=09=09=09=09=09=09=20"oldest/newest=20commit=20= timestamp=20xid:=20%u/%u;=20"=0A-=09=09=09=09=09=09=20"oldest=20running=20= xid=20%u;=20%s",=0A+=09=09=09=09=09=09=20"oldest=20running=20xid=20%u;=20= "=0A+=09=09=09=09=09=09=20"checksums=20%s;=20%s",=0A=20=09=09=09=09=09=09= =20LSN_FORMAT_ARGS(checkpoint->redo),=0A=20=09=09=09=09=09=09=20= checkpoint->ThisTimeLineID,=0A=20=09=09=09=09=09=09=20= checkpoint->PrevTimeLineID,=0A@@=20-88,6=20+124,7=20@@=20= xlog_desc(StringInfo=20buf,=20XLogReaderState=20*record)=0A=20=09=09=09=09= =09=09=20checkpoint->oldestCommitTsXid,=0A=20=09=09=09=09=09=09=20= checkpoint->newestCommitTsXid,=0A=20=09=09=09=09=09=09=20= checkpoint->oldestActiveXid,=0A+=09=09=09=09=09=09=20= get_checksum_state_string(checkpoint->dataChecksumState),=0A=20=09=09=09=09= =09=09=20(info=20=3D=3D=20XLOG_CHECKPOINT_SHUTDOWN)=20?=20"shutdown"=20:=20= "online");=0A=20=09}=0A=20=09else=20if=20(info=20=3D=3D=20XLOG_NEXTOID)=0A= @@=20-166,7=20+203,9=20@@=20xlog_desc(StringInfo=20buf,=20= XLogReaderState=20*record)=0A=20=09=09xl_checkpoint_redo=20xlrec;=0A=20=0A= =20=09=09memcpy(&xlrec,=20rec,=20sizeof(xl_checkpoint_redo));=0A-=09=09= appendStringInfo(buf,=20"wal_level=20%s",=20= get_wal_level_string(xlrec.wal_level));=0A+=09=09appendStringInfo(buf,=20= "wal_level=20%s;=20checksums=20%s",=0A+=09=09=09=09=09=09=20= get_wal_level_string(xlrec.wal_level),=0A+=09=09=09=09=09=09=20= get_checksum_state_string(xlrec.data_checksum_version));=0A=20=09}=0A=20=09= else=20if=20(info=20=3D=3D=20XLOG_LOGICAL_DECODING_STATUS_CHANGE)=0A=20=09= {=0A@@=20-241,6=20+280,21=20@@=20xlog_identify(uint8=20info)=0A=20=09= return=20id;=0A=20}=0A=20=0A+const=20char=20*=0A+xlog2_identify(uint8=20= info)=0A+{=0A+=09const=20char=20*id=20=3D=20NULL;=0A+=0A+=09switch=20= (info=20&=20~XLR_INFO_MASK)=0A+=09{=0A+=09=09case=20XLOG2_CHECKSUMS:=0A+=09= =09=09id=20=3D=20"CHECKSUMS";=0A+=09=09=09break;=0A+=09}=0A+=0A+=09= return=20id;=0A+}=0A+=0A=20/*=0A=20=20*=20Returns=20a=20string=20giving=20= information=20about=20all=20the=20blocks=20in=20an=0A=20=20*=20= XLogRecord.=0Adiff=20--git=20a/src/backend/access/transam/xlog.c=20= b/src/backend/access/transam/xlog.c=0Aindex=202c1c6f88b74..1ce81d00d85=20= 100644=0A---=20a/src/backend/access/transam/xlog.c=0A+++=20= b/src/backend/access/transam/xlog.c=0A@@=20-75,6=20+75,7=20@@=0A=20= #include=20"pgstat.h"=0A=20#include=20"port/atomics.h"=0A=20#include=20= "postmaster/bgwriter.h"=0A+#include=20"postmaster/datachecksum_state.h"=0A= =20#include=20"postmaster/startup.h"=0A=20#include=20= "postmaster/walsummarizer.h"=0A=20#include=20"postmaster/walwriter.h"=0A= @@=20-92,6=20+93,7=20@@=0A=20#include=20"storage/predicate.h"=0A=20= #include=20"storage/proc.h"=0A=20#include=20"storage/procarray.h"=0A= +#include=20"storage/procsignal.h"=0A=20#include=20"storage/reinit.h"=0A=20= #include=20"storage/spin.h"=0A=20#include=20"storage/sync.h"=0A@@=20= -553,6=20+555,9=20@@=20typedef=20struct=20XLogCtlData=0A=20=09=20*/=0A=20= =09XLogRecPtr=09lastFpwDisableRecPtr;=0A=20=0A+=09/*=20last=20= data_checksum_version=20we've=20seen=20*/=0A+=09uint32=09=09= data_checksum_version;=0A+=0A=20=09slock_t=09=09info_lck;=09=09/*=20= locks=20shared=20variables=20shown=20above=20*/=0A=20}=20XLogCtlData;=0A=20= =0A@@=20-650,6=20+655,21=20@@=20static=20XLogRecPtr=20= LocalMinRecoveryPoint;=0A=20static=20TimeLineID=20= LocalMinRecoveryPointTLI;=0A=20static=20bool=20updateMinRecoveryPoint=20= =3D=20true;=0A=20=0A+/*=0A+=20*=20Local=20state=20for=20Controlfile=20= data_checksum_version.=20=20After=20initialization=0A+=20*=20this=20is=20= only=20updated=20when=20absorbing=20a=20procsignal=20barrier=20during=20= interrupt=0A+=20*=20processing.=20=20The=20reason=20for=20keeping=20a=20= copy=20in=20backend-private=20memory=20is=20to=0A+=20*=20avoid=20locking=20= for=20interrogating=20the=20data=20checksum=20state.=20=20Possible=20= values=0A+=20*=20are=20the=20data=20checksum=20versions=20defined=20in=20= storage/checksum.h.=0A+=20*/=0A+static=20ChecksumStateType=20= LocalDataChecksumState=20=3D=200;=0A+=0A+/*=0A+=20*=20Variable=20backing=20= the=20GUC,=20keep=20it=20in=20sync=20with=20LocalDataChecksumState.=0A+=20= *=20See=20SetLocalDataChecksumState().=0A+=20*/=0A+int=09=09=09= data_checksums=20=3D=200;=0A+=0A=20/*=20For=20= WALInsertLockAcquire/Release=20functions=20*/=0A=20static=20int=09= MyLockNo=20=3D=200;=0A=20static=20bool=20holdingAllLocks=20=3D=20false;=0A= @@=20-717,6=20+737,8=20@@=20static=20void=20= WALInsertLockAcquireExclusive(void);=0A=20static=20void=20= WALInsertLockRelease(void);=0A=20static=20void=20= WALInsertLockUpdateInsertingAt(XLogRecPtr=20insertingAt);=0A=20=0A= +static=20void=20XLogChecksums(uint32=20new_type);=0A+=0A=20/*=0A=20=20*=20= Insert=20an=20XLOG=20record=20represented=20by=20an=20= already-constructed=20chain=20of=20data=0A=20=20*=20chunks.=20=20This=20= is=20a=20low-level=20routine;=20to=20construct=20the=20WAL=20record=20= header=0A@@=20-4254,6=20+4276,12=20@@=20InitControlFile(uint64=20= sysidentifier,=20uint32=20data_checksum_version)=0A=20=09= ControlFile->wal_log_hints=20=3D=20wal_log_hints;=0A=20=09= ControlFile->track_commit_timestamp=20=3D=20track_commit_timestamp;=0A=20= =09ControlFile->data_checksum_version=20=3D=20data_checksum_version;=0A+=0A= +=09/*=0A+=09=20*=20Set=20the=20data_checksum_version=20value=20into=20= XLogCtl,=20which=20is=20where=20all=0A+=09=20*=20processes=20get=20the=20= current=20value=20from.=0A+=09=20*/=0A+=09XLogCtl->data_checksum_version=20= =3D=20data_checksum_version;=0A=20}=0A=20=0A=20static=20void=0A@@=20= -4588,10=20+4616,6=20@@=20ReadControlFile(void)=0A=20=09=09= (SizeOfXLogLongPHD=20-=20SizeOfXLogShortPHD);=0A=20=0A=20=09= CalculateCheckpointSegments();=0A-=0A-=09/*=20Make=20the=20initdb=20= settings=20visible=20as=20GUC=20variables,=20too=20*/=0A-=09= SetConfigOption("data_checksums",=20DataChecksumsEnabled()=20?=20"yes"=20= :=20"no",=0A-=09=09=09=09=09PGC_INTERNAL,=20PGC_S_DYNAMIC_DEFAULT);=0A=20= }=0A=20=0A=20/*=0A@@=20-4625,13=20+4649,323=20@@=20= GetMockAuthenticationNonce(void)=0A=20}=0A=20=0A=20/*=0A-=20*=20Are=20= checksums=20enabled=20for=20data=20pages?=0A+=20*=20= DataChecksumsNeedWrite=0A+=20*=09=09Returns=20whether=20data=20checksums=20= must=20be=20written=20or=20not=0A+=20*=0A+=20*=20Returns=20true=20if=20= data=20checksums=20are=20enabled,=20or=20are=20in=20the=20process=20of=20= being=0A+=20*=20enabled.=20During=20"inprogress-on"=20and=20= "inprogress-off"=20states=20checksums=20must=0A+=20*=20be=20written=20= even=20though=20they=20are=20not=20verified=20(see=20= datachecksum_state.c=20for=0A+=20*=20a=20longer=20discussion).=0A+=20*=0A= +=20*=20This=20function=20is=20intended=20for=20callsites=20which=20are=20= about=20to=20write=20a=20data=20page=0A+=20*=20to=20storage,=20and=20= need=20to=20know=20whether=20to=20re-calculate=20the=20checksum=20for=20= the=0A+=20*=20page=20header.=20Calling=20this=20function=20must=20be=20= performed=20as=20close=20to=20the=20write=0A+=20*=20operation=20as=20= possible=20to=20keep=20the=20critical=20section=20short.=0A+=20*/=0A= +bool=0A+DataChecksumsNeedWrite(void)=0A+{=0A+=09return=20= (LocalDataChecksumState=20=3D=3D=20PG_DATA_CHECKSUM_VERSION=20||=0A+=09=09= =09LocalDataChecksumState=20=3D=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON=20||=0A= +=09=09=09LocalDataChecksumState=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF);=0A+}=0A+=0A+bool=0A= +DataChecksumsInProgress(void)=0A+{=0A+=09return=20= LocalDataChecksumState=20=3D=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON;=0A+}=0A= +=0A+/*=0A+=20*=20DataChecksumsNeedVerify=0A+=20*=09=09Returns=20whether=20= data=20checksums=20must=20be=20verified=20or=20not=0A+=20*=0A+=20*=20= Data=20checksums=20are=20only=20verified=20if=20they=20are=20fully=20= enabled=20in=20the=20cluster.=0A+=20*=20During=20the=20"inprogress-on"=20= and=20"inprogress-off"=20states=20they=20are=20only=0A+=20*=20updated,=20= not=20verified=20(see=20datachecksum_state.c=20for=20a=20longer=20= discussion).=0A+=20*=0A+=20*=20This=20function=20is=20intended=20for=20= callsites=20which=20have=20read=20data=20and=20are=20about=0A+=20*=20to=20= perform=20checksum=20validation=20based=20on=20the=20result=20of=20this.=20= =20Calling=20this=0A+=20*=20function=20must=20be=20performed=20as=20= close=20to=20the=20validation=20call=20as=20possible=20to=0A+=20*=20keep=20= the=20critical=20section=20short.=20This=20is=20in=20order=20to=20= protect=20against=20time=20of=0A+=20*=20check/time=20of=20use=20= situations=20around=20data=20checksum=20validation.=0A=20=20*/=0A=20bool=0A= -DataChecksumsEnabled(void)=0A+DataChecksumsNeedVerify(void)=0A=20{=0A+=09= return=20(LocalDataChecksumState=20=3D=3D=20PG_DATA_CHECKSUM_VERSION);=0A= +}=0A+=0A+/*=0A+=20*=20SetDataChecksumsOnInProgress=0A+=20*=09=09Sets=20= the=20data=20checksum=20state=20to=20"inprogress-on"=20to=20enable=20= checksums=0A+=20*=0A+=20*=20To=20start=20the=20process=20of=20enabling=20= data=20checksums=20in=20a=20running=20cluster=20the=0A+=20*=20= data_checksum_version=20state=20must=20be=20changed=20to=20= "inprogress-on".=20See=0A+=20*=20SetDataChecksumsOn=20below=20for=20a=20= description=20on=20how=20this=20state=20change=20works.=0A+=20*=20This=20= function=20blocks=20until=20all=20backends=20in=20the=20cluster=20have=20= acknowledged=20the=0A+=20*=20state=20transition.=0A+=20*/=0A+void=0A= +SetDataChecksumsOnInProgress(void)=0A+{=0A+=09uint64=09=09barrier;=0A+=0A= =20=09Assert(ControlFile=20!=3D=20NULL);=0A-=09return=20= (ControlFile->data_checksum_version=20>=200);=0A+=0A+=09/*=0A+=09=20*=20= The=20state=20transition=20is=20performed=20in=20a=20critical=20section=20= with=0A+=09=20*=20checkpoints=20held=20off=20to=20provide=20crash=20= safety.=0A+=09=20*/=0A+=09START_CRIT_SECTION();=0A+=09= MyProc->delayChkptFlags=20|=3D=20DELAY_CHKPT_START;=0A+=0A+=09= XLogChecksums(PG_DATA_CHECKSUM_INPROGRESS_ON);=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON;=0A= +=09SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON);=0A+=0A= +=09MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09= END_CRIT_SECTION();=0A+=0A+=09/*=0A+=09=20*=20Update=20the=20controlfile=20= before=20waiting=20since=20if=20we=20have=20an=20immediate=0A+=09=20*=20= shutdown=20while=20waiting=20we=20want=20to=20come=20back=20up=20with=20= checksums=20enabled.=0A+=09=20*/=0A+=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A+=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON;=0A+=09UpdateControlFile();=0A+=09= LWLockRelease(ControlFileLock);=0A+=0A+=09/*=0A+=09=20*=20Await=20state=20= change=20in=20all=20backends=20to=20ensure=20that=20all=20backends=20are=20= in=0A+=09=20*=20"inprogress-on".=20Once=20done=20we=20know=20that=20all=20= backends=20are=20writing=20data=0A+=09=20*=20checksums.=0A+=09=20*/=0A+=09= WaitForProcSignalBarrier(barrier);=0A+}=0A+=0A+/*=0A+=20*=20= SetDataChecksumsOn=0A+=20*=09=09Set=20data=20checksums=20state=20to=20= 'on'=20cluster-wide=0A+=20*=0A+=20*=20Enabling=20data=20checksums=20is=20= performed=20using=20two=20barriers,=20the=20first=20one=20to=0A+=20*=20= set=20the=20state=20to=20"inprogress-on"=20(done=20by=20= SetDataChecksumsOnInProgress())=0A+=20*=20and=20the=20second=20one=20to=20= set=20the=20state=20to=20"on"=20(done=20here).=20Below=20is=20a=20short=0A= +=20*=20description=20of=20the=20processing,=20a=20more=20detailed=20= write-up=20can=20be=20found=20in=0A+=20*=20datachecksum_state.c.=0A+=20*=0A= +=20*=20To=20start=20the=20process=20of=20enabling=20data=20checksums=20= in=20a=20running=20cluster=20the=0A+=20*=20data_checksum_version=20state=20= must=20be=20changed=20to=20"inprogress-on".=20=20This=20state=0A+=20*=20= requires=20data=20checksums=20to=20be=20written=20but=20not=20verified.=20= This=20ensures=20that=0A+=20*=20all=20data=20pages=20can=20be=20= checksummed=20without=20the=20risk=20of=20false=20negatives=20in=0A+=20*=20= validation=20during=20the=20process.=20=20When=20all=20existing=20pages=20= are=20guaranteed=20to=0A+=20*=20have=20checksums,=20and=20all=20new=20= pages=20will=20be=20initiated=20with=20checksums,=20the=0A+=20*=20state=20= can=20be=20changed=20to=20"on".=20Once=20the=20state=20is=20"on"=20= checksums=20will=20be=20both=0A+=20*=20written=20and=20verified.=0A+=20*=0A= +=20*=20This=20function=20blocks=20until=20all=20backends=20in=20the=20= cluster=20have=20acknowledged=20the=0A+=20*=20state=20transition.=0A+=20= */=0A+void=0A+SetDataChecksumsOn(void)=0A+{=0A+=09uint64=09=09barrier;=0A= +=0A+=09Assert(ControlFile=20!=3D=20NULL);=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=0A+=09/*=0A+=09=20*=20The=20= only=20allowed=20state=20transition=20to=20"on"=20is=20from=20= "inprogress-on"=20since=0A+=09=20*=20that=20state=20ensures=20that=20all=20= pages=20will=20have=20data=20checksums=20written.=20=20No=0A+=09=20*=20= such=20state=20transition=20exists,=20if=20it=20does=20happen=20it's=20= likely=20due=20to=20a=0A+=09=20*=20programmer=20error.=0A+=09=20*/=0A+=09= if=20(XLogCtl->data_checksum_version=20!=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON)=0A+=09{=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=09=09elog(WARNING,=0A+=09=09=09=20= "cannot=20set=20data=20checksums=20to=20\"on\",=20current=20state=20is=20= not=20\"inprogress-on\",=20disabling");=0A+=09=09SetDataChecksumsOff();=0A= +=09=09return;=0A+=09}=0A+=0A+=09SpinLockRelease(&XLogCtl->info_lck);=0A= +=0A+=09INJECTION_POINT("datachecksums-enable-checksums-delay",=20NULL);=0A= +=09START_CRIT_SECTION();=0A+=09MyProc->delayChkptFlags=20|=3D=20= DELAY_CHKPT_START;=0A+=0A+=09XLogChecksums(PG_DATA_CHECKSUM_VERSION);=0A= +=0A+=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_VERSION;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_ON);=0A+=0A+=09= MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09= END_CRIT_SECTION();=0A+=0A+=09/*=0A+=09=20*=20Update=20the=20controlfile=20= before=20waiting=20since=20if=20we=20have=20an=20immediate=0A+=09=20*=20= shutdown=20while=20waiting=20we=20want=20to=20come=20back=20up=20with=20= checksums=20enabled.=0A+=09=20*/=0A+=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A+=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_VERSION;=0A+=09UpdateControlFile();=0A+=09= LWLockRelease(ControlFileLock);=0A+=0A+=09= RequestCheckpoint(CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT=20|=20= CHECKPOINT_FAST);=0A+=0A+=09/*=0A+=09=20*=20Await=20state=20transition=20= to=20"on"=20in=20all=20backends.=20When=20done=20we=20know=20that=0A+=09=20= *=20data=20data=20checksums=20are=20both=20written=20and=20verified=20in=20= all=20backends.=0A+=09=20*/=0A+=09WaitForProcSignalBarrier(barrier);=0A= +}=0A+=0A+/*=0A+=20*=20SetDataChecksumsOff=0A+=20*=09=09Disables=20data=20= checksums=20cluster-wide=0A+=20*=0A+=20*=20Disabling=20data=20checksums=20= must=20be=20performed=20with=20two=20sets=20of=20barriers,=20each=0A+=20= *=20carrying=20a=20different=20state.=20The=20state=20is=20first=20set=20= to=20"inprogress-off"=0A+=20*=20during=20which=20checksums=20are=20still=20= written=20but=20not=20verified.=20This=20ensures=20that=0A+=20*=20= backends=20which=20have=20yet=20to=20observe=20the=20state=20change=20= from=20"on"=20won't=20get=0A+=20*=20validation=20errors=20on=20= concurrently=20modified=20pages.=20Once=20all=20backends=20have=0A+=20*=20= changed=20to=20"inprogress-off",=20the=20barrier=20for=20moving=20to=20= "off"=20can=20be=20emitted.=0A+=20*=20This=20function=20blocks=20until=20= all=20backends=20in=20the=20cluster=20have=20acknowledged=20the=0A+=20*=20= state=20transition.=0A+=20*/=0A+void=0A+SetDataChecksumsOff(void)=0A+{=0A= +=09uint64=09=09barrier;=0A+=0A+=09Assert(ControlFile=20!=3D=20NULL);=0A= +=0A+=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=0A+=09/*=20If=20data=20= checksums=20are=20already=20disabled=20there=20is=20nothing=20to=20do=20= */=0A+=09if=20(XLogCtl->data_checksum_version=20=3D=3D=200)=0A+=09{=0A+=09= =09SpinLockRelease(&XLogCtl->info_lck);=0A+=09=09return;=0A+=09}=0A+=0A+=09= /*=0A+=09=20*=20If=20data=20checksums=20are=20currently=20enabled=20we=20= first=20transition=20to=20the=0A+=09=20*=20"inprogress-off"=20state=20= during=20which=20backends=20continue=20to=20write=0A+=09=20*=20checksums=20= without=20verifying=20them.=20When=20all=20backends=20are=20in=0A+=09=20= *=20"inprogress-off"=20the=20next=20transition=20to=20"off"=20can=20be=20= performed,=20after=0A+=09=20*=20which=20all=20data=20checksum=20= processing=20is=20disabled.=0A+=09=20*/=0A+=09if=20= (XLogCtl->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_VERSION)=0A+=09= {=0A+=09=09SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09= START_CRIT_SECTION();=0A+=09=09MyProc->delayChkptFlags=20|=3D=20= DELAY_CHKPT_START;=0A+=0A+=09=09= XLogChecksums(PG_DATA_CHECKSUM_INPROGRESS_OFF);=0A+=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_INPROGRESS_OFF;=0A= +=09=09SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF);=0A+=0A= +=09=09MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09=09= END_CRIT_SECTION();=0A+=0A+=09=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A+=09=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_OFF;=0A+=09=09UpdateControlFile();=0A+=09=09= LWLockRelease(ControlFileLock);=0A+=0A+=09=09= RequestCheckpoint(CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT=20|=20= CHECKPOINT_FAST);=0A+=0A+=09=09/*=0A+=09=09=20*=20Update=20local=20state=20= in=20all=20backends=20to=20ensure=20that=20any=20backend=20in=0A+=09=09=20= *=20"on"=20state=20is=20changed=20to=20"inprogress-off".=0A+=09=09=20*/=0A= +=09=09WaitForProcSignalBarrier(barrier);=0A+=0A+=09=09/*=0A+=09=09=20*=20= At=20this=20point=20we=20know=20that=20no=20backends=20are=20verifying=20= data=20checksums=0A+=09=09=20*=20during=20reading.=20Next,=20we=20can=20= safely=20move=20to=20state=20"off"=20to=20also=0A+=09=09=20*=20stop=20= writing=20checksums.=0A+=09=09=20*/=0A+=09}=0A+=09else=0A+=09{=0A+=09=09= /*=0A+=09=09=20*=20Ending=20up=20here=20implies=20that=20the=20checksums=20= state=20is=20"inprogress-on"=0A+=09=09=20*=20or=20"inprogress-off"=20and=20= we=20can=20transition=20directly=20to=20"off"=20from=0A+=09=09=20*=20= there.=0A+=09=09=20*/=0A+=09=09SpinLockRelease(&XLogCtl->info_lck);=0A+=09= }=0A+=0A+=09START_CRIT_SECTION();=0A+=09/*=20Ensure=20that=20we=20don't=20= incur=20a=20checkpoint=20during=20disabling=20checksums=20*/=0A+=09= MyProc->delayChkptFlags=20|=3D=20DELAY_CHKPT_START;=0A+=0A+=09= XLogChecksums(PG_DATA_CHECKSUM_OFF);=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= XLogCtl->data_checksum_version=20=3D=200;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_OFF);=0A+=0A+=09= MyProc->delayChkptFlags=20&=3D=20~DELAY_CHKPT_START;=0A+=09= END_CRIT_SECTION();=0A+=0A+=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A+=09ControlFile->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_OFF;=0A+=09UpdateControlFile();=0A+=09= LWLockRelease(ControlFileLock);=0A+=0A+=09= RequestCheckpoint(CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT=20|=20= CHECKPOINT_FAST);=0A+=0A+=09WaitForProcSignalBarrier(barrier);=0A+}=0A+=0A= +/*=0A+=20*=20InitLocalDataChecksumState=0A+=20*=0A+=20*=20Set=20up=20= backend=20local=20caches=20of=20controldata=20variables=20which=20may=20= change=20at=0A+=20*=20any=20point=20during=20runtime=20and=20thus=20= require=20special=20cased=20locking.=20So=20far=0A+=20*=20this=20only=20= applies=20to=20data_checksum_version,=20but=20it's=20intended=20to=20be=20= general=0A+=20*=20purpose=20enough=20to=20handle=20future=20cases.=0A+=20= */=0A+void=0A+InitLocalDataChecksumState(void)=0A+{=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= SetLocalDataChecksumState(XLogCtl->data_checksum_version);=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+}=0A+=0A+void=0A= +SetLocalDataChecksumState(uint32=20data_checksum_version)=0A+{=0A+=09= LocalDataChecksumState=20=3D=20data_checksum_version;=0A+=0A+=09= data_checksums=20=3D=20data_checksum_version;=0A+}=0A+=0A+/*=20guc=20= hook=20*/=0A+const=20char=20*=0A+show_data_checksums(void)=0A+{=0A+=09= return=20get_checksum_state_string(LocalDataChecksumState);=0A=20}=0A=20=0A= =20/*=0A@@=20-4925,6=20+5259,7=20@@=20LocalProcessControlFile(bool=20= reset)=0A=20=09Assert(reset=20||=20ControlFile=20=3D=3D=20NULL);=0A=20=09= ControlFile=20=3D=20palloc_object(ControlFileData);=0A=20=09= ReadControlFile();=0A+=09= SetLocalDataChecksumState(ControlFile->data_checksum_version);=0A=20}=0A=20= =0A=20/*=0A@@=20-5094,6=20+5429,11=20@@=20XLOGShmemInit(void)=0A=20=09= XLogCtl->InstallXLogFileSegmentActive=20=3D=20false;=0A=20=09= XLogCtl->WalWriterSleeping=20=3D=20false;=0A=20=0A+=09/*=20Use=20the=20= checksum=20info=20from=20control=20file=20*/=0A+=09= XLogCtl->data_checksum_version=20=3D=20= ControlFile->data_checksum_version;=0A+=0A+=09= SetLocalDataChecksumState(XLogCtl->data_checksum_version);=0A+=0A=20=09= SpinLockInit(&XLogCtl->Insert.insertpos_lck);=0A=20=09= SpinLockInit(&XLogCtl->info_lck);=0A=20=09= pg_atomic_init_u64(&XLogCtl->logInsertResult,=20InvalidXLogRecPtr);=0A@@=20= -5168,6=20+5508,7=20@@=20BootStrapXLOG(uint32=20data_checksum_version)=0A= =20=09checkPoint.newestCommitTsXid=20=3D=20InvalidTransactionId;=0A=20=09= checkPoint.time=20=3D=20(pg_time_t)=20time(NULL);=0A=20=09= checkPoint.oldestActiveXid=20=3D=20InvalidTransactionId;=0A+=09= checkPoint.dataChecksumState=20=3D=20data_checksum_version;=0A=20=0A=20=09= TransamVariables->nextXid=20=3D=20checkPoint.nextXid;=0A=20=09= TransamVariables->nextOid=20=3D=20checkPoint.nextOid;=0A@@=20-6244,6=20= +6585,47=20@@=20StartupXLOG(void)=0A=20=09= pfree(endOfRecoveryInfo->recoveryStopReason);=0A=20=09= pfree(endOfRecoveryInfo);=0A=20=0A+=09/*=0A+=09=20*=20If=20we=20reach=20= this=20point=20with=20checksums=20in=20the=20state=20inprogress-on,=20it=0A= +=09=20*=20means=20that=20data=20checksums=20were=20in=20the=20process=20= of=20being=20enabled=20when=20the=0A+=09=20*=20cluster=20shut=20down.=20= Since=20processing=20didn't=20finish,=20the=20operation=20will=0A+=09=20= *=20have=20to=20be=20restarted=20from=20scratch=20since=20there=20is=20= no=20capability=20to=0A+=09=20*=20continue=20where=20it=20was=20when=20= the=20cluster=20shut=20down.=20Thus,=20revert=20the=0A+=09=20*=20state=20= back=20to=20off,=20and=20inform=20the=20user=20with=20a=20warning=20= message.=20Being=0A+=09=20*=20able=20to=20restart=20processing=20is=20a=20= TODO,=20but=20it=20wouldn't=20be=20possible=20to=0A+=09=20*=20restart=20= here=20since=20we=20cannot=20launch=20a=20dynamic=20background=20worker=0A= +=09=20*=20directly=20from=20here=20(it=20has=20to=20be=20from=20a=20= regular=20backend).=0A+=09=20*/=0A+=09if=20= (XLogCtl->data_checksum_version=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON)=0A+=09{=0A+=09=09= XLogChecksums(PG_DATA_CHECKSUM_OFF);=0A+=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=200;=0A+=09=09= SetLocalDataChecksumState(XLogCtl->data_checksum_version);=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09ereport(WARNING,=0A+=09= =09=09=09errmsg("enabling=20data=20checksums=20was=20interrupted"),=0A+=09= =09=09=09errhint("Data=20checksum=20processing=20must=20be=20manually=20= restarted=20for=20checksums=20to=20be=20enabled"));=0A+=09}=0A+=0A+=09/*=0A= +=09=20*=20If=20data=20checksums=20were=20being=20disabled=20when=20the=20= cluster=20was=20shut=20down,=0A+=09=20*=20we=20know=20that=20we=20have=20= a=20state=20where=20all=20backends=20have=20stopped=20validating=0A+=09=20= *=20checksums=20and=20we=20can=20move=20to=20off=20instead=20of=20= prompting=20the=20user=20to=0A+=09=20*=20perform=20any=20action.=0A+=09=20= */=0A+=09if=20(XLogCtl->data_checksum_version=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF)=0A+=09{=0A+=09=09= XLogChecksums(PG_DATA_CHECKSUM_OFF);=0A+=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=200;=0A+=09=09= SetLocalDataChecksumState(XLogCtl->data_checksum_version);=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=09}=0A+=0A=20=09/*=0A=20=09=20*=20= All=20done=20with=20end-of-recovery=20actions.=0A=20=09=20*=0A@@=20= -6549,7=20+6931,7=20@@=20GetRedoRecPtr(void)=0A=20=09XLogRecPtr=09ptr;=0A= =20=0A=20=09/*=0A-=09=20*=20The=20possibly=20not=20up-to-date=20copy=20= in=20XlogCtl=20is=20enough.=20Even=20if=20we=0A+=09=20*=20The=20possibly=20= not=20up-to-date=20copy=20in=20XLogCtl=20is=20enough.=20Even=20if=20we=0A= =20=09=20*=20grabbed=20a=20WAL=20insertion=20lock=20to=20read=20the=20= authoritative=20value=20in=0A=20=09=20*=20Insert->RedoRecPtr,=20someone=20= might=20update=20it=20just=20after=20we've=20released=0A=20=09=20*=20the=20= lock.=0A@@=20-7127,6=20+7509,12=20@@=20CreateCheckPoint(int=20flags)=0A=20= =09checkPoint.fullPageWrites=20=3D=20Insert->fullPageWrites;=0A=20=09= checkPoint.wal_level=20=3D=20wal_level;=0A=20=0A+=09/*=0A+=09=20*=20Get=20= the=20current=20data_checksum_version=20value=20from=20xlogctl,=20valid=20= at=20the=0A+=09=20*=20time=20of=20the=20checkpoint.=0A+=09=20*/=0A+=09= checkPoint.dataChecksumState=20=3D=20XLogCtl->data_checksum_version;=0A+=0A= =20=09if=20(shutdown)=0A=20=09{=0A=20=09=09XLogRecPtr=09curInsert=20=3D=20= XLogBytePosToRecPtr(Insert->CurrBytePos);=0A@@=20-7183,6=20+7571,9=20@@=20= CreateCheckPoint(int=20flags)=0A=20=0A=20=09=09WALInsertLockAcquire();=0A= =20=09=09redo_rec.wal_level=20=3D=20wal_level;=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= redo_rec.data_checksum_version=20=3D=20XLogCtl->data_checksum_version;=0A= +=09=09SpinLockRelease(&XLogCtl->info_lck);=0A=20=09=09= WALInsertLockRelease();=0A=20=0A=20=09=09/*=20Include=20WAL=20level=20in=20= record=20for=20WAL=20summarizer's=20benefit.=20*/=0A@@=20-7243,6=20= +7634,10=20@@=20CreateCheckPoint(int=20flags)=0A=20=09=09= checkPoint.nextOid=20+=3D=20TransamVariables->oidCount;=0A=20=09= LWLockRelease(OidGenLock);=0A=20=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09checkPoint.dataChecksumState=20= =3D=20XLogCtl->data_checksum_version;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A=20=09= checkPoint.logicalDecodingEnabled=20=3D=20IsLogicalDecodingEnabled();=0A=20= =0A=20=09MultiXactGetCheckptMulti(shutdown,=0A@@=20-7392,6=20+7787,9=20= @@=20CreateCheckPoint(int=20flags)=0A=20=09ControlFile->minRecoveryPoint=20= =3D=20InvalidXLogRecPtr;=0A=20=09ControlFile->minRecoveryPointTLI=20=3D=20= 0;=0A=20=0A+=09/*=20make=20sure=20we=20start=20with=20the=20checksum=20= version=20as=20of=20the=20checkpoint=20*/=0A+=09= ControlFile->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A= +=0A=20=09/*=0A=20=09=20*=20Persist=20unloggedLSN=20value.=20It's=20= reset=20on=20crash=20recovery,=20so=20this=20goes=0A=20=09=20*=20unused=20= on=20non-shutdown=20checkpoints,=20but=20seems=20useful=20to=20store=20= it=20always=0A@@=20-7535,6=20+7933,12=20@@=20= CreateEndOfRecoveryRecord(void)=0A=20=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A=20=09ControlFile->minRecoveryPoint=20=3D=20recptr;=0A=20= =09ControlFile->minRecoveryPointTLI=20=3D=20xlrec.ThisTimeLineID;=0A+=0A= +=09/*=20start=20with=20the=20latest=20checksum=20version=20(as=20of=20= the=20end=20of=20recovery)=20*/=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= ControlFile->data_checksum_version=20=3D=20= XLogCtl->data_checksum_version;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A=20=09UpdateControlFile();=0A=20= =09LWLockRelease(ControlFileLock);=0A=20=0A@@=20-7876,6=20+8280,10=20@@=20= CreateRestartPoint(int=20flags)=0A=20=09=09=09if=20(flags=20&=20= CHECKPOINT_IS_SHUTDOWN)=0A=20=09=09=09=09ControlFile->state=20=3D=20= DB_SHUTDOWNED_IN_RECOVERY;=0A=20=09=09}=0A+=0A+=09=09/*=20we=20shall=20= start=20with=20the=20latest=20checksum=20version=20*/=0A+=09=09= ControlFile->data_checksum_version=20=3D=20= lastCheckPoint.dataChecksumState;=0A+=0A=20=09=09UpdateControlFile();=0A=20= =09}=0A=20=09LWLockRelease(ControlFileLock);=0A@@=20-8314,6=20+8722,24=20= @@=20XLogReportParameters(void)=0A=20=09}=0A=20}=0A=20=0A+/*=0A+=20*=20= Log=20the=20new=20state=20of=20checksums=0A+=20*/=0A+static=20void=0A= +XLogChecksums(uint32=20new_type)=0A+{=0A+=09xl_checksum_state=20xlrec;=0A= +=09XLogRecPtr=09recptr;=0A+=0A+=09xlrec.new_checksum_state=20=3D=20= new_type;=0A+=0A+=09XLogBeginInsert();=0A+=09XLogRegisterData((char=20*)=20= &xlrec,=20sizeof(xl_checksum_state));=0A+=0A+=09recptr=20=3D=20= XLogInsert(RM_XLOG2_ID,=20XLOG2_CHECKSUMS);=0A+=09XLogFlush(recptr);=0A= +}=0A+=0A=20/*=0A=20=20*=20Update=20full_page_writes=20in=20shared=20= memory,=20and=20write=20an=0A=20=20*=20XLOG_FPW_CHANGE=20record=20if=20= necessary.=0A@@=20-8440,6=20+8866,11=20@@=20xlog_redo(XLogReaderState=20= *record)=0A=20=09=09MultiXactAdvanceOldest(checkPoint.oldestMulti,=0A=20=09= =09=09=09=09=09=09=20=20=20checkPoint.oldestMultiDB);=0A=20=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A+=09= =09SetLocalDataChecksumState(checkPoint.dataChecksumState);=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A=20=09=09/*=0A=20=09=09=20*=20= No=20need=20to=20set=20oldestClogXid=20here=20as=20well;=20it'll=20be=20= set=20when=20we=0A=20=09=09=20*=20redo=20an=20xl_clog_truncate=20if=20it=20= changed=20since=20initialization.=0A@@=20-8499,6=20+8930,7=20@@=20= xlog_redo(XLogReaderState=20*record)=0A=20=09=09/*=20= ControlFile->checkPointCopy=20always=20tracks=20the=20latest=20ckpt=20= XID=20*/=0A=20=09=09LWLockAcquire(ControlFileLock,=20LW_EXCLUSIVE);=0A=20= =09=09ControlFile->checkPointCopy.nextXid=20=3D=20checkPoint.nextXid;=0A= +=09=09ControlFile->data_checksum_version=20=3D=20= checkPoint.dataChecksumState;=0A=20=09=09LWLockRelease(ControlFileLock);=0A= =20=0A=20=09=09/*=0A@@=20-8525,6=20+8957,8=20@@=20= xlog_redo(XLogReaderState=20*record)=0A=20=09{=0A=20=09=09CheckPoint=09= checkPoint;=0A=20=09=09TimeLineID=09replayTLI;=0A+=09=09bool=09=09= new_state=20=3D=20false;=0A+=09=09int=09=09=09old_state;=0A=20=0A=20=09=09= memcpy(&checkPoint,=20XLogRecGetData(record),=20sizeof(CheckPoint));=0A=20= =09=09/*=20In=20an=20ONLINE=20checkpoint,=20treat=20the=20XID=20counter=20= as=20a=20minimum=20*/=0A@@=20-8563,6=20+8997,8=20@@=20= xlog_redo(XLogReaderState=20*record)=0A=20=09=09/*=20= ControlFile->checkPointCopy=20always=20tracks=20the=20latest=20ckpt=20= XID=20*/=0A=20=09=09LWLockAcquire(ControlFileLock,=20LW_EXCLUSIVE);=0A=20= =09=09ControlFile->checkPointCopy.nextXid=20=3D=20checkPoint.nextXid;=0A= +=09=09old_state=20=3D=20ControlFile->data_checksum_version;=0A+=09=09= ControlFile->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A= =20=09=09LWLockRelease(ControlFileLock);=0A=20=0A=20=09=09/*=20TLI=20= should=20not=20change=20in=20an=20on-line=20checkpoint=20*/=0A@@=20= -8574,6=20+9010,18=20@@=20xlog_redo(XLogReaderState=20*record)=0A=20=0A=20= =09=09RecoveryRestartPoint(&checkPoint,=20record);=0A=20=0A+=09=09/*=0A+=09= =09=20*=20If=20the=20data=20checksum=20state=20change=20we=20need=20to=20= emit=20a=20barrier.=0A+=09=09=20*/=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20checkPoint.dataChecksumState;=0A+=09= =09if=20(checkPoint.dataChecksumState=20!=3D=20old_state)=0A+=09=09=09= new_state=20=3D=20true;=0A+=09=09SpinLockRelease(&XLogCtl->info_lck);=0A= +=0A+=09=09if=20(new_state)=0A+=09=09=09= EmitAndWaitDataChecksumsBarrier(checkPoint.dataChecksumState);=0A+=0A=20=09= =09/*=0A=20=09=09=20*=20After=20replaying=20a=20checkpoint=20record,=20= free=20all=20smgr=20objects.=0A=20=09=09=20*=20Otherwise=20we=20would=20= never=20do=20so=20for=20dropped=20relations,=20as=20the=0A@@=20-8735,7=20= +9183,19=20@@=20xlog_redo(XLogReaderState=20*record)=0A=20=09}=0A=20=09= else=20if=20(info=20=3D=3D=20XLOG_CHECKPOINT_REDO)=0A=20=09{=0A-=09=09/*=20= nothing=20to=20do=20here,=20just=20for=20informational=20purposes=20*/=0A= +=09=09xl_checkpoint_redo=20redo_rec;=0A+=09=09bool=09=09new_state=20=3D=20= false;=0A+=0A+=09=09memcpy(&redo_rec,=20XLogRecGetData(record),=20= sizeof(xl_checkpoint_redo));=0A+=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20redo_rec.data_checksum_version;=0A= +=09=09if=20(redo_rec.data_checksum_version=20!=3D=20= ControlFile->data_checksum_version)=0A+=09=09=09new_state=20=3D=20true;=0A= +=09=09SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09if=20= (new_state)=0A+=09=09=09= EmitAndWaitDataChecksumsBarrier(redo_rec.data_checksum_version);=0A=20=09= }=0A=20=09else=20if=20(info=20=3D=3D=20= XLOG_LOGICAL_DECODING_STATUS_CHANGE)=0A=20=09{=0A@@=20-8788,6=20+9248,30=20= @@=20xlog_redo(XLogReaderState=20*record)=0A=20=09}=0A=20}=0A=20=0A+void=0A= +xlog2_redo(XLogReaderState=20*record)=0A+{=0A+=09uint8=09=09info=20=3D=20= XLogRecGetInfo(record)=20&=20~XLR_INFO_MASK;=0A+=0A+=09if=20(info=20=3D=3D= =20XLOG2_CHECKSUMS)=0A+=09{=0A+=09=09xl_checksum_state=20state;=0A+=0A+=09= =09memcpy(&state,=20XLogRecGetData(record),=20= sizeof(xl_checksum_state));=0A+=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20state.new_checksum_state;=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09/*=0A+=09=09=20*=20= Block=20on=20a=20procsignalbarrier=20to=20await=20all=20processes=20= having=20seen=20the=0A+=09=09=20*=20change=20to=20checksum=20status.=20= Once=20the=20barrier=20has=20been=20passed=20we=20can=0A+=09=09=20*=20= initiate=20the=20corresponding=20processing.=0A+=09=09=20*/=0A+=09=09= EmitAndWaitDataChecksumsBarrier(state.new_checksum_state);=0A+=09}=0A+}=0A= +=0A=20/*=0A=20=20*=20Return=20the=20extra=20open=20flags=20used=20for=20= opening=20a=20file,=20depending=20on=20the=0A=20=20*=20value=20of=20the=20= GUCs=20wal_sync_method,=20fsync=20and=20debug_io_direct.=0Adiff=20--git=20= a/src/backend/backup/basebackup.c=20b/src/backend/backup/basebackup.c=0A= index=20ab1fbae8001..9c79dadaacc=20100644=0A---=20= a/src/backend/backup/basebackup.c=0A+++=20= b/src/backend/backup/basebackup.c=0A@@=20-1613,10=20+1613,11=20@@=20= sendFile(bbsink=20*sink,=20const=20char=20*readfilename,=20const=20char=20= *tarfilename,=0A=20=09/*=0A=20=09=20*=20If=20we=20weren't=20told=20not=20= to=20verify=20checksums,=20and=20if=20checksums=20are=0A=20=09=20*=20= enabled=20for=20this=20cluster,=20and=20if=20this=20is=20a=20relation=20= file,=20then=20verify=0A-=09=20*=20the=20checksum.=0A+=09=20*=20the=20= checksum.=20=20We=20cannot=20at=20this=20point=20check=20if=20checksums=20= are=20enabled=0A+=09=20*=20or=20disabled=20as=20that=20might=20change,=20= thus=20we=20check=20at=20each=20point=20where=20we=0A+=09=20*=20could=20= be=20validating=20a=20checksum.=0A=20=09=20*/=0A-=09if=20= (!noverify_checksums=20&&=20DataChecksumsEnabled()=20&&=0A-=09=09= RelFileNumberIsValid(relfilenumber))=0A+=09if=20(!noverify_checksums=20= &&=20RelFileNumberIsValid(relfilenumber))=0A=20=09=09verify_checksum=20=3D= =20true;=0A=20=0A=20=09/*=0A@@=20-1749,7=20+1750,7=20@@=20= sendFile(bbsink=20*sink,=20const=20char=20*readfilename,=20const=20char=20= *tarfilename,=0A=20=09=09=20*=20If=20the=20amount=20of=20data=20we=20= were=20able=20to=20read=20was=20not=20a=20multiple=20of=0A=20=09=09=20*=20= BLCKSZ,=20we=20cannot=20verify=20checksums,=20which=20are=20block-level.=0A= =20=09=09=20*/=0A-=09=09if=20(verify_checksum=20&&=20(cnt=20%=20BLCKSZ=20= !=3D=200))=0A+=09=09if=20(verify_checksum=20&&=20= DataChecksumsNeedVerify()=20&&=20(cnt=20%=20BLCKSZ=20!=3D=200))=0A=20=09=09= {=0A=20=09=09=09ereport(WARNING,=0A=20=09=09=09=09=09(errmsg("could=20= not=20verify=20checksum=20in=20file=20\"%s\",=20block=20"=0A@@=20-1844,9=20= +1845,10=20@@=20sendFile(bbsink=20*sink,=20const=20char=20*readfilename,=20= const=20char=20*tarfilename,=0A=20=20*=20'blkno'=20is=20the=20block=20= number=20of=20the=20first=20page=20in=20the=20bbsink's=20buffer=0A=20=20= *=20relative=20to=20the=20start=20of=20the=20relation.=0A=20=20*=0A-=20*=20= 'verify_checksum'=20indicates=20whether=20we=20should=20try=20to=20= verify=20checksums=0A-=20*=20for=20the=20blocks=20we=20read.=20If=20we=20= do=20this,=20we'll=20update=20*checksum_failures=0A-=20*=20and=20issue=20= warnings=20as=20appropriate.=0A+=20*=20'verify_checksum'=20determines=20= if=20the=20user=20has=20asked=20to=20verify=20checksums,=20but=0A+=20*=20= since=20data=20checksums=20can=20be=20disabled,=20or=20become=20= disabled,=20we=20need=20to=20check=0A+=20*=20state=20before=20verifying=20= individual=20pages.=20=20If=20we=20do=20this,=20we'll=20update=0A+=20*=20= *checksum_failures=20and=20issue=20warnings=20as=20appropriate.=0A=20=20= */=0A=20static=20off_t=0A=20read_file_data_into_buffer(bbsink=20*sink,=20= const=20char=20*readfilename,=20int=20fd,=0A@@=20-1872,6=20+1874,13=20@@=20= read_file_data_into_buffer(bbsink=20*sink,=20const=20char=20= *readfilename,=20int=20fd,=0A=20=09=09int=09=09=09reread_cnt;=0A=20=09=09= uint16=09=09expected_checksum;=0A=20=0A+=09=09/*=0A+=09=09=20*=20The=20= data=20checksum=20state=20can=20change=20at=20any=20point,=20so=20we=20= need=20to=0A+=09=09=20*=20re-check=20before=20each=20page.=0A+=09=09=20= */=0A+=09=09if=20(!DataChecksumsNeedVerify())=0A+=09=09=09return=20cnt;=0A= +=0A=20=09=09page=20=3D=20sink->bbs_buffer=20+=20BLCKSZ=20*=20i;=0A=20=0A= =20=09=09/*=20If=20the=20page=20is=20OK,=20go=20on=20to=20the=20next=20= one.=20*/=0A@@=20-1894,7=20+1903,12=20@@=20= read_file_data_into_buffer(bbsink=20*sink,=20const=20char=20= *readfilename,=20int=20fd,=0A=20=09=09=20*=20allows=20us=20to=20wait=20= until=20we=20can=20be=20certain=20that=20no=20write=20to=20the=0A=20=09=09= =20*=20block=20is=20in=20progress.=20Since=20we=20don't=20have=20any=20= such=20thing=20right=20now,=0A=20=09=09=20*=20we=20just=20do=20this=20= and=20hope=20for=20the=20best.=0A+=09=09=20*=0A+=09=09=20*=20The=20data=20= checksum=20state=20may=20also=20have=20changed=20concurrently=20so=20= check=0A+=09=09=20*=20again.=0A=20=09=09=20*/=0A+=09=09if=20= (!DataChecksumsNeedVerify())=0A+=09=09=09return=20cnt;=0A=20=09=09= reread_cnt=20=3D=0A=20=09=09=09basebackup_read_file(fd,=20= sink->bbs_buffer=20+=20BLCKSZ=20*=20i,=0A=20=09=09=09=09=09=09=09=09=20= BLCKSZ,=20offset=20+=20BLCKSZ=20*=20i,=0A@@=20-2009,6=20+2023,9=20@@=20= verify_page_checksum(Page=20page,=20XLogRecPtr=20start_lsn,=20= BlockNumber=20blkno,=0A=20=09if=20(PageIsNew(page)=20||=20= PageGetLSN(page)=20>=3D=20start_lsn)=0A=20=09=09return=20true;=0A=20=0A+=09= if=20(!DataChecksumsNeedVerify())=0A+=09=09return=20true;=0A+=0A=20=09/*=20= Perform=20the=20actual=20checksum=20calculation.=20*/=0A=20=09checksum=20= =3D=20pg_checksum_page(page,=20blkno);=0A=20=0Adiff=20--git=20= a/src/backend/catalog/system_views.sql=20= b/src/backend/catalog/system_views.sql=0Aindex=20= e54018004db..eba25aa3e4d=20100644=0A---=20= a/src/backend/catalog/system_views.sql=0A+++=20= b/src/backend/catalog/system_views.sql=0A@@=20-1451,6=20+1451,25=20@@=20= CREATE=20VIEW=20pg_stat_progress_copy=20AS=0A=20=20=20=20=20FROM=20= pg_stat_get_progress_info('COPY')=20AS=20S=0A=20=20=20=20=20=20=20=20=20= LEFT=20JOIN=20pg_database=20D=20ON=20S.datid=20=3D=20D.oid;=0A=20=0A= +CREATE=20VIEW=20pg_stat_progress_data_checksums=20AS=0A+=20=20=20=20= SELECT=0A+=20=20=20=20=20=20=20=20S.pid=20AS=20pid,=20S.datid,=20= D.datname=20AS=20datname,=0A+=20=20=20=20=20=20=20=20CASE=20S.param1=20= WHEN=200=20THEN=20'enabling'=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20WHEN=201=20THEN=20'disabling'=0A+=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20WHEN=202=20THEN=20= 'waiting=20on=20temporary=20tables'=0A+=09=09=09=09=09=20=20WHEN=203=20= THEN=20'waiting=20on=20barrier'=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20WHEN=204=20THEN=20'done'=0A+=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20END=20AS=20phase,=0A+=20=20= =20=20=20=20=20=20CASE=20S.param2=20WHEN=20-1=20THEN=20NULL=20ELSE=20= S.param2=20END=20AS=20databases_total,=0A+=20=20=20=20=20=20=20=20= S.param3=20AS=20databases_done,=0A+=20=20=20=20=20=20=20=20CASE=20= S.param4=20WHEN=20-1=20THEN=20NULL=20ELSE=20S.param4=20END=20AS=20= relations_total,=0A+=20=20=20=20=20=20=20=20CASE=20S.param5=20WHEN=20-1=20= THEN=20NULL=20ELSE=20S.param5=20END=20AS=20relations_done,=0A+=20=20=20=20= =20=20=20=20CASE=20S.param6=20WHEN=20-1=20THEN=20NULL=20ELSE=20S.param6=20= END=20AS=20blocks_total,=0A+=20=20=20=20=20=20=20=20CASE=20S.param7=20= WHEN=20-1=20THEN=20NULL=20ELSE=20S.param7=20END=20AS=20blocks_done=0A+=20= =20=20=20FROM=20pg_stat_get_progress_info('DATACHECKSUMS')=20AS=20S=0A+=20= =20=20=20=20=20=20=20LEFT=20JOIN=20pg_database=20D=20ON=20S.datid=20=3D=20= D.oid=0A+=20=20=20=20ORDER=20BY=20S.datid;=20--=20return=20the=20= launcher=20process=20first=0A+=0A=20CREATE=20VIEW=20pg_user_mappings=20= AS=0A=20=20=20=20=20SELECT=0A=20=20=20=20=20=20=20=20=20U.oid=20=20=20=20= =20=20=20AS=20umid,=0Adiff=20--git=20a/src/backend/commands/dbcommands.c=20= b/src/backend/commands/dbcommands.c=0Aindex=209b18bb4a17e..55ab315c45a=20= 100644=0A---=20a/src/backend/commands/dbcommands.c=0A+++=20= b/src/backend/commands/dbcommands.c=0A@@=20-1044,7=20+1044,14=20@@=20= createdb(ParseState=20*pstate,=20const=20CreatedbStmt=20*stmt)=0A=20=09=09= if=20(pg_strcasecmp(strategy,=20"wal_log")=20=3D=3D=200)=0A=20=09=09=09= dbstrategy=20=3D=20CREATEDB_WAL_LOG;=0A=20=09=09else=20if=20= (pg_strcasecmp(strategy,=20"file_copy")=20=3D=3D=200)=0A+=09=09{=0A+=09=09= =09if=20(DataChecksumsInProgress())=0A+=09=09=09=09ereport(ERROR,=0A+=09=09= =09=09=09=09errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09=09=09= errmsg("create=20database=20strategy=20\"%s\"=20not=20allowed=20when=20= data=20checksums=20are=20being=20enabled",=0A+=09=09=09=09=09=09=09=20=20= =20strategy));=0A=20=09=09=09dbstrategy=20=3D=20CREATEDB_FILE_COPY;=0A+=09= =09}=0A=20=09=09else=0A=20=09=09=09ereport(ERROR,=0A=20=09=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0Adiff=20--git=20= a/src/backend/postmaster/Makefile=20b/src/backend/postmaster/Makefile=0A= index=200f4435d2d97..55044b2bc6f=20100644=0A---=20= a/src/backend/postmaster/Makefile=0A+++=20= b/src/backend/postmaster/Makefile=0A@@=20-18,6=20+18,7=20@@=20OBJS=20=3D=20= \=0A=20=09bgworker.o=20\=0A=20=09bgwriter.o=20\=0A=20=09checkpointer.o=20= \=0A+=09datachecksum_state.o=20\=0A=20=09fork_process.o=20\=0A=20=09= interrupt.o=20\=0A=20=09launch_backend.o=20\=0Adiff=20--git=20= a/src/backend/postmaster/auxprocess.c=20= b/src/backend/postmaster/auxprocess.c=0Aindex=20cf24f662d27..8fdc518b3a1=20= 100644=0A---=20a/src/backend/postmaster/auxprocess.c=0A+++=20= b/src/backend/postmaster/auxprocess.c=0A@@=20-15,6=20+15,7=20@@=0A=20= #include=20=0A=20#include=20=0A=20=0A+#include=20= "access/xlog.h"=0A=20#include=20"miscadmin.h"=0A=20#include=20"pgstat.h"=0A= =20#include=20"postmaster/auxprocess.h"=0A@@=20-69,6=20+70,24=20@@=20= AuxiliaryProcessMainCommon(void)=0A=20=0A=20=09ProcSignalInit(NULL,=20= 0);=0A=20=0A+=09/*=0A+=09=20*=20Initialize=20a=20local=20cache=20of=20= the=20data_checksum_version,=20to=20be=20updated=20by=0A+=09=20*=20the=20= procsignal-based=20barriers.=0A+=09=20*=0A+=09=20*=20This=20= intentionally=20happens=20after=20initializing=20the=20procsignal,=20= otherwise=0A+=09=20*=20we=20might=20miss=20a=20state=20change.=20This=20= means=20we=20can=20get=20a=20barrier=20for=20the=0A+=09=20*=20state=20= we've=20just=20initialized=20-=20but=20it=20can=20happen=20only=20once.=0A= +=09=20*=0A+=09=20*=20The=20postmaster=20(which=20is=20what=20gets=20= forked=20into=20the=20new=20child=20process)=0A+=09=20*=20does=20not=20= handle=20barriers,=20therefore=20it=20may=20not=20have=20the=20current=20= value=0A+=09=20*=20of=20LocalDataChecksumVersion=20value=20(it'll=20have=20= the=20value=20read=20from=20the=0A+=09=20*=20control=20file,=20which=20= may=20be=20arbitrarily=20old).=0A+=09=20*=0A+=09=20*=20NB:=20Even=20if=20= the=20postmaster=20handled=20barriers,=20the=20value=20might=20still=20= be=0A+=09=20*=20stale,=20as=20it=20might=20have=20changed=20after=20this=20= process=20forked.=0A+=09=20*/=0A+=09InitLocalDataChecksumState();=0A+=0A=20= =09/*=0A=20=09=20*=20Auxiliary=20processes=20don't=20run=20transactions,=20= but=20they=20may=20need=20a=0A=20=09=20*=20resource=20owner=20anyway=20= to=20manage=20buffer=20pins=20acquired=20outside=0Adiff=20--git=20= a/src/backend/postmaster/bgworker.c=20= b/src/backend/postmaster/bgworker.c=0Aindex=20f2a62489d9c..536aff7ca05=20= 100644=0A---=20a/src/backend/postmaster/bgworker.c=0A+++=20= b/src/backend/postmaster/bgworker.c=0A@@=20-18,6=20+18,7=20@@=0A=20= #include=20"pgstat.h"=0A=20#include=20"port/atomics.h"=0A=20#include=20= "postmaster/bgworker_internals.h"=0A+#include=20= "postmaster/datachecksum_state.h"=0A=20#include=20= "postmaster/postmaster.h"=0A=20#include=20= "replication/logicallauncher.h"=0A=20#include=20= "replication/logicalworker.h"=0A@@=20-144,7=20+145,14=20@@=20static=20= const=20struct=0A=20=09=09.fn_name=20=3D=20"TableSyncWorkerMain",=0A=20=09= =09.fn_addr=20=3D=20TableSyncWorkerMain=0A=20=09},=0A-=0A+=09{=0A+=09=09= .fn_name=20=3D=20"DataChecksumsWorkerLauncherMain",=0A+=09=09.fn_addr=20= =3D=20DataChecksumsWorkerLauncherMain=0A+=09},=0A+=09{=0A+=09=09.fn_name=20= =3D=20"DataChecksumsWorkerMain",=0A+=09=09.fn_addr=20=3D=20= DataChecksumsWorkerMain=0A+=09}=0A=20};=0A=20=0A=20/*=20Private=20= functions.=20*/=0Adiff=20--git=20= a/src/backend/postmaster/datachecksum_state.c=20= b/src/backend/postmaster/datachecksum_state.c=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..754edc5c746=0A---=20/dev/null=0A+++=20= b/src/backend/postmaster/datachecksum_state.c=0A@@=20-0,0=20+1,1610=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20datachecksum_state.c=0A+=20*=09=20=20Background=20= worker=20for=20enabling=20or=20disabling=20data=20checksums=20online=20= as=0A+=20*=09=20=20well=20as=20functionality=20for=20manipulating=20data=20= checksum=20state=0A+=20*=0A+=20*=20When=20enabling=20data=20checksums=20= on=20a=20cluster=20at=20initdb=20time=20or=20when=20shut=20down=0A+=20*=20= with=20pg_checksums,=20no=20extra=20process=20is=20required=20as=20each=20= page=20is=20checksummed,=0A+=20*=20and=20verified,=20when=20accessed.=20=20= When=20enabling=20checksums=20on=20an=20already=20running=0A+=20*=20= cluster,=20this=20worker=20will=20ensure=20that=20all=20pages=20are=20= checksummed=20before=0A+=20*=20verification=20of=20the=20checksums=20is=20= turned=20on.=20In=20the=20case=20of=20disabling=0A+=20*=20checksums,=20= the=20state=20transition=20is=20performed=20only=20in=20the=20control=20= file,=20no=0A+=20*=20changes=20are=20performed=20on=20the=20data=20= pages.=0A+=20*=0A+=20*=20Checksums=20can=20be=20either=20enabled=20or=20= disabled=20cluster-wide,=20with=20on/off=20being=0A+=20*=20the=20end=20= state=20for=20data_checksums.=0A+=20*=0A+=20*=201.=20Enabling=20= checksums=0A+=20*=20---------------------=0A+=20*=20When=20enabling=20= checksums=20in=20an=20online=20cluster,=20data_checksums=20will=20be=20= set=20to=0A+=20*=20"inprogress-on"=20which=20signals=20that=20write=20= operations=20MUST=20compute=20and=20write=0A+=20*=20the=20checksum=20on=20= the=20data=20page,=20but=20during=20reading=20the=20checksum=20SHALL=20= NOT=20be=0A+=20*=20verified.=20This=20ensures=20that=20all=20objects=20= created=20during=20when=20checksums=20are=0A+=20*=20being=20enabled=20= will=20have=20checksums=20set,=20but=20reads=20won't=20fail=20due=20to=20= missing=20or=0A+=20*=20invalid=20checksums.=20Invalid=20checksums=20can=20= be=20present=20in=20case=20the=20cluster=20had=0A+=20*=20checksums=20= enabled,=20then=20disabled=20them=20and=20updated=20the=20page=20while=20= they=20were=0A+=20*=20disabled.=0A+=20*=0A+=20*=20The=20= DataChecksumsWorker=20will=20compile=20a=20list=20of=20all=20databases=20= at=20the=20start,=0A+=20*=20any=20databases=20created=20concurrently=20= will=20see=20the=20in-progress=20state=20and=20will=0A+=20*=20be=20= checksummed=20automatically.=20=20All=20databases=20from=20the=20= original=20list=20MUST=20BE=0A+=20*=20successfully=20processed=20in=20= order=20for=20data=20checksums=20to=20be=20enabled,=20the=20only=0A+=20*=20= exception=20are=20databases=20which=20are=20dropped=20before=20having=20= been=20processed.=0A+=20*=0A+=20*=20For=20each=20database,=20all=20= relations=20which=20have=20storage=20are=20read=20and=20every=20data=0A+=20= *=20page=20is=20marked=20dirty=20to=20force=20a=20write=20with=20the=20= checksum.=20This=20will=20generate=0A+=20*=20a=20lot=20of=20WAL=20as=20= the=20entire=20database=20is=20read=20and=20written.=0A+=20*=0A+=20*=20= If=20the=20processing=20is=20interrupted=20by=20a=20cluster=20crash=20or=20= restart,=20it=20needs=20to=0A+=20*=20be=20restarted=20from=20the=20= beginning=20again=20as=20state=20isn't=20persisted.=0A+=20*=0A+=20*=202.=20= Disabling=20checksums=0A+=20*=20----------------------=0A+=20*=20When=20= disabling=20checksums,=20data_checksums=20will=20be=20set=20to=20= "inprogress-off"=0A+=20*=20which=20signals=20that=20checksums=20are=20= written=20but=20no=20longer=20need=20to=20be=20verified.=0A+=20*=20This=20= ensures=20that=20backends=20which=20have=20not=20yet=20transitioned=20to=20= the=0A+=20*=20"inprogress-off"=20state=20will=20still=20see=20valid=20= checksums=20on=20pages.=0A+=20*=0A+=20*=203.=20Synchronization=20and=20= Correctness=0A+=20*=20----------------------------------=0A+=20*=20The=20= processes=20involved=20in=20enabling=20or=20disabling=20data=20checksums=20= in=20an=0A+=20*=20online=20cluster=20must=20be=20properly=20synchronized=20= with=20the=20normal=20backends=0A+=20*=20serving=20concurrent=20queries=20= to=20ensure=20correctness.=20Correctness=20is=20defined=0A+=20*=20as=20= the=20following:=0A+=20*=0A+=20*=20=20=20=20-=20Backends=20SHALL=20NOT=20= violate=20the=20data_checksums=20state=20they=20have=20agreed=20to=0A+=20= *=20=20=20=20=20=20by=20acknowledging=20the=20procsignalbarrier:=20=20= This=20means=20that=20all=20backends=0A+=20*=20=20=20=20=20=20MUST=20= calculate=20and=20write=20data=20checksums=20during=20all=20states=20= except=20off;=0A+=20*=20=20=20=20=20=20MUST=20validate=20checksums=20= only=20in=20the=20'on'=20state.=0A+=20*=20=20=20=20-=20Data=20checksums=20= SHALL=20NOT=20be=20considered=20enabled=20cluster-wide=20until=20all=0A+=20= *=20=20=20=20=20=20currently=20connected=20backends=20have=20state=20= "on":=20This=20means=20that=20all=0A+=20*=20=20=20=20=20=20backends=20= must=20wait=20on=20the=20procsignalbarrier=20to=20be=20acknowledged=20by=20= all=0A+=20*=20=20=20=20=20=20before=20proceeding=20to=20validate=20data=20= checksums.=0A+=20*=0A+=20*=20There=20are=20two=20steps=20of=20= synchronization=20required=20for=20changing=20data_checksums=0A+=20*=20= in=20an=20online=20cluster:=20(i)=20changing=20state=20in=20the=20active=20= backends=20("on",=0A+=20*=20"off",=20"inprogress-on"=20and=20= "inprogress-off"),=20and=20(ii)=20ensuring=20no=0A+=20*=20incompatible=20= objects=20and=20processes=20are=20left=20in=20a=20database=20when=20= workers=20end.=0A+=20*=20The=20former=20deals=20with=20cluster-wide=20= agreement=20on=20data=20checksum=20state=20and=20the=0A+=20*=20latter=20= with=20ensuring=20that=20any=20concurrent=20activity=20cannot=20break=20= the=20data=0A+=20*=20checksum=20contract=20during=20processing.=0A+=20*=0A= +=20*=20Synchronizing=20the=20state=20change=20is=20done=20with=20= procsignal=20barriers.=20Before=0A+=20*=20updating=20the=20= data_checksums=20state=20in=20the=20control=20file,=20all=20other=20= backends=20must=20absorb=20the=0A+=20*=20barrier.=20=20Barrier=20= absorption=20will=20happen=20during=20interrupt=20processing,=20which=0A= +=20*=20means=20that=20connected=20backends=20will=20change=20state=20at=20= different=20times.=20=20If=0A+=20*=20waiting=20for=20a=20barrier=20is=20= done=20during=20startup,=20for=20example=20during=20replay,=20it=0A+=20*=20= is=20important=20to=20realize=20that=20any=20locks=20held=20by=20the=20= startup=20process=20might=0A+=20*=20cause=20deadlocks=20if=20backends=20= end=20up=20waiting=20for=20those=20locks=20while=20startup=0A+=20*=20is=20= waiting=20for=20a=20procsignalbarrier.=0A+=20*=0A+=20*=203.1=20When=20= Enabling=20Data=20Checksums=0A+=20*=20--------------------------------=0A= +=20*=20A=20process=20which=20fails=20to=20observe=20data=20checksums=20= being=20enabled=20can=20induce=20two=0A+=20*=20types=20of=20errors:=20= failing=20to=20write=20the=20checksum=20when=20modifying=20the=20page=20= and=0A+=20*=20failing=20to=20validate=20the=20data=20checksum=20on=20the=20= page=20when=20reading=20it.=0A+=20*=0A+=20*=20When=20processing=20starts=20= all=20backends=20belong=20to=20one=20of=20the=20below=20sets,=20with=0A+=20= *=20one=20if=20Bd=20and=20Bi=20being=20empty:=0A+=20*=0A+=20*=20Bg:=20= Backend=20updating=20the=20global=20state=20and=20emitting=20the=20= procsignalbarrier=0A+=20*=20Bd:=20Backends=20in=20"off"=20state=0A+=20*=20= Bi:=20Backends=20in=20"inprogress-on"=20state=0A+=20*=0A+=20*=20If=20= processing=20is=20started=20in=20an=20online=20cluster=20then=20all=20= backends=20are=20in=20Bd.=0A+=20*=20If=20processing=20was=20halted=20by=20= the=20cluster=20shutting=20down=20(due=20to=20a=20crash=20or=0A+=20*=20= intentional=20restart),=20the=20controlfile=20state=20"inprogress-on"=20= will=20be=20observed=0A+=20*=20on=20system=20startup=20and=20all=20= backends=20will=20be=20placed=20in=20Bd.=20The=20controlfile=0A+=20*=20= state=20will=20also=20be=20set=20to=20"off".=0A+=20*=0A+=20*=20Backends=20= transition=20Bd=20->=20Bi=20via=20a=20procsignalbarrier=20which=20is=20= emitted=20by=20the=0A+=20*=20DataChecksumsLauncher.=20=20When=20all=20= backends=20have=20acknowledged=20the=20barrier=20then=0A+=20*=20Bd=20= will=20be=20empty=20and=20the=20next=20phase=20can=20begin:=20= calculating=20and=20writing=20data=0A+=20*=20checksums=20with=20= DataChecksumsWorkers.=20=20When=20the=20DataChecksumsWorker=20processes=0A= +=20*=20have=20finished=20writing=20checksums=20on=20all=20pages,=20data=20= checksums=20are=20enabled=0A+=20*=20cluster-wide=20via=20another=20= procsignalbarrier.=20There=20are=20four=20sets=20of=20backends=0A+=20*=20= where=20Bd=20shall=20be=20an=20empty=20set:=0A+=20*=0A+=20*=20Bg:=20= Backend=20updating=20the=20global=20state=20and=20emitting=20the=20= procsignalbarrier=0A+=20*=20Bd:=20Backends=20in=20"off"=20state=0A+=20*=20= Be:=20Backends=20in=20"on"=20state=0A+=20*=20Bi:=20Backends=20in=20= "inprogress-on"=20state=0A+=20*=0A+=20*=20Backends=20in=20Bi=20and=20Be=20= will=20write=20checksums=20when=20modifying=20a=20page,=20but=20only=0A+=20= *=20backends=20in=20Be=20will=20verify=20the=20checksum=20during=20= reading.=20The=20Bg=20backend=20is=0A+=20*=20blocked=20waiting=20for=20= all=20backends=20in=20Bi=20to=20process=20interrupts=20and=20move=20to=0A= +=20*=20Be.=20Any=20backend=20starting=20while=20Bg=20is=20waiting=20on=20= the=20procsignalbarrier=20will=0A+=20*=20observe=20the=20global=20state=20= being=20"on"=20and=20will=20thus=20automatically=20belong=20to=0A+=20*=20= Be.=20=20Checksums=20are=20enabled=20cluster-wide=20when=20Bi=20is=20an=20= empty=20set.=20Bi=20and=20Be=0A+=20*=20are=20compatible=20sets=20while=20= still=20operating=20based=20on=20their=20local=20state=20as=0A+=20*=20= both=20write=20data=20checksums.=0A+=20*=0A+=20*=203.2=20When=20= Disabling=20Data=20Checksums=0A+=20*=20---------------------------------=0A= +=20*=20A=20process=20which=20fails=20to=20observe=20that=20data=20= checksums=20have=20been=20disabled=0A+=20*=20can=20induce=20two=20types=20= of=20errors:=20writing=20the=20checksum=20when=20modifying=20the=0A+=20*=20= page=20and=20validating=20a=20data=20checksum=20which=20is=20no=20longer=20= correct=20due=20to=0A+=20*=20modifications=20to=20the=20page.=20The=20= former=20is=20not=20an=20error=20per=20se=20as=20data=0A+=20*=20= integrity=20is=20maintained,=20but=20it=20is=20wasteful.=20=20The=20= latter=20will=20cause=20errors=0A+=20*=20in=20user=20operations.=20=20= Assuming=20the=20following=20sets=20of=20backends:=0A+=20*=0A+=20*=20Bg:=20= Backend=20updating=20the=20global=20state=20and=20emitting=20the=20= procsignalbarrier=0A+=20*=20Bd:=20Backends=20in=20"off"=20state=0A+=20*=20= Be:=20Backends=20in=20"on"=20state=0A+=20*=20Bo:=20Backends=20in=20= "inprogress-off"=20state=0A+=20*=20Bi:=20Backends=20in=20"inprogress-on"=20= state=0A+=20*=0A+=20*=20Backends=20transition=20from=20the=20Be=20state=20= to=20Bd=20like=20so:=20Be=20->=20Bo=20->=20Bd.=20=20From=0A+=20*=20all=20= other=20states,=20the=20transition=20can=20be=20straight=20to=20Bd.=0A+=20= *=0A+=20*=20The=20goal=20is=20to=20transition=20all=20backends=20to=20Bd=20= making=20the=20others=20empty=20sets.=0A+=20*=20Backends=20in=20Bo=20= write=20data=20checksums,=20but=20don't=20validate=20them,=20such=20that=0A= +=20*=20backends=20still=20in=20Be=20can=20continue=20to=20validate=20= pages=20until=20the=20barrier=20has=0A+=20*=20been=20absorbed=20such=20= that=20they=20are=20in=20Bo.=20Once=20all=20backends=20are=20in=20Bo,=20= the=0A+=20*=20barrier=20to=20transition=20to=20"off"=20can=20be=20raised=20= and=20all=20backends=20can=20safely=0A+=20*=20stop=20writing=20data=20= checksums=20as=20no=20backend=20is=20enforcing=20data=20checksum=0A+=20*=20= validation=20any=20longer.=0A+=20*=0A+=20*=204.=20Future=20opportunities=20= for=20optimizations=0A+=20*=20-----------------------------------------=0A= +=20*=20Below=20are=20some=20potential=20optimizations=20and=20= improvements=20which=20were=20brought=0A+=20*=20up=20during=20reviews=20= of=20this=20feature,=20but=20which=20weren't=20implemented=20in=20the=0A= +=20*=20initial=20version.=20These=20are=20ideas=20listed=20without=20= any=20validation=20on=20their=0A+=20*=20feasibility=20or=20potential=20= payoff.=20More=20discussion=20on=20(most=20of)=20these=20can=20be=0A+=20= *=20found=20on=20the=20-hackers=20threads=20linked=20to=20in=20the=20= commit=20message=20of=20this=0A+=20*=20feature.=0A+=20*=0A+=20*=20=20=20= *=20Launching=20datachecksumsworker=20for=20resuming=20operation=20from=20= the=20startup=0A+=20*=20=20=20=20=20process:=20Currently=20users=20have=20= to=20restart=20processing=20manually=20after=20a=0A+=20*=20=20=20=20=20= restart=20since=20dynamic=20background=20worker=20cannot=20be=20started=20= from=20the=0A+=20*=20=20=20=20=20postmaster.=20Changing=20the=20startup=20= process=20could=20make=20restarting=20the=0A+=20*=20=20=20=20=20= processing=20automatic=20on=20cluster=20restart.=0A+=20*=20=20=20*=20= Avoid=20dirtying=20the=20page=20when=20checksums=20already=20match:=20= Iff=20the=20checksum=0A+=20*=20=20=20=20=20on=20the=20page=20happens=20= to=20already=20match=20we=20still=20dirty=20the=20page.=20It=20should=0A= +=20*=20=20=20=20=20be=20enough=20to=20only=20do=20the=20= log_newpage_buffer()=20call=20in=20that=20case.=0A+=20*=20=20=20*=20= Teach=20pg_checksums=20to=20avoid=20checksummed=20pages=20when=20= pg_checksums=20is=20used=0A+=20*=20=20=20=20=20to=20enable=20checksums=20= on=20a=20cluster=20which=20is=20in=20inprogress-on=20state=20and=0A+=20*=20= =20=20=20=20may=20have=20checksummed=20pages=20(make=20pg_checksums=20be=20= able=20to=20resume=20an=0A+=20*=20=20=20=20=20online=20operation).=20= This=20should=20only=20be=20attempted=20for=20wal_level=20minimal.=0A+=20= *=20=20=20*=20Restartability=20(not=20necessarily=20with=20page=20= granularity).=0A+=20*=20=20=20*=20Avoid=20processing=20databases=20which=20= were=20created=20during=20inprogress-on.=0A+=20*=20=20=20=20=20Right=20= now=20all=20databases=20are=20processed=20regardless=20to=20be=20safe.=0A= +=20*=20=20=20*=20Teach=20CREATE=20DATABASE=20to=20calculate=20checksums=20= for=20databases=20created=0A+=20*=20=20=20=20=20during=20inprogress-on=20= with=20a=20template=20database=20which=20has=20yet=20to=20be=0A+=20*=20=20= =20=20=20processed.=0A+=20*=0A+=20*=0A+=20*=20Portions=20Copyright=20(c)=20= 1996-2026,=20PostgreSQL=20Global=20Development=20Group=0A+=20*=20= Portions=20Copyright=20(c)=201994,=20Regents=20of=20the=20University=20= of=20California=0A+=20*=0A+=20*=0A+=20*=20IDENTIFICATION=0A+=20*=09=20=20= src/backend/postmaster/datachecksum_state.c=0A+=20*=0A+=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres.h"=0A+=0A+#include=20"access/genam.h"=0A= +#include=20"access/heapam.h"=0A+#include=20"access/htup_details.h"=0A= +#include=20"access/xact.h"=0A+#include=20"access/xlog.h"=0A+#include=20= "access/xloginsert.h"=0A+#include=20"catalog/indexing.h"=0A+#include=20= "catalog/pg_class.h"=0A+#include=20"catalog/pg_database.h"=0A+#include=20= "commands/progress.h"=0A+#include=20"commands/vacuum.h"=0A+#include=20= "common/relpath.h"=0A+#include=20"miscadmin.h"=0A+#include=20"pgstat.h"=0A= +#include=20"postmaster/bgworker.h"=0A+#include=20= "postmaster/bgwriter.h"=0A+#include=20"postmaster/datachecksum_state.h"=0A= +#include=20"storage/bufmgr.h"=0A+#include=20"storage/checksum.h"=0A= +#include=20"storage/ipc.h"=0A+#include=20"storage/latch.h"=0A+#include=20= "storage/lmgr.h"=0A+#include=20"storage/lwlock.h"=0A+#include=20= "storage/procarray.h"=0A+#include=20"storage/smgr.h"=0A+#include=20= "tcop/tcopprot.h"=0A+#include=20"utils/builtins.h"=0A+#include=20= "utils/fmgroids.h"=0A+#include=20"utils/injection_point.h"=0A+#include=20= "utils/lsyscache.h"=0A+#include=20"utils/ps_status.h"=0A+#include=20= "utils/syscache.h"=0A+#include=20"utils/wait_event.h"=0A+=0A+/*=0A+=20*=20= Configuration=20of=20conditions=20which=20must=20match=20when=20= absorbing=20a=20procsignal=0A+=20*=20barrier=20during=20data=20checksum=20= enable/disable=20operations.=20=20A=20single=20function=0A+=20*=20is=20= used=20for=20absorbing=20all=20barriers,=20and=20the=20current=20and=20= target=20states=20must=0A+=20*=20be=20defined=20as=20a=20from/to=20tuple=20= in=20the=20checksum_barriers=20struct.=0A+=20*/=0A+typedef=20struct=20= ChecksumBarrierCondition=0A+{=0A+=09/*=20Current=20state=20of=20data=20= checksums=20*/=0A+=09int=09=09=09from;=0A+=09/*=20Target=20state=20for=20= data=20checksums=20*/=0A+=09int=09=09=09to;=0A+}=20= ChecksumBarrierCondition;=0A+=0A+static=20const=20= ChecksumBarrierCondition=20checksum_barriers[6]=20=3D=0A+{=0A+=09/*=0A+=09= =20*=20Disabling=20checksums:=20If=20checksums=20are=20currently=20= enabled,=20disabling=20must=0A+=09=20*=20go=20through=20the=20= 'inprogress-off'=20state.=0A+=09=20*/=0A+=09{PG_DATA_CHECKSUM_VERSION,=20= PG_DATA_CHECKSUM_INPROGRESS_OFF},=0A+=09= {PG_DATA_CHECKSUM_INPROGRESS_OFF,=20PG_DATA_CHECKSUM_OFF},=0A+=0A+=09/*=0A= +=09=20*=20If=20checksums=20are=20in=20the=20process=20of=20being=20= enabled,=20but=20are=20not=20yet=20being=0A+=09=20*=20verified,=20we=20= can=20abort=20by=20going=20back=20to=20'off'=20state.=0A+=09=20*/=0A+=09= {PG_DATA_CHECKSUM_INPROGRESS_ON,=20PG_DATA_CHECKSUM_OFF},=0A+=0A+=09/*=0A= +=09=20*=20Enabling=20checksums=20must=20normally=20go=20through=20the=20= 'inprogress-on'=20state.=0A+=09=20*/=0A+=09{PG_DATA_CHECKSUM_OFF,=20= PG_DATA_CHECKSUM_INPROGRESS_ON},=0A+=09{PG_DATA_CHECKSUM_INPROGRESS_ON,=20= PG_DATA_CHECKSUM_VERSION},=0A+=0A+=09/*=0A+=09=20*=20If=20checksums=20= are=20being=20disabled=20but=20all=20backends=20are=20still=20computing=0A= +=09=20*=20checksums,=20we=20can=20go=20straight=20back=20to=20'on'=0A+=09= =20*/=0A+=09{PG_DATA_CHECKSUM_INPROGRESS_OFF,=20= PG_DATA_CHECKSUM_VERSION},=0A+};=0A+=0A+/*=0A+=20*=20Signaling=20between=20= backends=20calling=20pg_enable/disable_data_checksums,=20the=0A+=20*=20= checksums=20launcher=20process,=20and=20the=20checksums=20worker=20= process.=0A+=20*=0A+=20*=20This=20struct=20is=20protected=20by=20= DataChecksumsWorkerLock=0A+=20*/=0A+typedef=20struct=20= DataChecksumsStateStruct=0A+{=0A+=09/*=0A+=09=20*=20These=20are=20set=20= by=20pg_{enable|disable}_data_checksums,=20to=20tell=20the=0A+=09=20*=20= launcher=20what=20the=20target=20state=20is.=0A+=09=20*/=0A+=09= DataChecksumsWorkerOperation=20launch_operation;=0A+=09int=09=09=09= launch_cost_delay;=0A+=09int=09=09=09launch_cost_limit;=0A+=0A+=09/*=0A+=09= =20*=20Is=20a=20launcher=20process=20is=20currently=20running?=20=20This=20= is=20set=20by=20the=20main=0A+=09=20*=20launcher=20process,=20after=20it=20= has=20read=20the=20above=20launch_*=20parameters.=0A+=09=20*/=0A+=09bool=09= =09launcher_running;=0A+=0A+=09/*=0A+=09=20*=20Is=20a=20worker=20process=20= currently=20running?=20=20This=20is=20set=20by=20the=20worker=0A+=09=20*=20= launcher=20when=20it=20starts=20waiting=20for=20a=20worker=20process=20= to=20finish.=0A+=09=20*/=0A+=09int=09=09=09worker_running;=0A+=0A+=09/*=0A= +=09=20*=20These=20fields=20indicate=20the=20target=20state=20that=20the=20= launcher=20is=20currently=0A+=09=20*=20working=20towards.=20They=20can=20= be=20different=20from=20the=20corresponding=20launch_*=0A+=09=20*=20= fields,=20if=20a=20new=20pg_enable/disable_data_checksums()=20call=20was=20= made=20while=0A+=09=20*=20the=20launcher/worker=20was=20already=20= running.=0A+=09=20*=0A+=09=20*=20The=20below=20members=20are=20set=20= when=20the=20launcher=20starts,=20and=20are=20only=0A+=09=20*=20accessed=20= read-only=20by=20the=20single=20worker.=20Thus,=20we=20can=20access=20= these=0A+=09=20*=20without=20a=20lock.=20If=20multiple=20workers,=20or=20= dynamic=20cost=20parameters,=20are=0A+=09=20*=20supported=20at=20some=20= point=20then=20this=20would=20need=20to=20be=20revisited.=0A+=09=20*/=0A= +=09DataChecksumsWorkerOperation=20operation;=0A+=09int=09=09=09= cost_delay;=0A+=09int=09=09=09cost_limit;=0A+=0A+=09/*=0A+=09=20*=20= Signaling=20between=20the=20launcher=20and=20the=20worker=20process.=0A+=09= =20*=0A+=09=20*=20As=20there=20is=20only=20a=20single=20worker,=20and=20= the=20launcher=20won't=20read=20these=0A+=09=20*=20until=20the=20worker=20= exits,=20they=20can=20be=20accessed=20without=20the=20need=20for=20a=0A+=09= =20*=20lock.=20If=20multiple=20workers=20are=20supported=20then=20this=20= will=20have=20to=20be=0A+=09=20*=20revisited.=0A+=09=20*/=0A+=0A+=09/*=20= result,=20set=20by=20worker=20before=20exiting=20*/=0A+=09= DataChecksumsWorkerResult=20success;=0A+=0A+=09/*=0A+=09=20*=20tells=20= the=20worker=20process=20whether=20it=20should=20also=20process=20the=20= shared=0A+=09=20*=20catalogs=0A+=09=20*/=0A+=09bool=09=09= process_shared_catalogs;=0A+}=20DataChecksumsStateStruct;=0A+=0A+/*=20= Shared=20memory=20segment=20for=20datachecksumsworker=20*/=0A+static=20= DataChecksumsStateStruct=20*DataChecksumState;=0A+=0A+typedef=20struct=20= DataChecksumsWorkerDatabase=0A+{=0A+=09Oid=09=09=09dboid;=0A+=09char=09=20= =20=20*dbname;=0A+}=20DataChecksumsWorkerDatabase;=0A+=0A+/*=20Flag=20= set=20by=20the=20interrupt=20handler=20*/=0A+static=20volatile=20= sig_atomic_t=20abort_requested=20=3D=20false;=0A+=0A+/*=0A+=20*=20Have=20= we=20set=20the=20DataChecksumsStateStruct->launcher_running=20flag?=0A+=20= *=20If=20we=20have,=20we=20need=20to=20clear=20it=20before=20exiting!=0A= +=20*/=0A+static=20volatile=20sig_atomic_t=20launcher_running=20=3D=20= false;=0A+=0A+/*=20Are=20we=20enabling=20data=20checksums,=20or=20= disabling=20them?=20*/=0A+static=20DataChecksumsWorkerOperation=20= operation;=0A+=0A+/*=20Prototypes=20*/=0A+static=20bool=20= DatabaseExists(Oid=20dboid);=0A+static=20List=20= *BuildDatabaseList(void);=0A+static=20List=20*BuildRelationList(bool=20= temp_relations,=20bool=20include_shared);=0A+static=20void=20= FreeDatabaseList(List=20*dblist);=0A+static=20DataChecksumsWorkerResult=20= ProcessDatabase(DataChecksumsWorkerDatabase=20*db);=0A+static=20bool=20= ProcessAllDatabases(void);=0A+static=20bool=20= ProcessSingleRelationFork(Relation=20reln,=20ForkNumber=20forkNum,=20= BufferAccessStrategy=20strategy);=0A+static=20void=20= launcher_cancel_handler(SIGNAL_ARGS);=0A+static=20void=20= WaitForAllTransactionsToFinish(void);=0A+=0A= +/************************************************************************= *****=0A+=20*=20Functionality=20for=20manipulating=20the=20data=20= checksum=20state=20in=20the=20cluster=0A+=20*/=0A+=0A+void=0A= +EmitAndWaitDataChecksumsBarrier(uint32=20state)=0A+{=0A+=09uint64=09=09= barrier;=0A+=0A+=09switch=20(state)=0A+=09{=0A+=09=09case=20= PG_DATA_CHECKSUM_INPROGRESS_ON:=0A+=09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON);=0A+=09=09= =09WaitForProcSignalBarrier(barrier);=0A+=09=09=09break;=0A+=0A+=09=09= case=20PG_DATA_CHECKSUM_INPROGRESS_OFF:=0A+=09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF);=0A+=09= =09=09WaitForProcSignalBarrier(barrier);=0A+=09=09=09break;=0A+=0A+=09=09= case=20PG_DATA_CHECKSUM_VERSION:=0A+=09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_ON);=0A+=09=09=09= WaitForProcSignalBarrier(barrier);=0A+=09=09=09break;=0A+=0A+=09=09case=20= PG_DATA_CHECKSUM_OFF:=0A+=09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_OFF);=0A+=09=09=09= WaitForProcSignalBarrier(barrier);=0A+=09=09=09break;=0A+=0A+=09=09= default:=0A+=09=09=09Assert(false);=0A+=09}=0A+}=0A+=0A+/*=0A+=20*=20= AbsorbDataChecksumsBarrier=0A+=20*=09=09Generic=20function=20for=20= absorbing=20data=20checksum=20state=20changes=0A+=20*=0A+=20*=20All=20= procsignalbarriers=20regarding=20data=20checksum=20state=20changes=20are=20= absorbed=0A+=20*=20with=20this=20function.=20=20The=20set=20of=20= conditions=20required=20for=20the=20state=20change=20to=0A+=20*=20be=20= accepted=20are=20listed=20in=20the=20checksum_barriers=20struct,=20= target_state=20is=0A+=20*=20used=20to=20look=20up=20the=20relevant=20= entry.=0A+=20*/=0A+bool=0A= +AbsorbDataChecksumsBarrier(ProcSignalBarrierType=20barrier)=0A+{=0A+=09= uint32=09=09target_state;=0A+=09int=09=09=09current=20=3D=20= data_checksums;=0A+=09bool=09=09found=20=3D=20false;=0A+=0A+=09/*=0A+=09=20= *=20Translate=20the=20barrier=20condition=20to=20the=20target=20state,=20= doing=20it=20here=0A+=09=20*=20instead=20of=20in=20the=20procsignal=20= code=20saves=20the=20latter=20from=20knowing=20about=0A+=09=20*=20= checksum=20states.=0A+=09=20*/=0A+=09switch=20(barrier)=0A+=09{=0A+=09=09= case=20PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON:=0A+=09=09=09= target_state=20=3D=20PG_DATA_CHECKSUM_INPROGRESS_ON;=0A+=09=09=09break;=0A= +=09=09case=20PROCSIGNAL_BARRIER_CHECKSUM_ON:=0A+=09=09=09target_state=20= =3D=20PG_DATA_CHECKSUM_VERSION;=0A+=09=09=09break;=0A+=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF:=0A+=09=09=09target_state=20=3D= =20PG_DATA_CHECKSUM_INPROGRESS_OFF;=0A+=09=09=09break;=0A+=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_OFF:=0A+=09=09=09target_state=20=3D=20= PG_DATA_CHECKSUM_OFF;=0A+=09=09=09break;=0A+=09=09default:=0A+=09=09=09= elog(ERROR,=20"incorrect=20barrier=20\"%i\"=20received",=20barrier);=0A+=09= }=0A+=0A+=09/*=0A+=09=20*=20If=20the=20target=20state=20matches=20the=20= current=20state=20then=20the=20barrier=20has=20been=0A+=09=20*=20= repeated.=0A+=09=20*/=0A+=09if=20(current=20=3D=3D=20target_state)=0A+=09= =09return=20true;=0A+=0A+=09/*=0A+=09=20*=20If=20the=20cluster=20is=20in=20= recovery=20we=20skip=20the=20validation=20of=20current=20state=0A+=09=20= *=20since=20the=20replay=20is=20trusted.=0A+=09=20*/=0A+=09if=20= (RecoveryInProgress())=0A+=09{=0A+=09=09= SetLocalDataChecksumState(target_state);=0A+=09=09return=20true;=0A+=09}=0A= +=0A+=09/*=0A+=09=20*=20Find=20the=20barrier=20condition=20definition=20= for=20the=20target=20state.=20Not=20finding=0A+=09=20*=20a=20condition=20= would=20be=20a=20grave=20programmer=20error=20as=20the=20states=20are=20= a=0A+=09=20*=20discrete=20set.=0A+=09=20*/=0A+=09for=20(int=20i=20=3D=20= 0;=20i=20<=20lengthof(checksum_barriers)=20&&=20!found;=20i++)=0A+=09{=0A= +=09=09if=20(checksum_barriers[i].from=20=3D=3D=20current=20&&=20= checksum_barriers[i].to=20=3D=3D=20target_state)=0A+=09=09=09found=20=3D=20= true;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20If=20the=20relevant=20state=20= criteria=20aren't=20satisfied,=20throw=20an=20error=20which=0A+=09=20*=20= will=20be=20caught=20by=20the=20procsignal=20machinery=20for=20a=20later=20= retry.=0A+=09=20*/=0A+=09if=20(!found)=0A+=09=09ereport(ERROR,=0A+=09=09=09= =09errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09= errmsg("incorrect=20data=20checksum=20state=20%i=20for=20target=20state=20= %i",=0A+=09=09=09=09=09=20=20=20current,=20target_state));=0A+=0A+=09= SetLocalDataChecksumState(target_state);=0A+=09return=20true;=0A+}=0A+=0A= +=0A+/*=0A+=20*=20Disables=20data=20checksums=20for=20the=20cluster,=20= if=20applicable.=20Starts=20a=20background=0A+=20*=20worker=20which=20= turns=20off=20the=20data=20checksums.=0A+=20*/=0A+Datum=0A= +disable_data_checksums(PG_FUNCTION_ARGS)=0A+{=0A+=09if=20(!superuser())=0A= +=09=09ereport(ERROR,=0A+=09=09=09=09= errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),=0A+=09=09=09=09errmsg("must=20= be=20superuser=20to=20change=20data=20checksum=20state"));=0A+=0A+=09= StartDataChecksumsWorkerLauncher(DISABLE_DATACHECKSUMS,=200,=200);=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+/*=0A+=20*=20Enables=20data=20checksums=20= for=20the=20cluster,=20if=20applicable.=20=20Supports=20vacuum-=0A+=20*=20= like=20cost=20based=20throttling=20to=20limit=20system=20load.=20Starts=20= a=20background=20worker=0A+=20*=20which=20updates=20data=20checksums=20= on=20existing=20data.=0A+=20*/=0A+Datum=0A= +enable_data_checksums(PG_FUNCTION_ARGS)=0A+{=0A+=09int=09=09=09= cost_delay=20=3D=20PG_GETARG_INT32(0);=0A+=09int=09=09=09cost_limit=20=3D=20= PG_GETARG_INT32(1);=0A+=0A+=09if=20(!superuser())=0A+=09=09= ereport(ERROR,=0A+=09=09=09=09errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),=0A= +=09=09=09=09errmsg("must=20be=20superuser=20to=20change=20data=20= checksum=20state"));=0A+=0A+=09if=20(cost_delay=20<=200)=0A+=09=09= ereport(ERROR,=0A+=09=09=09=09errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A= +=09=09=09=09errmsg("cost=20delay=20cannot=20be=20a=20negative=20= value"));=0A+=0A+=09if=20(cost_limit=20<=3D=200)=0A+=09=09ereport(ERROR,=0A= +=09=09=09=09errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09= errmsg("cost=20limit=20must=20be=20greater=20than=20zero"));=0A+=0A+=09= StartDataChecksumsWorkerLauncher(ENABLE_DATACHECKSUMS,=20cost_delay,=20= cost_limit);=0A+=0A+=09PG_RETURN_VOID();=0A+}=0A+=0A+=0A= +/************************************************************************= *****=0A+=20*=20Functionality=20for=20running=20the=20= datachecksumsworker=20and=20associated=20launcher=0A+=20*/=0A+=0A+/*=0A+=20= *=20StartDataChecksumsWorkerLauncher=0A+=20*=09=09Main=20entry=20point=20= for=20datachecksumsworker=20launcher=20process=0A+=20*=0A+=20*=20The=20= main=20entrypoint=20for=20starting=20data=20checksums=20processing=20for=20= enabling=20as=0A+=20*=20well=20as=20disabling.=0A+=20*/=0A+void=0A= +StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A+=09= =09=09=09=09=09=09=09=20int=20cost_delay,=0A+=09=09=09=09=09=09=09=09=20= int=20cost_limit)=0A+{=0A+=09BackgroundWorker=20bgw;=0A+=09= BackgroundWorkerHandle=20*bgw_handle;=0A+=09bool=09=09launcher_running;=0A= +=09DataChecksumsWorkerOperation=20launcher_running_op;=0A+=0A+#ifdef=20= USE_ASSERT_CHECKING=0A+=09/*=20The=20cost=20delay=20settings=20have=20no=20= effect=20when=20disabling=20*/=0A+=09if=20(op=20=3D=3D=20= DISABLE_DATACHECKSUMS)=0A+=09=09Assert(cost_delay=20=3D=3D=200=20&&=20= cost_limit=20=3D=3D=200);=0A+#endif=0A+=0A+=09= INJECTION_POINT("datachecksumsworker-startup-delay",=20NULL);=0A+=0A+=09= /*=20Store=20the=20desired=20state=20in=20shared=20memory=20*/=0A+=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=0A+=09= DataChecksumState->launch_operation=20=3D=20op;=0A+=09= DataChecksumState->launch_cost_delay=20=3D=20cost_delay;=0A+=09= DataChecksumState->launch_cost_limit=20=3D=20cost_limit;=0A+=0A+=09/*=20= Is=20the=20launcher=20already=20running?=20If=20so,=20what=20is=20it=20= doing?=20*/=0A+=09launcher_running=20=3D=20= DataChecksumState->launcher_running;=0A+=09if=20(launcher_running)=0A+=09= =09launcher_running_op=20=3D=20DataChecksumState->operation;=0A+=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09/*=0A+=09=20*=20Launch=20= a=20new=20launcher=20process,=20if=20it's=20not=20running=20already.=0A+=09= =20*=0A+=09=20*=20If=20the=20launcher=20is=20currently=20busy=20enabling=20= the=20checksums,=20and=20we=20want=0A+=09=20*=20them=20disabled=20(or=20= vice=20versa),=20the=20launcher=20will=20notice=20that=20at=20latest=0A+=09= =20*=20when=20it's=20about=20to=20exit,=20and=20will=20loop=20back=20= process=20the=20new=20request.=20So=0A+=09=20*=20if=20the=20launcher=20= is=20already=20running,=20we=20don't=20need=20to=20do=20anything=20more=0A= +=09=20*=20here=20to=20abort=20it.=0A+=09=20*=0A+=09=20*=20If=20you=20= call=20pg_enable/disable_data_checksums()=20twice=20in=20a=20row,=20= before=0A+=09=20*=20the=20launcher=20has=20had=20a=20chance=20to=20start=20= up,=20we=20still=20end=20up=20launching=20it=0A+=09=20*=20twice.=20=20= That's=20OK,=20the=20second=20invocation=20will=20see=20that=20a=20= launcher=20is=0A+=09=20*=20already=20running=20and=20exit=20quickly.=0A+=09= =20*=0A+=09=20*=20TODO:=20We=20could=20optimize=20here=20and=20skip=20= launching=20the=20launcher,=20if=20we=20are=0A+=09=20*=20already=20in=20= the=20desired=20state,=20i.e.=20if=20the=20checksums=20are=20already=20= enabled=0A+=09=20*=20and=20you=20call=20pg_enable_data_checksums().=0A+=09= =20*/=0A+=09if=20(!launcher_running)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20= Prepare=20the=20BackgroundWorker=20and=20launch=20it.=0A+=09=09=20*/=0A+=09= =09memset(&bgw,=200,=20sizeof(bgw));=0A+=09=09bgw.bgw_flags=20=3D=20= BGWORKER_SHMEM_ACCESS=20|=20BGWORKER_BACKEND_DATABASE_CONNECTION;=0A+=09=09= bgw.bgw_start_time=20=3D=20BgWorkerStart_RecoveryFinished;=0A+=09=09= snprintf(bgw.bgw_library_name,=20BGW_MAXLEN,=20"postgres");=0A+=09=09= snprintf(bgw.bgw_function_name,=20BGW_MAXLEN,=20= "DataChecksumsWorkerLauncherMain");=0A+=09=09snprintf(bgw.bgw_name,=20= BGW_MAXLEN,=20"datachecksum=20launcher");=0A+=09=09= snprintf(bgw.bgw_type,=20BGW_MAXLEN,=20"datachecksum=20launcher");=0A+=09= =09bgw.bgw_restart_time=20=3D=20BGW_NEVER_RESTART;=0A+=09=09= bgw.bgw_notify_pid=20=3D=20MyProcPid;=0A+=09=09bgw.bgw_main_arg=20=3D=20= (Datum)=200;=0A+=0A+=09=09if=20(!RegisterDynamicBackgroundWorker(&bgw,=20= &bgw_handle))=0A+=09=09=09ereport(ERROR,=0A+=09=09=09=09=09= errcode(ERRCODE_INSUFFICIENT_RESOURCES),=0A+=09=09=09=09=09= errmsg("failed=20to=20start=20background=20worker=20to=20process=20data=20= checksums"));=0A+=09}=0A+=09else=0A+=09{=0A+=09=09if=20= (launcher_running_op=20=3D=3D=20op)=0A+=09=09=09ereport(ERROR,=0A+=09=09=09= =09=09errmsg("data=20checksum=20processing=20already=20running"));=0A+=09= }=0A+}=0A+=0A+/*=0A+=20*=20ProcessSingleRelationFork=0A+=20*=09=09Enable=20= data=20checksums=20in=20a=20single=20relation/fork.=0A+=20*=0A+=20*=20= Returns=20true=20if=20successful,=20and=20false=20if=20*aborted*.=20On=20= error,=20an=20actual=0A+=20*=20error=20is=20raised=20in=20the=20lower=20= levels.=0A+=20*/=0A+static=20bool=0A+ProcessSingleRelationFork(Relation=20= reln,=20ForkNumber=20forkNum,=20BufferAccessStrategy=20strategy)=0A+{=0A= +=09BlockNumber=20numblocks=20=3D=20= RelationGetNumberOfBlocksInFork(reln,=20forkNum);=0A+=09char=09=09= activity[NAMEDATALEN=20*=202=20+=20128];=0A+=09char=09=20=20=20*relns;=0A= +=0A+=09relns=20=3D=20get_namespace_name(RelationGetNamespace(reln));=0A= +=0A+=09/*=20Report=20the=20current=20relation=20to=20pgstat_activity=20= */=0A+=09snprintf(activity,=20sizeof(activity)=20-=201,=20"processing:=20= %s.%s=20(%s,=20%u=20blocks)",=0A+=09=09=09=20(relns=20?=20relns=20:=20= ""),=20RelationGetRelationName(reln),=20forkNames[forkNum],=20= numblocks);=0A+=09pgstat_report_activity(STATE_RUNNING,=20activity);=0A+=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_BLOCKS_TOTAL,=20= numblocks);=0A+=09if=20(relns)=0A+=09=09pfree(relns);=0A+=0A+=09/*=0A+=09= =20*=20We=20are=20looping=20over=20the=20blocks=20which=20existed=20at=20= the=20time=20of=20process=0A+=09=20*=20start,=20which=20is=20safe=20= since=20new=20blocks=20are=20created=20with=20checksums=20set=0A+=09=20*=20= already=20due=20to=20the=20state=20being=20"inprogress-on".=0A+=09=20*/=0A= +=09for=20(BlockNumber=20blknum=20=3D=200;=20blknum=20<=20numblocks;=20= blknum++)=0A+=09{=0A+=09=09Buffer=09=09buf=20=3D=20= ReadBufferExtended(reln,=20forkNum,=20blknum,=20RBM_NORMAL,=20strategy);=0A= +=0A+=09=09/*=20Need=20to=20get=20an=20exclusive=20lock=20to=20mark=20= the=20buffer=20as=20dirty=20*/=0A+=09=09LockBuffer(buf,=20= BUFFER_LOCK_EXCLUSIVE);=0A+=0A+=09=09/*=0A+=09=09=20*=20Mark=20the=20= buffer=20as=20dirty=20and=20force=20a=20full=20page=20write.=20=20We=20= have=20to=0A+=09=09=20*=20re-write=20the=20page=20to=20WAL=20even=20if=20= the=20checksum=20hasn't=20changed,=0A+=09=09=20*=20because=20if=20there=20= is=20a=20replica=20it=20might=20have=20a=20slightly=20different=0A+=09=09= =20*=20version=20of=20the=20page=20with=20an=20invalid=20checksum,=20= caused=20by=20unlogged=0A+=09=09=20*=20changes=20(e.g.=20hintbits)=20on=20= the=20primary=20happening=20while=20checksums=0A+=09=09=20*=20were=20= off.=20This=20can=20happen=20if=20there=20was=20a=20valid=20checksum=20= on=20the=20page=0A+=09=09=20*=20at=20one=20point=20in=20the=20past,=20so=20= only=20when=20checksums=20are=20first=20on,=20then=0A+=09=09=20*=20off,=20= and=20then=20turned=20on=20again.=20=20TODO:=20investigate=20if=20this=20= could=20be=0A+=09=09=20*=20avoided=20if=20the=20checksum=20is=20= calculated=20to=20be=20correct=20and=20wal_level=0A+=09=09=20*=20is=20= set=20to=20"minimal",=0A+=09=09=20*/=0A+=09=09START_CRIT_SECTION();=0A+=09= =09MarkBufferDirty(buf);=0A+=09=09log_newpage_buffer(buf,=20false);=0A+=09= =09END_CRIT_SECTION();=0A+=0A+=09=09UnlockReleaseBuffer(buf);=0A+=0A+=09=09= /*=0A+=09=09=20*=20This=20is=20the=20only=20place=20where=20we=20check=20= if=20we=20are=20asked=20to=20abort,=20the=0A+=09=09=20*=20abortion=20= will=20bubble=20up=20from=20here.=0A+=09=09=20*/=0A+=09=09= Assert(operation=20=3D=3D=20ENABLE_DATACHECKSUMS);=0A+=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_SHARED);=0A+=09=09if=20= (DataChecksumState->launch_operation=20=3D=3D=20DISABLE_DATACHECKSUMS)=0A= +=09=09=09abort_requested=20=3D=20true;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09=09if=20= (abort_requested)=0A+=09=09=09return=20false;=0A+=0A+=09=09/*=20update=20= the=20block=20counter=20*/=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_BLOCKS_DONE,=0A+=09=09= =09=09=09=09=09=09=09=20(blknum=20+=201));=0A+=0A+=09=09/*=0A+=09=09=20*=20= Processing=20is=20re-using=20the=20vacuum=20cost=20delay=20for=20process=0A= +=09=09=20*=20throttling,=20hence=20why=20we=20call=20vacuum=20APIs=20= here.=0A+=09=09=20*/=0A+=09=09vacuum_delay_point(false);=0A+=09}=0A+=0A+=09= return=20true;=0A+}=0A+=0A+/*=0A+=20*=20ProcessSingleRelationByOid=0A+=20= *=09=09Process=20a=20single=20relation=20based=20on=20oid.=0A+=20*=0A+=20= *=20Returns=20true=20if=20successful,=20and=20false=20if=20*aborted*.=20= On=20error,=20an=20actual=0A+=20*=20error=20is=20raised=20in=20the=20= lower=20levels.=0A+=20*/=0A+static=20bool=0A= +ProcessSingleRelationByOid(Oid=20relationId,=20BufferAccessStrategy=20= strategy)=0A+{=0A+=09Relation=09rel;=0A+=09bool=09=09aborted=20=3D=20= false;=0A+=0A+=09StartTransactionCommand();=0A+=0A+=09rel=20=3D=20= try_relation_open(relationId,=20AccessShareLock);=0A+=09if=20(rel=20=3D=3D= =20NULL)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20Relation=20no=20longer=20= exists.=20We=20don't=20consider=20this=20an=20error=20since=0A+=09=09=20= *=20there=20are=20no=20pages=20in=20it=20that=20need=20data=20checksums,=20= and=20thus=20return=0A+=09=09=20*=20true.=20The=20worker=20operates=20= off=20a=20list=20of=20relations=20generated=20at=20the=0A+=09=09=20*=20= start=20of=20processing,=20so=20relations=20being=20dropped=20in=20the=20= meantime=20is=0A+=09=09=20*=20to=20be=20expected.=0A+=09=09=20*/=0A+=09=09= CommitTransactionCommand();=0A+=09=09pgstat_report_activity(STATE_IDLE,=20= NULL);=0A+=09=09return=20true;=0A+=09}=0A+=09RelationGetSmgr(rel);=0A+=0A= +=09for=20(ForkNumber=20fnum=20=3D=200;=20fnum=20<=3D=20MAX_FORKNUM;=20= fnum++)=0A+=09{=0A+=09=09if=20(smgrexists(rel->rd_smgr,=20fnum))=0A+=09=09= {=0A+=09=09=09if=20(!ProcessSingleRelationFork(rel,=20fnum,=20strategy))=0A= +=09=09=09{=0A+=09=09=09=09aborted=20=3D=20true;=0A+=09=09=09=09break;=0A= +=09=09=09}=0A+=09=09}=0A+=09}=0A+=09relation_close(rel,=20= AccessShareLock);=0A+=0A+=09CommitTransactionCommand();=0A+=09= pgstat_report_activity(STATE_IDLE,=20NULL);=0A+=0A+=09return=20!aborted;=0A= +}=0A+=0A+/*=0A+=20*=20ProcessDatabase=0A+=20*=09=09Enable=20data=20= checksums=20in=20a=20single=20database.=0A+=20*=0A+=20*=20We=20do=20this=20= by=20launching=20a=20dynamic=20background=20worker=20into=20this=20= database,=20and=0A+=20*=20waiting=20for=20it=20to=20finish.=20=20We=20= have=20to=20do=20this=20in=20a=20separate=20worker,=20since=0A+=20*=20= each=20process=20can=20only=20be=20connected=20to=20one=20database=20= during=20its=20lifetime.=0A+=20*/=0A+static=20DataChecksumsWorkerResult=0A= +ProcessDatabase(DataChecksumsWorkerDatabase=20*db)=0A+{=0A+=09= BackgroundWorker=20bgw;=0A+=09BackgroundWorkerHandle=20*bgw_handle;=0A+=09= BgwHandleStatus=20status;=0A+=09pid_t=09=09pid;=0A+=09char=09=09= activity[NAMEDATALEN=20+=2064];=0A+=0A+=09DataChecksumState->success=20=3D= =20DATACHECKSUMSWORKER_FAILED;=0A+=0A+=09memset(&bgw,=200,=20= sizeof(bgw));=0A+=09bgw.bgw_flags=20=3D=20BGWORKER_SHMEM_ACCESS=20|=20= BGWORKER_BACKEND_DATABASE_CONNECTION;=0A+=09bgw.bgw_start_time=20=3D=20= BgWorkerStart_RecoveryFinished;=0A+=09snprintf(bgw.bgw_library_name,=20= BGW_MAXLEN,=20"postgres");=0A+=09snprintf(bgw.bgw_function_name,=20= BGW_MAXLEN,=20"%s",=20"DataChecksumsWorkerMain");=0A+=09= snprintf(bgw.bgw_name,=20BGW_MAXLEN,=20"datachecksum=20worker");=0A+=09= snprintf(bgw.bgw_type,=20BGW_MAXLEN,=20"datachecksum=20worker");=0A+=09= bgw.bgw_restart_time=20=3D=20BGW_NEVER_RESTART;=0A+=09bgw.bgw_notify_pid=20= =3D=20MyProcPid;=0A+=09bgw.bgw_main_arg=20=3D=20= ObjectIdGetDatum(db->dboid);=0A+=0A+=09/*=0A+=09=20*=20If=20there=20are=20= no=20worker=20slots=20available,=20there=20is=20little=20we=20can=20do.=20= =20If=0A+=09=20*=20we=20retry=20in=20a=20bit=20it's=20still=20unlikely=20= that=20the=20user=20has=20managed=20to=0A+=09=20*=20reconfigure=20in=20= the=20meantime=20and=20we'd=20be=20run=20through=20retries=20fast.=0A+=09= =20*/=0A+=09if=20(!RegisterDynamicBackgroundWorker(&bgw,=20&bgw_handle))=0A= +=09{=0A+=09=09ereport(WARNING,=0A+=09=09=09=09errmsg("could=20not=20= start=20background=20worker=20for=20enabling=20data=20checksums=20in=20= database=20\"%s\"",=0A+=09=09=09=09=09=20=20=20db->dbname),=0A+=09=09=09=09= errhint("The=20\"%s\"=20setting=20might=20be=20too=20low.",=20= "max_worker_processes"));=0A+=09=09return=20DATACHECKSUMSWORKER_FAILED;=0A= +=09}=0A+=0A+=09status=20=3D=20= WaitForBackgroundWorkerStartup(bgw_handle,=20&pid);=0A+=09if=20(status=20= =3D=3D=20BGWH_STOPPED)=0A+=09{=0A+=09=09ereport(WARNING,=0A+=09=09=09=09= errmsg("could=20not=20start=20background=20worker=20for=20enabling=20= data=20checksums=20in=20database=20\"%s\"",=0A+=09=09=09=09=09=20=20=20= db->dbname),=0A+=09=09=09=09errhint("More=20details=20on=20the=20error=20= might=20be=20found=20in=20the=20server=20log."));=0A+=0A+=09=09/*=0A+=09=09= =20*=20Heuristic=20to=20see=20if=20the=20database=20was=20dropped,=20and=20= if=20it=20was=20we=20can=0A+=09=09=20*=20treat=20it=20as=20not=20an=20= error,=20else=20treat=20as=20fatal=20and=20error=20out.=20TODO:=0A+=09=09= =20*=20this=20could=20probably=20be=20improved=20with=20a=20tighter=20= check.=0A+=09=09=20*/=0A+=09=09if=20(DatabaseExists(db->dboid))=0A+=09=09= =09return=20DATACHECKSUMSWORKER_FAILED;=0A+=09=09else=0A+=09=09=09return=20= DATACHECKSUMSWORKER_DROPDB;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20If=20the=20= postmaster=20crashed=20we=20cannot=20end=20up=20with=20a=20processed=20= database=20so=0A+=09=20*=20we=20have=20no=20alternative=20other=20than=20= exiting.=20When=20enabling=20checksums=20we=0A+=09=20*=20won't=20at=20= this=20time=20have=20changed=20the=20data=20checksums=20state=20in=20= pg_control=0A+=09=20*=20to=20enabled=20so=20when=20the=20cluster=20comes=20= back=20up=20processing=20will=20have=20to=20be=0A+=09=20*=20restarted.=0A= +=09=20*/=0A+=09if=20(status=20=3D=3D=20BGWH_POSTMASTER_DIED)=0A+=09=09= ereport(FATAL,=0A+=09=09=09=09errcode(ERRCODE_ADMIN_SHUTDOWN),=0A+=09=09=09= =09errmsg("cannot=20enable=20data=20checksums=20without=20the=20= postmaster=20process"),=0A+=09=09=09=09errhint("Restart=20the=20database=20= and=20restart=20data=20checksum=20processing=20by=20calling=20= pg_enable_data_checksums()."));=0A+=0A+=09Assert(status=20=3D=3D=20= BGWH_STARTED);=0A+=09ereport(LOG,=0A+=09=09=09errmsg("initiating=20data=20= checksum=20processing=20in=20database=20\"%s\"",=0A+=09=09=09=09=20=20=20= db->dbname));=0A+=0A+=09/*=20Save=20the=20pid=20of=20the=20worker=20so=20= we=20can=20signal=20it=20later=20*/=0A+=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09= DataChecksumState->worker_running=20=3D=20pid;=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09snprintf(activity,=20= sizeof(activity)=20-=201,=0A+=09=09=09=20"Waiting=20for=20worker=20in=20= database=20%s=20(pid=20%ld)",=20db->dbname,=20(long)=20pid);=0A+=09= pgstat_report_activity(STATE_RUNNING,=20activity);=0A+=0A+=09status=20=3D=20= WaitForBackgroundWorkerShutdown(bgw_handle);=0A+=09if=20(status=20=3D=3D=20= BGWH_POSTMASTER_DIED)=0A+=09=09ereport(FATAL,=0A+=09=09=09=09= errcode(ERRCODE_ADMIN_SHUTDOWN),=0A+=09=09=09=09errmsg("postmaster=20= exited=20during=20data=20checksum=20processing=20in=20\"%s\"",=0A+=09=09=09= =09=09=20=20=20db->dbname),=0A+=09=09=09=09errhint("Restart=20the=20= database=20and=20restart=20data=20checksum=20processing=20by=20calling=20= pg_enable_data_checksums()."));=0A+=0A+=09if=20= (DataChecksumState->success=20=3D=3D=20DATACHECKSUMSWORKER_ABORTED)=0A+=09= =09ereport(LOG,=0A+=09=09=09=09errmsg("data=20checksums=20processing=20= was=20aborted=20in=20database=20\"%s\"",=0A+=09=09=09=09=09=20=20=20= db->dbname));=0A+=0A+=09pgstat_report_activity(STATE_IDLE,=20NULL);=0A+=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09= DataChecksumState->worker_running=20=3D=20InvalidPid;=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09return=20= DataChecksumState->success;=0A+}=0A+=0A+/*=0A+=20*=20launcher_exit=0A+=20= *=0A+=20*=20Internal=20routine=20for=20cleaning=20up=20state=20when=20= the=20launcher=20process=20exits.=20We=0A+=20*=20need=20to=20clean=20up=20= the=20abort=20flag=20to=20ensure=20that=20processing=20started=20again=20= if=0A+=20*=20it=20was=20previously=20aborted=20(note:=20started=20again,=20= *not*=20restarted=20from=20where=0A+=20*=20it=20left=20off).=0A+=20*/=0A= +static=20void=0A+launcher_exit(int=20code,=20Datum=20arg)=0A+{=0A+=09= abort_requested=20=3D=20false;=0A+=0A+=09if=20(launcher_running)=0A+=09{=0A= +=09=09LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09= launcher_running=20=3D=20false;=0A+=09=09= DataChecksumState->launcher_running=20=3D=20false;=0A+=0A+=09=09if=20= (DataChecksumState->worker_running=20!=3D=20InvalidPid)=0A+=09=09{=0A+=09= =09=09ereport(LOG,=0A+=09=09=09=09=09errmsg("data=20checksums=20launcher=20= exiting=20while=20worker=20is=20still=20running,=20signalling=20= worker"));=0A+=09=09=09kill(DataChecksumState->worker_running,=20= SIGTERM);=0A+=09=09}=0A+=09=09LWLockRelease(DataChecksumsWorkerLock);=0A= +=09}=0A+=0A+=09/*=0A+=09=20*=20If=20the=20launcher=20is=20exiting=20= before=20data=20checksums=20are=20enabled=20then=20set=0A+=09=20*=20the=20= state=20to=20off=20since=20processing=20cannot=20be=20resumed.=0A+=09=20= */=0A+=09if=20(DataChecksumsInProgress())=0A+=09=09= SetDataChecksumsOff();=0A+}=0A+=0A+/*=0A+=20*=20launcher_cancel_handler=0A= +=20*=0A+=20*=20Internal=20routine=20for=20reacting=20to=20SIGINT=20and=20= flagging=20the=20worker=20to=20abort.=0A+=20*=20The=20worker=20won't=20= be=20interrupted=20immediately=20but=20will=20check=20for=20abort=20flag=0A= +=20*=20between=20each=20block=20in=20a=20relation.=0A+=20*/=0A+static=20= void=0A+launcher_cancel_handler(SIGNAL_ARGS)=0A+{=0A+=09int=09=09=09= save_errno=20=3D=20errno;=0A+=0A+=09abort_requested=20=3D=20true;=0A+=0A= +=09/*=0A+=09=20*=20There=20is=20no=20sleeping=20in=20the=20main=20loop,=20= the=20flag=20will=20be=20checked=0A+=09=20*=20periodically=20in=20= ProcessSingleRelationFork.=20The=20worker=20does=20however=0A+=09=20*=20= sleep=20when=20waiting=20for=20concurrent=20transactions=20to=20end=20so=20= we=20still=20need=0A+=09=20*=20to=20set=20the=20latch.=0A+=09=20*/=0A+=09= SetLatch(MyLatch);=0A+=0A+=09errno=20=3D=20save_errno;=0A+}=0A+=0A+/*=0A= +=20*=20WaitForAllTransactionsToFinish=0A+=20*=09=09Blocks=20awaiting=20= all=20current=20transactions=20to=20finish=0A+=20*=0A+=20*=20Returns=20= when=20all=20transactions=20which=20are=20active=20at=20the=20call=20of=20= the=20function=0A+=20*=20have=20ended,=20or=20if=20the=20postmaster=20= dies=20while=20waiting.=20If=20the=20postmaster=20dies=0A+=20*=20the=20= abort=20flag=20will=20be=20set=20to=20indicate=20that=20the=20caller=20= of=20this=20shouldn't=0A+=20*=20proceed.=0A+=20*=0A+=20*=20NB:=20this=20= will=20return=20early,=20if=20aborted=20by=20SIGINT=20or=20if=20the=20= target=20state=0A+=20*=20is=20changed=20while=20we're=20running.=0A+=20= */=0A+static=20void=0A+WaitForAllTransactionsToFinish(void)=0A+{=0A+=09= TransactionId=20waitforxid;=0A+=0A+=09LWLockAcquire(XidGenLock,=20= LW_SHARED);=0A+=09waitforxid=20=3D=20= XidFromFullTransactionId(TransamVariables->nextXid);=0A+=09= LWLockRelease(XidGenLock);=0A+=0A+=09while=20= (TransactionIdPrecedes(GetOldestActiveTransactionId(false,=20true),=20= waitforxid))=0A+=09{=0A+=09=09char=09=09activity[64];=0A+=09=09int=09=09=09= rc;=0A+=0A+=09=09/*=20Oldest=20running=20xid=20is=20older=20than=20us,=20= so=20wait=20*/=0A+=09=09snprintf(activity,=0A+=09=09=09=09=20= sizeof(activity),=0A+=09=09=09=09=20"Waiting=20for=20current=20= transactions=20to=20finish=20(waiting=20for=20%u)",=0A+=09=09=09=09=20= waitforxid);=0A+=09=09pgstat_report_activity(STATE_RUNNING,=20activity);=0A= +=0A+=09=09/*=20Retry=20every=203=20seconds=20*/=0A+=09=09= ResetLatch(MyLatch);=0A+=09=09rc=20=3D=20WaitLatch(MyLatch,=0A+=09=09=09=09= =09=20=20=20WL_LATCH_SET=20|=20WL_TIMEOUT=20|=20WL_POSTMASTER_DEATH,=0A+=09= =09=09=09=09=20=20=203000,=0A+=09=09=09=09=09=20=20=20= WAIT_EVENT_CHECKSUM_ENABLE_STARTCONDITION);=0A+=0A+=09=09/*=0A+=09=09=20= *=20If=20the=20postmaster=20died=20we=20won't=20be=20able=20to=20enable=20= checksums=0A+=09=09=20*=20cluster-wide=20so=20abort=20and=20hope=20to=20= continue=20when=20restarted.=0A+=09=09=20*/=0A+=09=09if=20(rc=20&=20= WL_POSTMASTER_DEATH)=0A+=09=09=09ereport(FATAL,=0A+=09=09=09=09=09= errcode(ERRCODE_ADMIN_SHUTDOWN),=0A+=09=09=09=09=09errmsg("postmaster=20= exited=20during=20data=20checksums=20processing"),=0A+=09=09=09=09=09= errhint("Data=20checksums=20processing=20must=20be=20restarted=20= manually=20after=20cluster=20restart."));=0A+=0A+=09=09= CHECK_FOR_INTERRUPTS();=0A+=0A+=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_SHARED);=0A+=09=09if=20= (DataChecksumState->launch_operation=20!=3D=20operation)=0A+=09=09=09= abort_requested=20=3D=20true;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09if=20(abort_requested)=0A= +=09=09=09break;=0A+=09}=0A+=0A+=09pgstat_report_activity(STATE_IDLE,=20= NULL);=0A+=09return;=0A+}=0A+=0A+/*=0A+=20*=20= DataChecksumsWorkerLauncherMain=0A+=20*=0A+=20*=20Main=20function=20for=20= launching=20dynamic=20background=20workers=20for=20processing=20data=0A+=20= *=20checksums=20in=20databases.=20This=20function=20has=20the=20bgworker=20= management,=20with=0A+=20*=20ProcessAllDatabases=20being=20responsible=20= for=20looping=20over=20the=20databases=20and=0A+=20*=20initiating=20= processing.=0A+=20*/=0A+void=0A+DataChecksumsWorkerLauncherMain(Datum=20= arg)=0A+{=0A+=09on_shmem_exit(launcher_exit,=200);=0A+=0A+=09= ereport(DEBUG1,=0A+=09=09=09errmsg("background=20worker=20= \"datachecksums=20launcher\"=20started"));=0A+=0A+=09pqsignal(SIGTERM,=20= die);=0A+=09pqsignal(SIGINT,=20launcher_cancel_handler);=0A+=09= pqsignal(SIGUSR1,=20procsignal_sigusr1_handler);=0A+=09pqsignal(SIGUSR2,=20= SIG_IGN);=0A+=0A+=09BackgroundWorkerUnblockSignals();=0A+=0A+=09= MyBackendType=20=3D=20B_DATACHECKSUMSWORKER_LAUNCHER;=0A+=09= init_ps_display(NULL);=0A+=0A+=09= INJECTION_POINT("datachecksumsworker-launcher-delay",=20NULL);=0A+=0A+=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=0A+=09if=20= (DataChecksumState->launcher_running)=0A+=09{=0A+=09=09ereport(LOG,=0A+=09= =09=09=09errmsg("background=20worker=20\"datachecksums=20launcher\"=20= already=20running,=20exiting"));=0A+=09=09/*=20Launcher=20was=20already=20= running,=20let=20it=20finish=20*/=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09return;=0A+=09}=0A+=0A+=09= launcher_running=20=3D=20true;=0A+=0A+=09/*=20Initialize=20a=20= connection=20to=20shared=20catalogs=20only=20*/=0A+=09= BackgroundWorkerInitializeConnectionByOid(InvalidOid,=20InvalidOid,=20= 0);=0A+=0A+=09operation=20=3D=20DataChecksumState->launch_operation;=0A+=09= DataChecksumState->launcher_running=20=3D=20true;=0A+=09= DataChecksumState->operation=20=3D=20operation;=0A+=09= DataChecksumState->cost_delay=20=3D=20= DataChecksumState->launch_cost_delay;=0A+=09= DataChecksumState->cost_limit=20=3D=20= DataChecksumState->launch_cost_limit;=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09/*=0A+=09=20*=20The=20= target=20state=20can=20change=20while=20we=20are=20busy=20= enabling/disabling=0A+=09=20*=20checksums,=20if=20the=20user=20calls=20= pg_disable/enable_data_checksums()=20before=0A+=09=20*=20we=20are=20= finished=20with=20the=20previous=20request.=20In=20that=20case,=20we=20= will=20loop=0A+=09=20*=20back=20here,=20to=20process=20the=20new=20= request.=0A+=09=20*/=0A+again:=0A+=0A+=09= pgstat_progress_start_command(PROGRESS_COMMAND_DATACHECKSUMS,=0A+=09=09=09= =09=09=09=09=09=20=20InvalidOid);=0A+=0A+=09if=20(operation=20=3D=3D=20= ENABLE_DATACHECKSUMS)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20If=20we=20are=20= asked=20to=20enable=20checksums=20in=20a=20cluster=20which=20already=20= has=0A+=09=09=20*=20checksums=20enabled,=20exit=20immediately=20as=20= there=20is=20nothing=20more=20to=20do.=0A+=09=09=20*/=0A+=09=09if=20= (DataChecksumsNeedVerify())=0A+=09=09=09goto=20done;=0A+=0A+=09=09= ereport(LOG,=0A+=09=09=09=09errmsg("enabling=20data=20checksums=20= requested,=20starting=20data=20checksum=20calculation"));=0A+=0A+=09=09= /*=0A+=09=09=20*=20Set=20the=20state=20to=20inprogress-on=20and=20wait=20= on=20the=20procsignal=20barrier.=0A+=09=09=20*/=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_ENABLING);=0A+=09=09= SetDataChecksumsOnInProgress();=0A+=0A+=09=09/*=0A+=09=09=20*=20All=20= backends=20are=20now=20in=20inprogress-on=20state=20and=20are=20writing=20= data=0A+=09=09=20*=20checksums.=20=20Start=20processing=20all=20data=20= at=20rest.=0A+=09=09=20*/=0A+=09=09if=20(!ProcessAllDatabases())=0A+=09=09= {=0A+=09=09=09/*=0A+=09=09=09=20*=20If=20the=20target=20state=20changed=20= during=20processing=20then=20it's=20not=20a=0A+=09=09=09=20*=20failure,=20= so=20restart=20processing=20instead.=0A+=09=09=09=20*/=0A+=09=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09=09if=20= (DataChecksumState->launch_operation=20!=3D=20operation)=0A+=09=09=09{=0A= +=09=09=09=09LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09=09=09goto=20= done;=0A+=09=09=09}=0A+=09=09=09LWLockRelease(DataChecksumsWorkerLock);=0A= +=09=09=09ereport(ERROR,=0A+=09=09=09=09=09= errcode(ERRCODE_INSUFFICIENT_RESOURCES),=0A+=09=09=09=09=09= errmsg("unable=20to=20enable=20data=20checksums=20in=20cluster"));=0A+=09= =09}=0A+=0A+=09=09/*=0A+=09=09=20*=20Data=20checksums=20have=20been=20= set=20on=20all=20pages,=20set=20the=20state=20to=20on=20in=0A+=09=09=20*=20= order=20to=20instruct=20backends=20to=20validate=20checksums=20on=20= reading.=0A+=09=09=20*/=0A+=09=09SetDataChecksumsOn();=0A+=0A+=09=09= ereport(LOG,=0A+=09=09=09=09errmsg("data=20checksums=20are=20now=20= enabled"));=0A+=09}=0A+=09else=20if=20(operation=20=3D=3D=20= DISABLE_DATACHECKSUMS)=0A+=09{=0A+=09=09ereport(LOG,=0A+=09=09=09=09= errmsg("disabling=20data=20checksums=20requested"));=0A+=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_DISABLING);=0A+=09=09= SetDataChecksumsOff();=0A+=09=09ereport(LOG,=0A+=09=09=09=09errmsg("data=20= checksums=20are=20now=20disabled"));=0A+=09}=0A+=09else=0A+=09=09= Assert(false);=0A+=0A+done:=0A+=0A+=09/*=0A+=09=20*=20This=20state=20= will=20only=20be=20displayed=20for=20a=20fleeting=20moment,=20but=20for=20= the=0A+=09=20*=20sake=20of=20correctness=20it=20is=20still=20added=20= before=20ending=20the=20command.=0A+=09=20*/=0A+=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_DONE);=0A+=0A+=09/*=0A+=09=20= *=20All=20done.=20But=20before=20we=20exit,=20check=20if=20the=20target=20= state=20was=20changed=0A+=09=20*=20while=20we=20were=20running.=20In=20= that=20case=20we=20will=20have=20to=20start=20all=20over=0A+=09=20*=20= again.=0A+=09=20*/=0A+=09LWLockAcquire(DataChecksumsWorkerLock,=20= LW_EXCLUSIVE);=0A+=09if=20(DataChecksumState->launch_operation=20!=3D=20= operation)=0A+=09{=0A+=09=09DataChecksumState->operation=20=3D=20= DataChecksumState->launch_operation;=0A+=09=09operation=20=3D=20= DataChecksumState->launch_operation;=0A+=09=09= DataChecksumState->cost_delay=20=3D=20= DataChecksumState->launch_cost_delay;=0A+=09=09= DataChecksumState->cost_limit=20=3D=20= DataChecksumState->launch_cost_limit;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09goto=20again;=0A+=09}=0A= +=0A+=09/*=20Shut=20down=20progress=20reporting=20as=20we=20are=20done=20= */=0A+=09pgstat_progress_end_command();=0A+=0A+=09launcher_running=20=3D=20= false;=0A+=09DataChecksumState->launcher_running=20=3D=20false;=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A+}=0A+=0A+/*=0A+=20*=20= ProcessAllDatabases=0A+=20*=09=09Compute=20the=20list=20of=20all=20= databases=20and=20process=20checksums=20in=20each=0A+=20*=0A+=20*=20This=20= will=20generate=20a=20list=20of=20databases=20to=20process=20for=20= enabling=20checksums.=0A+=20*=20If=20a=20database=20encounters=20a=20= failure=20then=20processing=20will=20end=20immediately=20and=0A+=20*=20= return=20an=20error.=0A+=20*/=0A+static=20bool=0A= +ProcessAllDatabases(void)=0A+{=0A+=09List=09=20=20=20*DatabaseList;=0A+=09= int=09=09=09cumulative_total=20=3D=200;=0A+=0A+=09/*=20Set=20up=20so=20= first=20run=20processes=20shared=20catalogs,=20not=20once=20in=20every=20= db=20*/=0A+=09DataChecksumState->process_shared_catalogs=20=3D=20true;=0A= +=0A+=09/*=20Get=20a=20list=20of=20all=20databases=20to=20process=20*/=0A= +=09WaitForAllTransactionsToFinish();=0A+=09DatabaseList=20=3D=20= BuildDatabaseList();=0A+=0A+=09/*=0A+=09=20*=20Update=20progress=20= reporting=20with=20the=20total=20number=20of=20databases=20we=20need=20= to=0A+=09=20*=20process.=20=20This=20number=20should=20not=20be=20= changed=20during=20processing,=20the=0A+=09=20*=20columns=20for=20= processed=20databases=20is=20instead=20increased=20such=20that=20it=20= can=0A+=09=20*=20be=20compared=20against=20the=20total.=0A+=09=20*/=0A+=09= {=0A+=09=09const=20int=09index[]=20=3D=20{=0A+=09=09=09= PROGRESS_DATACHECKSUMS_DBS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_DBS_DONE,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_DONE,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_BLOCKS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_BLOCKS_DONE,=0A+=09=09};=0A+=0A+=09=09int64=09=09= vals[6];=0A+=0A+=09=09vals[0]=20=3D=20list_length(DatabaseList);=0A+=09=09= vals[1]=20=3D=200;=0A+=09=09/*=20translated=20to=20NULL=20*/=0A+=09=09= vals[2]=20=3D=20-1;=0A+=09=09vals[3]=20=3D=20-1;=0A+=09=09vals[4]=20=3D=20= -1;=0A+=09=09vals[5]=20=3D=20-1;=0A+=0A+=09=09= pgstat_progress_update_multi_param(6,=20index,=20vals);=0A+=09}=0A+=0A+=09= foreach_ptr(DataChecksumsWorkerDatabase,=20db,=20DatabaseList)=0A+=09{=0A= +=09=09DataChecksumsWorkerResult=20result;=0A+=0A+=09=09result=20=3D=20= ProcessDatabase(db);=0A+=0A+=09=09/*=20Allow=20a=20test=20process=20to=20= alter=20the=20result=20of=20the=20operation=20*/=0A+=09=09= INJECTION_POINT("datachecksumsworker-modify-db-result",=20&result);=0A+=0A= +=09=09pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_DBS_DONE,=0A+=09= =09=09=09=09=09=09=09=09=20++cumulative_total);=0A+=0A+=09=09if=20= (result=20=3D=3D=20DATACHECKSUMSWORKER_FAILED)=0A+=09=09{=0A+=09=09=09/*=0A= +=09=09=09=20*=20Disable=20checksums=20on=20cluster,=20because=20we=20= failed=20one=20of=20the=0A+=09=09=09=20*=20databases=20and=20this=20is=20= an=20all=20or=20nothing=20process.=0A+=09=09=09=20*/=0A+=09=09=09= SetDataChecksumsOff();=0A+=09=09=09ereport(ERROR,=0A+=09=09=09=09=09= errcode(ERRCODE_INSUFFICIENT_RESOURCES),=0A+=09=09=09=09=09errmsg("data=20= checksums=20failed=20to=20get=20enabled=20in=20all=20databases,=20= aborting"),=0A+=09=09=09=09=09errhint("The=20server=20log=20might=20have=20= more=20information=20on=20the=20cause=20of=20the=20error."));=0A+=09=09}=0A= +=09=09else=20if=20(result=20=3D=3D=20DATACHECKSUMSWORKER_ABORTED=20||=20= abort_requested)=0A+=09=09{=0A+=09=09=09/*=20Abort=20flag=20set,=20so=20= exit=20the=20whole=20process=20*/=0A+=09=09=09return=20false;=0A+=09=09}=0A= +=0A+=09=09/*=0A+=09=09=20*=20When=20one=20database=20has=20completed,=20= it=20will=20have=20done=20shared=20catalogs=0A+=09=09=20*=20so=20we=20= don't=20have=20to=20process=20them=20again.=0A+=09=09=20*/=0A+=09=09= DataChecksumState->process_shared_catalogs=20=3D=20false;=0A+=09}=0A+=0A= +=09FreeDatabaseList(DatabaseList);=0A+=0A+=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_WAITING_BARRIER);=0A+=09= return=20true;=0A+}=0A+=0A+/*=0A+=20*=20DataChecksumStateSize=0A+=20*=09=09= Compute=20required=20space=20for=20datachecksumsworker-related=20shared=20= memory=0A+=20*/=0A+Size=0A+DataChecksumsShmemSize(void)=0A+{=0A+=09Size=09= =09size;=0A+=0A+=09size=20=3D=20sizeof(DataChecksumsStateStruct);=0A+=09= size=20=3D=20MAXALIGN(size);=0A+=0A+=09return=20size;=0A+}=0A+=0A+/*=0A+=20= *=20DataChecksumStateInit=0A+=20*=09=09Allocate=20and=20initialize=20= datachecksumsworker-related=20shared=20memory=0A+=20*/=0A+void=0A= +DataChecksumsShmemInit(void)=0A+{=0A+=09bool=09=09found;=0A+=0A+=09= DataChecksumState=20=3D=20(DataChecksumsStateStruct=20*)=0A+=09=09= ShmemInitStruct("DataChecksumsWorker=20Data",=0A+=09=09=09=09=09=09= DataChecksumsShmemSize(),=0A+=09=09=09=09=09=09&found);=0A+=09if=20= (!found)=0A+=09=09MemSet(DataChecksumState,=200,=20= DataChecksumsShmemSize());=0A+}=0A+=0A+/*=0A+=20*=20DatabaseExists=0A+=20= *=0A+=20*=20Scans=20the=20system=20catalog=20to=20check=20if=20a=20= database=20with=20the=20given=20Oid=20exist=0A+=20*=20and=20returns=20= true=20if=20it=20is=20found,=20else=20false.=0A+=20*/=0A+static=20bool=0A= +DatabaseExists(Oid=20dboid)=0A+{=0A+=09Relation=09rel;=0A+=09= ScanKeyData=20skey;=0A+=09SysScanDesc=20scan;=0A+=09bool=09=09found;=0A+=09= HeapTuple=09tuple;=0A+=0A+=09StartTransactionCommand();=0A+=0A+=09rel=20= =3D=20table_open(DatabaseRelationId,=20AccessShareLock);=0A+=09= ScanKeyInit(&skey,=0A+=09=09=09=09Anum_pg_database_oid,=0A+=09=09=09=09= BTEqualStrategyNumber,=20F_OIDEQ,=0A+=09=09=09=09dboid);=0A+=09scan=20=3D=20= systable_beginscan(rel,=20DatabaseOidIndexId,=20true,=20SnapshotSelf,=0A= +=09=09=09=09=09=09=09=20=201,=20&skey);=0A+=09tuple=20=3D=20= systable_getnext(scan);=0A+=09found=20=3D=20HeapTupleIsValid(tuple);=0A+=0A= +=09systable_endscan(scan);=0A+=09table_close(rel,=20AccessShareLock);=0A= +=0A+=09CommitTransactionCommand();=0A+=0A+=09return=20found;=0A+}=0A+=0A= +/*=0A+=20*=20BuildDatabaseList=0A+=20*=09=09Compile=20a=20list=20of=20= all=20currently=20available=20databases=20in=20the=20cluster=0A+=20*=0A+=20= *=20This=20creates=20the=20list=20of=20databases=20for=20the=20= datachecksumsworker=20workers=20to=0A+=20*=20add=20checksums=20to.=20If=20= the=20caller=20wants=20to=20ensure=20that=20no=20concurrently=0A+=20*=20= running=20CREATE=20DATABASE=20calls=20exist,=20this=20needs=20to=20be=20= preceded=20by=20a=20call=0A+=20*=20to=20= WaitForAllTransactionsToFinish().=0A+=20*/=0A+static=20List=20*=0A= +BuildDatabaseList(void)=0A+{=0A+=09List=09=20=20=20*DatabaseList=20=3D=20= NIL;=0A+=09Relation=09rel;=0A+=09TableScanDesc=20scan;=0A+=09HeapTuple=09= tup;=0A+=09MemoryContext=20ctx=20=3D=20CurrentMemoryContext;=0A+=09= MemoryContext=20oldctx;=0A+=0A+=09StartTransactionCommand();=0A+=0A+=09= rel=20=3D=20table_open(DatabaseRelationId,=20AccessShareLock);=0A+=09= scan=20=3D=20table_beginscan_catalog(rel,=200,=20NULL);=0A+=0A+=09while=20= (HeapTupleIsValid(tup=20=3D=20heap_getnext(scan,=20= ForwardScanDirection)))=0A+=09{=0A+=09=09Form_pg_database=20pgdb=20=3D=20= (Form_pg_database)=20GETSTRUCT(tup);=0A+=09=09= DataChecksumsWorkerDatabase=20*db;=0A+=0A+=09=09oldctx=20=3D=20= MemoryContextSwitchTo(ctx);=0A+=0A+=09=09db=20=3D=20= (DataChecksumsWorkerDatabase=20*)=20= palloc0(sizeof(DataChecksumsWorkerDatabase));=0A+=0A+=09=09db->dboid=20=3D= =20pgdb->oid;=0A+=09=09db->dbname=20=3D=20= pstrdup(NameStr(pgdb->datname));=0A+=0A+=09=09DatabaseList=20=3D=20= lappend(DatabaseList,=20db);=0A+=0A+=09=09MemoryContextSwitchTo(oldctx);=0A= +=09}=0A+=0A+=09table_endscan(scan);=0A+=09table_close(rel,=20= AccessShareLock);=0A+=0A+=09CommitTransactionCommand();=0A+=0A+=09return=20= DatabaseList;=0A+}=0A+=0A+static=20void=0A+FreeDatabaseList(List=20= *dblist)=0A+{=0A+=09if=20(!dblist)=0A+=09=09return;=0A+=0A+=09= foreach_ptr(DataChecksumsWorkerDatabase,=20db,=20dblist)=0A+=09{=0A+=09=09= if=20(db->dbname=20!=3D=20NULL)=0A+=09=09=09pfree(db->dbname);=0A+=09}=0A= +=0A+=09list_free_deep(dblist);=0A+}=0A+=0A+/*=0A+=20*=20= BuildRelationList=0A+=20*=09=09Compile=20a=20list=20of=20relations=20in=20= the=20database=0A+=20*=0A+=20*=20Returns=20a=20list=20of=20OIDs=20for=20= the=20request=20relation=20types.=20If=20temp_relations=0A+=20*=20is=20= True=20then=20only=20temporary=20relations=20are=20returned.=20If=20= temp_relations=20is=0A+=20*=20False=20then=20non-temporary=20relations=20= which=20have=20data=20checksums=20are=20returned.=0A+=20*=20If=20= include_shared=20is=20True=20then=20shared=20relations=20are=20included=20= as=20well=20in=20a=0A+=20*=20non-temporary=20list.=20include_shared=20= has=20no=20relevance=20when=20building=20a=20list=20of=0A+=20*=20= temporary=20relations.=0A+=20*/=0A+static=20List=20*=0A= +BuildRelationList(bool=20temp_relations,=20bool=20include_shared)=0A+{=0A= +=09List=09=20=20=20*RelationList=20=3D=20NIL;=0A+=09Relation=09rel;=0A+=09= TableScanDesc=20scan;=0A+=09HeapTuple=09tup;=0A+=09MemoryContext=20ctx=20= =3D=20CurrentMemoryContext;=0A+=09MemoryContext=20oldctx;=0A+=0A+=09= StartTransactionCommand();=0A+=0A+=09rel=20=3D=20= table_open(RelationRelationId,=20AccessShareLock);=0A+=09scan=20=3D=20= table_beginscan_catalog(rel,=200,=20NULL);=0A+=0A+=09while=20= (HeapTupleIsValid(tup=20=3D=20heap_getnext(scan,=20= ForwardScanDirection)))=0A+=09{=0A+=09=09Form_pg_class=20pgc=20=3D=20= (Form_pg_class)=20GETSTRUCT(tup);=0A+=0A+=09=09/*=20Only=20include=20= temporary=20relations=20when=20explicitly=20asked=20to=20*/=0A+=09=09if=20= (pgc->relpersistence=20=3D=3D=20RELPERSISTENCE_TEMP)=0A+=09=09{=0A+=09=09= =09if=20(!temp_relations)=0A+=09=09=09=09continue;=0A+=09=09}=0A+=09=09= else=0A+=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20If=20we=20are=20only=20= interested=20in=20temp=20relations=20then=20continue=0A+=09=09=09=20*=20= immediately=20as=20the=20current=20relation=20isn't=20a=20temp=20= relation.=0A+=09=09=09=20*/=0A+=09=09=09if=20(temp_relations)=0A+=09=09=09= =09continue;=0A+=0A+=09=09=09if=20(!RELKIND_HAS_STORAGE(pgc->relkind))=0A= +=09=09=09=09continue;=0A+=0A+=09=09=09if=20(pgc->relisshared=20&&=20= !include_shared)=0A+=09=09=09=09continue;=0A+=09=09}=0A+=0A+=09=09oldctx=20= =3D=20MemoryContextSwitchTo(ctx);=0A+=09=09RelationList=20=3D=20= lappend_oid(RelationList,=20pgc->oid);=0A+=09=09= MemoryContextSwitchTo(oldctx);=0A+=09}=0A+=0A+=09table_endscan(scan);=0A= +=09table_close(rel,=20AccessShareLock);=0A+=0A+=09= CommitTransactionCommand();=0A+=0A+=09return=20RelationList;=0A+}=0A+=0A= +/*=0A+=20*=20DataChecksumsWorkerMain=0A+=20*=0A+=20*=20Main=20function=20= for=20enabling=20checksums=20in=20a=20single=20database,=20This=20is=20= the=0A+=20*=20function=20set=20as=20the=20bgw_function_name=20in=20the=20= dynamic=20background=20worker=0A+=20*=20process=20initiated=20for=20each=20= database=20by=20the=20worker=20launcher.=20After=20enabling=0A+=20*=20= data=20checksums=20in=20each=20applicable=20relation=20in=20the=20= database,=20it=20will=20wait=20for=0A+=20*=20all=20temporary=20relations=20= that=20were=20present=20when=20the=20function=20started=20to=0A+=20*=20= disappear=20before=20returning.=20This=20is=20required=20since=20we=20= cannot=20rewrite=0A+=20*=20existing=20temporary=20relations=20with=20= data=20checksums.=0A+=20*/=0A+void=0A+DataChecksumsWorkerMain(Datum=20= arg)=0A+{=0A+=09Oid=09=09=09dboid=20=3D=20DatumGetObjectId(arg);=0A+=09= List=09=20=20=20*RelationList=20=3D=20NIL;=0A+=09List=09=20=20=20= *InitialTempTableList=20=3D=20NIL;=0A+=09BufferAccessStrategy=20= strategy;=0A+=09bool=09=09aborted=20=3D=20false;=0A+=09int64=09=09= rels_done;=0A+=0A+=09operation=20=3D=20ENABLE_DATACHECKSUMS;=0A+=0A+=09= pqsignal(SIGTERM,=20die);=0A+=09pqsignal(SIGUSR1,=20= procsignal_sigusr1_handler);=0A+=0A+=09BackgroundWorkerUnblockSignals();=0A= +=0A+=09MyBackendType=20=3D=20B_DATACHECKSUMSWORKER_WORKER;=0A+=09= init_ps_display(NULL);=0A+=0A+=09= BackgroundWorkerInitializeConnectionByOid(dboid,=20InvalidOid,=0A+=09=09=09= =09=09=09=09=09=09=09=09=20=20BGWORKER_BYPASS_ALLOWCONN);=0A+=0A+=09/*=20= worker=20will=20have=20a=20separate=20entry=20in=20= pg_stat_progress_data_checksums=20*/=0A+=09= pgstat_progress_start_command(PROGRESS_COMMAND_DATACHECKSUMS,=0A+=09=09=09= =09=09=09=09=09=20=20InvalidOid);=0A+=0A+=09/*=0A+=09=20*=20Get=20a=20= list=20of=20all=20temp=20tables=20present=20as=20we=20start=20in=20this=20= database.=20We=0A+=09=20*=20need=20to=20wait=20until=20they=20are=20all=20= gone=20until=20we=20are=20done,=20since=20we=20cannot=0A+=09=20*=20= access=20these=20relations=20and=20modify=20them.=0A+=09=20*/=0A+=09= InitialTempTableList=20=3D=20BuildRelationList(true,=20false);=0A+=0A+=09= /*=0A+=09=20*=20Enable=20vacuum=20cost=20delay,=20if=20any.=20=20While=20= this=20process=20isn't=20doing=20any=0A+=09=20*=20vacuuming,=20we=20are=20= re-using=20the=20infrastructure=20that=20vacuum=20cost=20delay=0A+=09=20= *=20provides=20rather=20than=20inventing=20something=20bespoke.=20This=20= is=20an=20internal=0A+=09=20*=20implementation=20detail=20and=20care=20= should=20be=20taken=20to=20avoid=20it=20bleeding=0A+=09=20*=20through=20= to=20the=20user=20to=20avoid=20confusion.=0A+=09=20*/=0A+=09= Assert(DataChecksumState->operation=20=3D=3D=20ENABLE_DATACHECKSUMS);=0A= +=09VacuumCostDelay=20=3D=20DataChecksumState->cost_delay;=0A+=09= VacuumCostLimit=20=3D=20DataChecksumState->cost_limit;=0A+=09= VacuumCostActive=20=3D=20(VacuumCostDelay=20>=200);=0A+=09= VacuumCostBalance=20=3D=200;=0A+=09VacuumCostPageHit=20=3D=200;=0A+=09= VacuumCostPageMiss=20=3D=200;=0A+=09VacuumCostPageDirty=20=3D=200;=0A+=0A= +=09/*=0A+=09=20*=20Create=20and=20set=20the=20vacuum=20strategy=20as=20= our=20buffer=20strategy.=0A+=09=20*/=0A+=09strategy=20=3D=20= GetAccessStrategy(BAS_VACUUM);=0A+=0A+=09RelationList=20=3D=20= BuildRelationList(false,=0A+=09=09=09=09=09=09=09=09=09=20= DataChecksumState->process_shared_catalogs);=0A+=0A+=09/*=20Update=20the=20= total=20number=20of=20relations=20to=20be=20processed=20in=20this=20DB.=20= */=0A+=09{=0A+=09=09const=20int=09index[]=20=3D=20{=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_DONE=0A+=09=09};=0A+=0A+=09=09int64=09=09= vals[2];=0A+=0A+=09=09vals[0]=20=3D=20list_length(RelationList);=0A+=09=09= vals[1]=20=3D=200;=0A+=0A+=09=09pgstat_progress_update_multi_param(2,=20= index,=20vals);=0A+=09}=0A+=0A+=09/*=20Process=20the=20relations=20*/=0A= +=09rels_done=20=3D=200;=0A+=09foreach_oid(reloid,=20RelationList)=0A+=09= {=0A+=09=09CHECK_FOR_INTERRUPTS();=0A+=0A+=09=09if=20= (!ProcessSingleRelationByOid(reloid,=20strategy))=0A+=09=09{=0A+=09=09=09= aborted=20=3D=20true;=0A+=09=09=09break;=0A+=09=09}=0A+=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_RELS_DONE,=0A+=09=09=09= =09=09=09=09=09=09=20++rels_done);=0A+=09}=0A+=09= list_free(RelationList);=0A+=0A+=09if=20(aborted)=0A+=09{=0A+=09=09= DataChecksumState->success=20=3D=20DATACHECKSUMSWORKER_ABORTED;=0A+=09=09= ereport(DEBUG1,=0A+=09=09=09=09errmsg("data=20checksum=20processing=20= aborted=20in=20database=20OID=20%u",=0A+=09=09=09=09=09=20=20=20dboid));=0A= +=09=09return;=0A+=09}=0A+=0A+=09/*=20The=20worker=20is=20about=20to=20= wait=20for=20temporary=20tables=20to=20go=20away.=20*/=0A+=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_WAITING_TEMPREL);=0A+=0A+=09= /*=0A+=09=20*=20Wait=20for=20all=20temp=20tables=20that=20existed=20when=20= we=20started=20to=20go=20away.=20This=0A+=09=20*=20is=20necessary=20= since=20we=20cannot=20"reach"=20them=20to=20enable=20checksums.=20Any=20= temp=0A+=09=20*=20tables=20created=20after=20we=20started=20will=20= already=20have=20checksums=20in=20them=0A+=09=20*=20(due=20to=20the=20= "inprogress-on"=20state),=20so=20no=20need=20to=20wait=20for=20those.=0A= +=09=20*/=0A+=09for=20(;;)=0A+=09{=0A+=09=09List=09=20=20=20= *CurrentTempTables;=0A+=09=09int=09=09=09numleft;=0A+=09=09char=09=09= activity[64];=0A+=0A+=09=09CurrentTempTables=20=3D=20= BuildRelationList(true,=20false);=0A+=09=09numleft=20=3D=200;=0A+=09=09= foreach_oid(tmptbloid,=20InitialTempTableList)=0A+=09=09{=0A+=09=09=09if=20= (list_member_oid(CurrentTempTables,=20tmptbloid))=0A+=09=09=09=09= numleft++;=0A+=09=09}=0A+=09=09list_free(CurrentTempTables);=0A+=0A+=09=09= INJECTION_POINT("datachecksumsworker-fake-temptable-wait",=20&numleft);=0A= +=0A+=09=09if=20(numleft=20=3D=3D=200)=0A+=09=09=09break;=0A+=0A+=09=09= /*=0A+=09=09=20*=20At=20least=20one=20temp=20table=20is=20left=20to=20= wait=20for,=20indicate=20in=20pgstat=0A+=09=09=20*=20activity=20and=20= progress=20reporting.=0A+=09=09=20*/=0A+=09=09snprintf(activity,=0A+=09=09= =09=09=20sizeof(activity),=0A+=09=09=09=09=20"Waiting=20for=20%d=20temp=20= tables=20to=20be=20removed",=20numleft);=0A+=09=09= pgstat_report_activity(STATE_RUNNING,=20activity);=0A+=0A+=09=09/*=20= Retry=20every=203=20seconds=20*/=0A+=09=09ResetLatch(MyLatch);=0A+=09=09= (void)=20WaitLatch(MyLatch,=0A+=09=09=09=09=09=09=20WL_LATCH_SET=20|=20= WL_TIMEOUT=20|=20WL_EXIT_ON_PM_DEATH,=0A+=09=09=09=09=09=09=203000,=0A+=09= =09=09=09=09=09=20WAIT_EVENT_CHECKSUM_ENABLE_TEMPTABLE_WAIT);=0A+=0A+=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09aborted=20= =3D=20DataChecksumState->launch_operation=20!=3D=20operation;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09=09= CHECK_FOR_INTERRUPTS();=0A+=0A+=09=09if=20(aborted=20||=20= abort_requested)=0A+=09=09{=0A+=09=09=09DataChecksumState->success=20=3D=20= DATACHECKSUMSWORKER_ABORTED;=0A+=09=09=09ereport(LOG,=0A+=09=09=09=09=09= errmsg("data=20checksum=20processing=20aborted=20in=20database=20OID=20= %u",=0A+=09=09=09=09=09=09=20=20=20dboid));=0A+=09=09=09return;=0A+=09=09= }=0A+=09}=0A+=0A+=09list_free(InitialTempTableList);=0A+=0A+=09/*=20= worker=20done=20*/=0A+=09pgstat_progress_end_command();=0A+=0A+=09= DataChecksumState->success=20=3D=20DATACHECKSUMSWORKER_SUCCESSFUL;=0A+}=0A= diff=20--git=20a/src/backend/postmaster/meson.build=20= b/src/backend/postmaster/meson.build=0Aindex=20e1f70726604..6cba23bbeef=20= 100644=0A---=20a/src/backend/postmaster/meson.build=0A+++=20= b/src/backend/postmaster/meson.build=0A@@=20-6,6=20+6,7=20@@=20= backend_sources=20+=3D=20files(=0A=20=20=20'bgworker.c',=0A=20=20=20= 'bgwriter.c',=0A=20=20=20'checkpointer.c',=0A+=20=20= 'datachecksum_state.c',=0A=20=20=20'fork_process.c',=0A=20=20=20= 'interrupt.c',=0A=20=20=20'launch_backend.c',=0Adiff=20--git=20= a/src/backend/postmaster/postmaster.c=20= b/src/backend/postmaster/postmaster.c=0Aindex=20abf0c97569e..eb4f3eb72d4=20= 100644=0A---=20a/src/backend/postmaster/postmaster.c=0A+++=20= b/src/backend/postmaster/postmaster.c=0A@@=20-2991,6=20+2991,11=20@@=20= PostmasterStateMachine(void)=0A=20=09=09=09=09=09=09=09=09=09B_INVALID,=0A= =20=09=09=09=09=09=09=09=09=09B_STANDALONE_BACKEND);=0A=20=0A+=09=09=09= /*=20also=20add=20data=20checksums=20processes=20*/=0A+=09=09=09= remainMask=20=3D=20btmask_add(remainMask,=0A+=09=09=09=09=09=09=09=09=09= B_DATACHECKSUMSWORKER_LAUNCHER,=0A+=09=09=09=09=09=09=09=09=09= B_DATACHECKSUMSWORKER_WORKER);=0A+=0A=20=09=09=09/*=20All=20types=20= should=20be=20included=20in=20targetMask=20or=20remainMask=20*/=0A=20=09=09= =09Assert((remainMask.mask=20|=20targetMask.mask)=20=3D=3D=20= BTYPE_MASK_ALL.mask);=0A=20=09=09}=0Adiff=20--git=20= a/src/backend/replication/logical/decode.c=20= b/src/backend/replication/logical/decode.c=0Aindex=20= 3c027bcb2f7..57aaef57c61=20100644=0A---=20= a/src/backend/replication/logical/decode.c=0A+++=20= b/src/backend/replication/logical/decode.c=0A@@=20-189,6=20+189,22=20@@=20= xlog_decode(LogicalDecodingContext=20*ctx,=20XLogRecordBuffer=20*buf)=0A=20= =09}=0A=20}=0A=20=0A+void=0A+xlog2_decode(LogicalDecodingContext=20*ctx,=20= XLogRecordBuffer=20*buf)=0A+{=0A+=09uint8=09=09info=20=3D=20= XLogRecGetInfo(buf->record)=20&=20~XLR_INFO_MASK;=0A+=0A+=09= ReorderBufferProcessXid(ctx->reorder,=20XLogRecGetXid(buf->record),=20= buf->origptr);=0A+=0A+=09switch=20(info)=0A+=09{=0A+=09=09case=20= XLOG2_CHECKSUMS:=0A+=09=09=09break;=0A+=09=09default:=0A+=09=09=09= elog(ERROR,=20"unexpected=20RM_XLOG2_ID=20record=20type:=20%u",=20info);=0A= +=09}=0A+}=0A+=0A=20/*=0A=20=20*=20Handle=20rmgr=20XACT_ID=20records=20= for=20LogicalDecodingProcessRecord().=0A=20=20*/=0Adiff=20--git=20= a/src/backend/storage/buffer/bufmgr.c=20= b/src/backend/storage/buffer/bufmgr.c=0Aindex=205c64570020d..3cc0b0bdd92=20= 100644=0A---=20a/src/backend/storage/buffer/bufmgr.c=0A+++=20= b/src/backend/storage/buffer/bufmgr.c=0A@@=20-8567,6=20+8567,13=20@@=20= buffer_readv_complete_one(PgAioTargetData=20*td,=20uint8=20buf_off,=20= Buffer=20buffer,=0A=20=09if=20(flags=20&=20= READ_BUFFERS_IGNORE_CHECKSUM_FAILURES)=0A=20=09=09piv_flags=20|=3D=20= PIV_IGNORE_CHECKSUM_FAILURE;=0A=20=0A+=09/*=0A+=09=20*=20If=20the=20= buffers=20are=20marked=20for=20zero=20on=20error,=20we=20want=20to=20log=20= that=20in=0A+=09=20*=20case=20of=20a=20checksum=20failure.=0A+=09=20*/=0A= +=09if=20(flags=20&=20READ_BUFFERS_ZERO_ON_ERROR)=0A+=09=09piv_flags=20= |=3D=20PIV_ZERO_BUFFERS_ON_ERROR;=0A+=0A=20=09/*=20Check=20for=20garbage=20= data.=20*/=0A=20=09if=20(!failed)=0A=20=09{=0Adiff=20--git=20= a/src/backend/storage/ipc/ipci.c=20b/src/backend/storage/ipc/ipci.c=0A= index=20d692d419846..7aab5da3386=20100644=0A---=20= a/src/backend/storage/ipc/ipci.c=0A+++=20= b/src/backend/storage/ipc/ipci.c=0A@@=20-31,6=20+31,7=20@@=0A=20#include=20= "postmaster/autovacuum.h"=0A=20#include=20= "postmaster/bgworker_internals.h"=0A=20#include=20= "postmaster/bgwriter.h"=0A+#include=20"postmaster/datachecksum_state.h"=0A= =20#include=20"postmaster/walsummarizer.h"=0A=20#include=20= "replication/logicallauncher.h"=0A=20#include=20"replication/origin.h"=0A= @@=20-142,6=20+143,7=20@@=20CalculateShmemSize(void)=0A=20=09size=20=3D=20= add_size(size,=20AioShmemSize());=0A=20=09size=20=3D=20add_size(size,=20= WaitLSNShmemSize());=0A=20=09size=20=3D=20add_size(size,=20= LogicalDecodingCtlShmemSize());=0A+=09size=20=3D=20add_size(size,=20= DataChecksumsShmemSize());=0A=20=0A=20=09/*=20include=20additional=20= requested=20shmem=20from=20preload=20libraries=20*/=0A=20=09size=20=3D=20= add_size(size,=20total_addin_request);=0A@@=20-310,6=20+312,7=20@@=20= CreateOrAttachShmemStructs(void)=0A=20=09PgArchShmemInit();=0A=20=09= ApplyLauncherShmemInit();=0A=20=09SlotSyncShmemInit();=0A+=09= DataChecksumsShmemInit();=0A=20=0A=20=09/*=0A=20=09=20*=20Set=20up=20= other=20modules=20that=20need=20some=20shared=20memory=20space=0Adiff=20= --git=20a/src/backend/storage/ipc/procsignal.c=20= b/src/backend/storage/ipc/procsignal.c=0Aindex=20= 7e017c8d53b..f1ab3aa3fe0=20100644=0A---=20= a/src/backend/storage/ipc/procsignal.c=0A+++=20= b/src/backend/storage/ipc/procsignal.c=0A@@=20-22,6=20+22,7=20@@=0A=20= #include=20"miscadmin.h"=0A=20#include=20"pgstat.h"=0A=20#include=20= "port/pg_bitutils.h"=0A+#include=20"postmaster/datachecksum_state.h"=0A=20= #include=20"replication/logicalctl.h"=0A=20#include=20= "replication/logicalworker.h"=0A=20#include=20"replication/walsender.h"=0A= @@=20-582,6=20+583,13=20@@=20ProcessProcSignalBarrier(void)=0A=20=09=09=09= =09=09case=20PROCSIGNAL_BARRIER_UPDATE_XLOG_LOGICAL_INFO:=0A=20=09=09=09=09= =09=09processed=20=3D=20ProcessBarrierUpdateXLogLogicalInfo();=0A=20=09=09= =09=09=09=09break;=0A+=0A+=09=09=09=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON:=0A+=09=09=09=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_ON:=0A+=09=09=09=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF:=0A+=09=09=09=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_OFF:=0A+=09=09=09=09=09=09processed=20=3D=20= AbsorbDataChecksumsBarrier(type);=0A+=09=09=09=09=09=09break;=0A=20=09=09= =09=09}=0A=20=0A=20=09=09=09=09/*=0Adiff=20--git=20= a/src/backend/storage/page/README=20b/src/backend/storage/page/README=0A= index=20e30d7ac59ad..73c36a63908=20100644=0A---=20= a/src/backend/storage/page/README=0A+++=20= b/src/backend/storage/page/README=0A@@=20-10,7=20+10,9=20@@=20= http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf,=20discussed=0A= =202010/12/22=20on=20-hackers=20list.=0A=20=0A=20Current=20= implementation=20requires=20this=20be=20enabled=20system-wide=20at=20= initdb=20time,=20or=0A-by=20using=20the=20pg_checksums=20tool=20on=20an=20= offline=20cluster.=0A+by=20using=20the=20pg_checksums=20tool=20on=20an=20= offline=20cluster.=20=20Checksums=20can=20also=20be=0A+enabled=20at=20= runtime=20using=20pg_enable_data_checksums(),=20and=20disabled=20by=20= using=0A+pg_disable_data_checksums().=0A=20=0A=20The=20checksum=20is=20= not=20valid=20at=20all=20times=20on=20a=20data=20page!!=0A=20The=20= checksum=20is=20valid=20when=20the=20page=20leaves=20the=20shared=20pool=20= and=20is=20checked=0Adiff=20--git=20a/src/backend/storage/page/bufpage.c=20= b/src/backend/storage/page/bufpage.c=0Aindex=2056f1f7ae9fc..1fdfda59edd=20= 100644=0A---=20a/src/backend/storage/page/bufpage.c=0A+++=20= b/src/backend/storage/page/bufpage.c=0A@@=20-107,7=20+107,15=20@@=20= PageIsVerified(PageData=20*page,=20BlockNumber=20blkno,=20int=20flags,=20= bool=20*checksum_fail=0A=20=09=20*/=0A=20=09if=20(!PageIsNew(page))=0A=20= =09{=0A-=09=09if=20(DataChecksumsEnabled())=0A+=09=09/*=0A+=09=09=20*=20= There=20shouldn't=20be=20any=20check=20for=20interrupt=20calls=20= happening=20in=20this=0A+=09=09=20*=20codepath,=20but=20just=20to=20be=20= on=20the=20safe=20side=20we=20hold=20interrupts=20since=0A+=09=09=20*=20= if=20they=20did=20happen=20the=20data=20checksum=20state=20could=20= change=20during=0A+=09=09=20*=20verifying=20checksums,=20which=20could=20= lead=20to=20incorrect=20verification=0A+=09=09=20*=20results.=0A+=09=09=20= */=0A+=09=09HOLD_INTERRUPTS();=0A+=09=09if=20(DataChecksumsNeedVerify())=0A= =20=09=09{=0A=20=09=09=09checksum=20=3D=20pg_checksum_page(page,=20= blkno);=0A=20=0A@@=20-118,6=20+126,7=20@@=20PageIsVerified(PageData=20= *page,=20BlockNumber=20blkno,=20int=20flags,=20bool=20*checksum_fail=0A=20= =09=09=09=09=09*checksum_failure_p=20=3D=20true;=0A=20=09=09=09}=0A=20=09= =09}=0A+=09=09RESUME_INTERRUPTS();=0A=20=0A=20=09=09/*=0A=20=09=09=20*=20= The=20following=20checks=20don't=20prove=20the=20header=20is=20correct,=20= only=20that=0A@@=20-151,8=20+160,9=20@@=20PageIsVerified(PageData=20= *page,=20BlockNumber=20blkno,=20int=20flags,=20bool=20*checksum_fail=0A=20= =09=09if=20((flags=20&=20(PIV_LOG_WARNING=20|=20PIV_LOG_LOG))=20!=3D=20= 0)=0A=20=09=09=09ereport(flags=20&=20PIV_LOG_WARNING=20?=20WARNING=20:=20= LOG,=0A=20=09=09=09=09=09(errcode(ERRCODE_DATA_CORRUPTED),=0A-=09=09=09=09= =09=20errmsg("page=20verification=20failed,=20calculated=20checksum=20%u=20= but=20expected=20%u",=0A-=09=09=09=09=09=09=09checksum,=20= p->pd_checksum)));=0A+=09=09=09=09=09=20errmsg("page=20verification=20= failed,=20calculated=20checksum=20%u=20but=20expected=20%u%s",=0A+=09=09=09= =09=09=09=09checksum,=20p->pd_checksum,=0A+=09=09=09=09=09=09=09(flags=20= &=20PIV_ZERO_BUFFERS_ON_ERROR=20?=20",=20buffer=20will=20be=20zeroed"=20= :=20""))));=0A=20=0A=20=09=09if=20(header_sane=20&&=20(flags=20&=20= PIV_IGNORE_CHECKSUM_FAILURE))=0A=20=09=09=09return=20true;=0A@@=20= -1507,9=20+1517,14=20@@=20PageIndexTupleOverwrite(Page=20page,=20= OffsetNumber=20offnum,=0A=20void=0A=20PageSetChecksum(Page=20page,=20= BlockNumber=20blkno)=0A=20{=0A+=09HOLD_INTERRUPTS();=0A=20=09/*=20If=20= we=20don't=20need=20a=20checksum,=20just=20return=20*/=0A-=09if=20= (PageIsNew(page)=20||=20!DataChecksumsEnabled())=0A+=09if=20= (PageIsNew(page)=20||=20!DataChecksumsNeedWrite())=0A+=09{=0A+=09=09= RESUME_INTERRUPTS();=0A=20=09=09return;=0A+=09}=0A=20=0A=20=09= ((PageHeader)=20page)->pd_checksum=20=3D=20pg_checksum_page(page,=20= blkno);=0A+=09RESUME_INTERRUPTS();=0A=20}=0Adiff=20--git=20= a/src/backend/utils/activity/pgstat_backend.c=20= b/src/backend/utils/activity/pgstat_backend.c=0Aindex=20= 7727fed3bda..04fe13e64c6=20100644=0A---=20= a/src/backend/utils/activity/pgstat_backend.c=0A+++=20= b/src/backend/utils/activity/pgstat_backend.c=0A@@=20-380,6=20+380,8=20= @@=20pgstat_tracks_backend_bktype(BackendType=20bktype)=0A=20=09=09case=20= B_CHECKPOINTER:=0A=20=09=09case=20B_IO_WORKER:=0A=20=09=09case=20= B_STARTUP:=0A+=09=09case=20B_DATACHECKSUMSWORKER_LAUNCHER:=0A+=09=09case=20= B_DATACHECKSUMSWORKER_WORKER:=0A=20=09=09=09return=20false;=0A=20=0A=20=09= =09case=20B_AUTOVAC_WORKER:=0Adiff=20--git=20= a/src/backend/utils/activity/pgstat_io.c=20= b/src/backend/utils/activity/pgstat_io.c=0Aindex=20= 28de24538dc..2be26e92283=20100644=0A---=20= a/src/backend/utils/activity/pgstat_io.c=0A+++=20= b/src/backend/utils/activity/pgstat_io.c=0A@@=20-362,6=20+362,8=20@@=20= pgstat_tracks_io_bktype(BackendType=20bktype)=0A=20=09=09case=20= B_LOGGER:=0A=20=09=09=09return=20false;=0A=20=0A+=09=09case=20= B_DATACHECKSUMSWORKER_LAUNCHER:=0A+=09=09case=20= B_DATACHECKSUMSWORKER_WORKER:=0A=20=09=09case=20B_AUTOVAC_LAUNCHER:=0A=20= =09=09case=20B_AUTOVAC_WORKER:=0A=20=09=09case=20B_BACKEND:=0Adiff=20= --git=20a/src/backend/utils/activity/wait_event_names.txt=20= b/src/backend/utils/activity/wait_event_names.txt=0Aindex=20= 6be80d2daad..0a6d16f8154=20100644=0A---=20= a/src/backend/utils/activity/wait_event_names.txt=0A+++=20= b/src/backend/utils/activity/wait_event_names.txt=0A@@=20-119,6=20+119,8=20= @@=20CHECKPOINT_DELAY_COMPLETE=09"Waiting=20for=20a=20backend=20that=20= blocks=20a=20checkpoint=20from=20c=0A=20CHECKPOINT_DELAY_START=09= "Waiting=20for=20a=20backend=20that=20blocks=20a=20checkpoint=20from=20= starting."=0A=20CHECKPOINT_DONE=09"Waiting=20for=20a=20checkpoint=20to=20= complete."=0A=20CHECKPOINT_START=09"Waiting=20for=20a=20checkpoint=20to=20= start."=0A+CHECKSUM_ENABLE_STARTCONDITION=09"Waiting=20for=20data=20= checksums=20enabling=20to=20start."=0A+CHECKSUM_ENABLE_TEMPTABLE_WAIT=09= "Waiting=20for=20temporary=20tables=20to=20be=20dropped=20for=20data=20= checksums=20to=20be=20enabled."=0A=20EXECUTE_GATHER=09"Waiting=20for=20= activity=20from=20a=20child=20process=20while=20executing=20a=20= Gather=20plan=20node."=0A=20HASH_BATCH_ALLOCATE=09= "Waiting=20for=20an=20elected=20Parallel=20Hash=20participant=20to=20= allocate=20a=20hash=20table."=0A=20HASH_BATCH_ELECT=09"Waiting=20to=20= elect=20a=20Parallel=20Hash=20participant=20to=20allocate=20a=20hash=20= table."=0A@@=20-365,6=20+367,7=20@@=20SerialControl=09"Waiting=20to=20= read=20or=20update=20shared=20pg_serial=20s=0A=20= AioWorkerSubmissionQueue=09"Waiting=20to=20access=20AIO=20worker=20= submission=20queue."=0A=20WaitLSN=09"Waiting=20to=20read=20or=20update=20= shared=20Wait-for-LSN=20state."=0A=20LogicalDecodingControl=09"Waiting=20= to=20read=20or=20update=20logical=20decoding=20status=20information."=0A= +DataChecksumsWorker=09"Waiting=20for=20data=20checksums=20worker."=0A=20= =0A=20#=0A=20#=20END=20OF=20PREDEFINED=20LWLOCKS=20(DO=20NOT=20CHANGE=20= THIS=20LINE)=0Adiff=20--git=20a/src/backend/utils/adt/pgstatfuncs.c=20= b/src/backend/utils/adt/pgstatfuncs.c=0Aindex=209185a8e6b83..1408de387ea=20= 100644=0A---=20a/src/backend/utils/adt/pgstatfuncs.c=0A+++=20= b/src/backend/utils/adt/pgstatfuncs.c=0A@@=20-297,6=20+297,8=20@@=20= pg_stat_get_progress_info(PG_FUNCTION_ARGS)=0A=20=09=09cmdtype=20=3D=20= PROGRESS_COMMAND_BASEBACKUP;=0A=20=09else=20if=20(pg_strcasecmp(cmd,=20= "COPY")=20=3D=3D=200)=0A=20=09=09cmdtype=20=3D=20PROGRESS_COMMAND_COPY;=0A= +=09else=20if=20(pg_strcasecmp(cmd,=20"DATACHECKSUMS")=20=3D=3D=200)=0A+=09= =09cmdtype=20=3D=20PROGRESS_COMMAND_DATACHECKSUMS;=0A=20=09else=0A=20=09=09= ereport(ERROR,=0A=20=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A@@=20-1182,9=20+1184,6=20@@=20= pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)=0A=20=09int64=09=09= result;=0A=20=09PgStat_StatDBEntry=20*dbentry;=0A=20=0A-=09if=20= (!DataChecksumsEnabled())=0A-=09=09PG_RETURN_NULL();=0A-=0A=20=09if=20= ((dbentry=20=3D=20pgstat_fetch_stat_dbentry(dbid))=20=3D=3D=20NULL)=0A=20= =09=09result=20=3D=200;=0A=20=09else=0A@@=20-1200,9=20+1199,6=20@@=20= pg_stat_get_db_checksum_last_failure(PG_FUNCTION_ARGS)=0A=20=09= TimestampTz=20result;=0A=20=09PgStat_StatDBEntry=20*dbentry;=0A=20=0A-=09= if=20(!DataChecksumsEnabled())=0A-=09=09PG_RETURN_NULL();=0A-=0A=20=09if=20= ((dbentry=20=3D=20pgstat_fetch_stat_dbentry(dbid))=20=3D=3D=20NULL)=0A=20= =09=09result=20=3D=200;=0A=20=09else=0Adiff=20--git=20= a/src/backend/utils/init/miscinit.c=20= b/src/backend/utils/init/miscinit.c=0Aindex=20ba191977697..7ffc808073a=20= 100644=0A---=20a/src/backend/utils/init/miscinit.c=0A+++=20= b/src/backend/utils/init/miscinit.c=0A@@=20-845,7=20+845,8=20@@=20= InitializeSessionUserIdStandalone(void)=0A=20=09=20*=20workers,=20in=20= slot=20sync=20worker=20and=20in=20background=20workers.=0A=20=09=20*/=0A=20= =09Assert(!IsUnderPostmaster=20||=20AmAutoVacuumWorkerProcess()=20||=0A-=09= =09=20=20=20AmLogicalSlotSyncWorkerProcess()=20||=20= AmBackgroundWorkerProcess());=0A+=09=09=20=20=20= AmLogicalSlotSyncWorkerProcess()=20||=20AmBackgroundWorkerProcess()=20||=0A= +=09=09=20=20=20AmDataChecksumsWorkerProcess());=0A=20=0A=20=09/*=20call=20= only=20once=20*/=0A=20=09Assert(!OidIsValid(AuthenticatedUserId));=0A= diff=20--git=20a/src/backend/utils/init/postinit.c=20= b/src/backend/utils/init/postinit.c=0Aindex=20577ef5effbb..ecbe2d4b097=20= 100644=0A---=20a/src/backend/utils/init/postinit.c=0A+++=20= b/src/backend/utils/init/postinit.c=0A@@=20-758,6=20+758,24=20@@=20= InitPostgres(const=20char=20*in_dbname,=20Oid=20dboid,=0A=20=0A=20=09= ProcSignalInit(MyCancelKey,=20MyCancelKeyLength);=0A=20=0A+=09/*=0A+=09=20= *=20Initialize=20a=20local=20cache=20of=20the=20data_checksum_version,=20= to=20be=20updated=20by=0A+=09=20*=20the=20procsignal-based=20barriers.=0A= +=09=20*=0A+=09=20*=20This=20intentionally=20happens=20after=20= initializing=20the=20procsignal,=20otherwise=0A+=09=20*=20we=20might=20= miss=20a=20state=20change.=20This=20means=20we=20can=20get=20a=20barrier=20= for=20the=0A+=09=20*=20state=20we've=20just=20initialized.=0A+=09=20*=0A= +=09=20*=20The=20postmaster=20(which=20is=20what=20gets=20forked=20into=20= the=20new=20child=20process)=0A+=09=20*=20does=20not=20handle=20= barriers,=20therefore=20it=20may=20not=20have=20the=20current=20value=0A= +=09=20*=20of=20LocalDataChecksumVersion=20value=20(it'll=20have=20the=20= value=20read=20from=20the=0A+=09=20*=20control=20file,=20which=20may=20= be=20arbitrarily=20old).=0A+=09=20*=0A+=09=20*=20NB:=20Even=20if=20the=20= postmaster=20handled=20barriers,=20the=20value=20might=20still=20be=0A+=09= =20*=20stale,=20as=20it=20might=20have=20changed=20after=20this=20= process=20forked.=0A+=09=20*/=0A+=09InitLocalDataChecksumState();=0A+=0A=20= =09/*=0A=20=09=20*=20Also=20set=20up=20timeout=20handlers=20needed=20for=20= backend=20operation.=20=20We=20need=0A=20=09=20*=20these=20in=20every=20= case=20except=20bootstrap.=0A@@=20-886,7=20+904,7=20@@=20= InitPostgres(const=20char=20*in_dbname,=20Oid=20dboid,=0A=20=09=09=09=09=09= =20errhint("You=20should=20immediately=20run=20CREATE=20USER=20\"%s\"=20= SUPERUSER;.",=0A=20=09=09=09=09=09=09=09=20username=20!=3D=20NULL=20?=20= username=20:=20"postgres")));=0A=20=09}=0A-=09else=20if=20= (AmBackgroundWorkerProcess())=0A+=09else=20if=20= (AmBackgroundWorkerProcess()=20||=20AmDataChecksumsWorkerProcess())=0A=20= =09{=0A=20=09=09if=20(username=20=3D=3D=20NULL=20&&=20= !OidIsValid(useroid))=0A=20=09=09{=0Adiff=20--git=20= a/src/backend/utils/misc/guc_parameters.dat=20= b/src/backend/utils/misc/guc_parameters.dat=0Aindex=20= e556b8844d8..eb7e3dfa460=20100644=0A---=20= a/src/backend/utils/misc/guc_parameters.dat=0A+++=20= b/src/backend/utils/misc/guc_parameters.dat=0A@@=20-571,11=20+571,12=20= @@=0A=20=20=20max=20=3D>=20'1.0',=0A=20},=0A=20=0A-{=20name=20=3D>=20= 'data_checksums',=20type=20=3D>=20'bool',=20context=20=3D>=20= 'PGC_INTERNAL',=20group=20=3D>=20'PRESET_OPTIONS',=0A+{=20name=20=3D>=20= 'data_checksums',=20type=20=3D>=20'enum',=20context=20=3D>=20= 'PGC_INTERNAL',=20group=20=3D>=20'PRESET_OPTIONS',=0A=20=20=20short_desc=20= =3D>=20'Shows=20whether=20data=20checksums=20are=20turned=20on=20for=20= this=20cluster.',=0A=20=20=20flags=20=3D>=20'GUC_NOT_IN_SAMPLE=20|=20= GUC_DISALLOW_IN_FILE=20|=20GUC_RUNTIME_COMPUTED',=0A=20=20=20variable=20= =3D>=20'data_checksums',=0A-=20=20boot_val=20=3D>=20'false',=0A+=20=20= boot_val=20=3D>=20'PG_DATA_CHECKSUM_OFF',=0A+=20=20options=20=3D>=20= 'data_checksums_options',=0A=20},=0A=20=0A=20#=20Can't=20be=20set=20by=20= ALTER=20SYSTEM=20as=20it=20can=20lead=20to=20recursive=20definition=0A= diff=20--git=20a/src/backend/utils/misc/guc_tables.c=20= b/src/backend/utils/misc/guc_tables.c=0Aindex=201e14b7b4af0..d9ca13baff9=20= 100644=0A---=20a/src/backend/utils/misc/guc_tables.c=0A+++=20= b/src/backend/utils/misc/guc_tables.c=0A@@=20-501,6=20+501,14=20@@=20= static=20const=20struct=20config_enum_entry=20= file_extend_method_options[]=20=3D=20{=0A=20=09{NULL,=200,=20false}=0A=20= };=0A=20=0A+static=20const=20struct=20config_enum_entry=20= data_checksums_options[]=20=3D=20{=0A+=09{"on",=20= PG_DATA_CHECKSUM_VERSION,=20true},=0A+=09{"off",=20PG_DATA_CHECKSUM_OFF,=20= true},=0A+=09{"inprogress-on",=20PG_DATA_CHECKSUM_INPROGRESS_ON,=20= true},=0A+=09{"inprogress-off",=20PG_DATA_CHECKSUM_INPROGRESS_OFF,=20= true},=0A+=09{NULL,=200,=20false}=0A+};=0A+=0A=20/*=0A=20=20*=20Options=20= for=20enum=20values=20stored=20in=20other=20modules=0A=20=20*/=0A@@=20= -629,7=20+637,6=20@@=20static=20int=09shared_memory_size_in_huge_pages;=0A= =20static=20int=09wal_block_size;=0A=20static=20int=09num_os_semaphores;=0A= =20static=20int=09effective_wal_level=20=3D=20WAL_LEVEL_REPLICA;=0A= -static=20bool=20data_checksums;=0A=20static=20bool=20integer_datetimes;=0A= =20=0A=20#ifdef=20USE_ASSERT_CHECKING=0Adiff=20--git=20= a/src/backend/utils/misc/postgresql.conf.sample=20= b/src/backend/utils/misc/postgresql.conf.sample=0Aindex=20= 2c5e98d1d4d..59363fed1a2=20100644=0A---=20= a/src/backend/utils/misc/postgresql.conf.sample=0A+++=20= b/src/backend/utils/misc/postgresql.conf.sample=0A@@=20-543,11=20+543,11=20= @@=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#=20archiver=20=20=20= =20=20=20=20=20=20=20=20=20=20=20autovacuum=0A=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20#=20backend=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20bgworker=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#=20= bgwriter=20=20=20=20=20=20=20=20=20=20=20=20=20=20checkpointer=0A-=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20#=20ioworker=20=20=20=20=20=20=20=20=20= =20=20=20=20=20postmaster=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= #=20slotsyncworker=20=20=20=20=20=20=20=20startup=0A-=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20#=20syslogger=20=20=20=20=20=20=20=20=20=20=20=20= =20walreceiver=0A-=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#=20= walsummarizer=20=20=20=20=20=20=20=20=20walwriter=0A-=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20#=20walsender=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20#=20checksums=20=20=20=20=20=20=20=20=20=20=20=20=20ioworker=0A= +=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#=20postmaster=20=20=20=20= =20=20=20=20=20=20=20=20slotsyncworker=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20#=20startup=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= syslogger=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#=20= walreceiver=20=20=20=20=20=20=20=20=20=20=20walsummarizer=0A+=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20#=20walwriter=20=20=20=20=20=20=20=20=20= =20=20=20=20walsender=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= #=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20#=20Level=20values=20= in=20order=20of=20decreasing=0A=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20#=20detail:=0Adiff=20--git=20a/src/bin/pg_checksums/pg_checksums.c=20= b/src/bin/pg_checksums/pg_checksums.c=0Aindex=20301e256fbb1..2a38f1d688b=20= 100644=0A---=20a/src/bin/pg_checksums/pg_checksums.c=0A+++=20= b/src/bin/pg_checksums/pg_checksums.c=0A@@=20-585,7=20+585,7=20@@=20= main(int=20argc,=20char=20*argv[])=0A=20=09=09ControlFile->state=20!=3D=20= DB_SHUTDOWNED_IN_RECOVERY)=0A=20=09=09pg_fatal("cluster=20must=20be=20= shut=20down");=0A=20=0A-=09if=20(ControlFile->data_checksum_version=20=3D=3D= =200=20&&=0A+=09if=20(ControlFile->data_checksum_version=20!=3D=20= PG_DATA_CHECKSUM_VERSION=20&&=0A=20=09=09mode=20=3D=3D=20PG_MODE_CHECK)=0A= =20=09=09pg_fatal("data=20checksums=20are=20not=20enabled=20in=20= cluster");=0A=20=0A@@=20-593,7=20+593,7=20@@=20main(int=20argc,=20char=20= *argv[])=0A=20=09=09mode=20=3D=3D=20PG_MODE_DISABLE)=0A=20=09=09= pg_fatal("data=20checksums=20are=20already=20disabled=20in=20cluster");=0A= =20=0A-=09if=20(ControlFile->data_checksum_version=20>=200=20&&=0A+=09if=20= (ControlFile->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_VERSION=20= &&=0A=20=09=09mode=20=3D=3D=20PG_MODE_ENABLE)=0A=20=09=09pg_fatal("data=20= checksums=20are=20already=20enabled=20in=20cluster");=0A=20=0Adiff=20= --git=20a/src/bin/pg_controldata/pg_controldata.c=20= b/src/bin/pg_controldata/pg_controldata.c=0Aindex=20= a4060309ae0..fe5fc5ec133=20100644=0A---=20= a/src/bin/pg_controldata/pg_controldata.c=0A+++=20= b/src/bin/pg_controldata/pg_controldata.c=0A@@=20-287,6=20+287,8=20@@=20= main(int=20argc,=20char=20*argv[])=0A=20=09=09=20=20=20= ControlFile->checkPointCopy.oldestCommitTsXid);=0A=20=09printf(_("Latest=20= checkpoint's=20newestCommitTsXid:%u\n"),=0A=20=09=09=20=20=20= ControlFile->checkPointCopy.newestCommitTsXid);=0A+=09printf(_("Latest=20= checkpoint's=20data_checksum_version:%u\n"),=0A+=09=09=20=20=20= ControlFile->checkPointCopy.dataChecksumState);=0A=20=09printf(_("Time=20= of=20latest=20checkpoint:=20=20=20=20=20=20=20=20=20=20=20=20%s\n"),=0A=20= =09=09=20=20=20ckpttime_str);=0A=20=09printf(_("Fake=20LSN=20counter=20= for=20unlogged=20rels:=20=20=20%X/%08X\n"),=0Adiff=20--git=20= a/src/bin/pg_upgrade/controldata.c=20b/src/bin/pg_upgrade/controldata.c=0A= index=20aa6e8b4de5d..b7fd4b83757=20100644=0A---=20= a/src/bin/pg_upgrade/controldata.c=0A+++=20= b/src/bin/pg_upgrade/controldata.c=0A@@=20-15,6=20+15,7=20@@=0A=20= #include=20"access/xlog_internal.h"=0A=20#include=20"common/string.h"=0A=20= #include=20"pg_upgrade.h"=0A+#include=20"storage/bufpage.h"=0A=20=0A=20=0A= =20/*=0A@@=20-736,6=20+737,14=20@@=20check_control_data(ControlData=20= *oldctrl,=0A=20=09=20*=20check_for_isn_and_int8_passing_mismatch().=0A=20= =09=20*/=0A=20=0A+=09/*=0A+=09=20*=20If=20data=20checksums=20are=20in=20= any=20in-progress=20state=20then=20disallow=20the=0A+=09=20*=20upgrade.=20= The=20user=20should=20either=20let=20the=20process=20finish,=20or=20turn=20= off=0A+=09=20*=20data=20checksums,=20before=20retrying.=0A+=09=20*/=0A+=09= if=20(oldctrl->data_checksum_version=20>=20PG_DATA_CHECKSUM_VERSION)=0A+=09= =09pg_fatal("checksums=20are=20being=20enabled=20in=20the=20old=20= cluster");=0A+=0A=20=09/*=0A=20=09=20*=20We=20might=20eventually=20allow=20= upgrades=20from=20checksum=20to=20no-checksum=0A=20=09=20*=20clusters.=0A= diff=20--git=20a/src/bin/pg_waldump/t/001_basic.pl=20= b/src/bin/pg_waldump/t/001_basic.pl=0Aindex=20a268f0f1dd0..7dd1c3dd63e=20= 100644=0A---=20a/src/bin/pg_waldump/t/001_basic.pl=0A+++=20= b/src/bin/pg_waldump/t/001_basic.pl=0A@@=20-79,7=20+79,8=20@@=20BRIN=0A=20= CommitTs=0A=20ReplicationOrigin=0A=20Generic=0A-LogicalMessage$/,=0A= +LogicalMessage=0A+XLOG2$/,=0A=20=09'rmgr=20list');=0A=20=0A=20=0Adiff=20= --git=20a/src/include/access/rmgrlist.h=20= b/src/include/access/rmgrlist.h=0Aindex=203352b5f8532..ae32ef16d67=20= 100644=0A---=20a/src/include/access/rmgrlist.h=0A+++=20= b/src/include/access/rmgrlist.h=0A@@=20-47,3=20+47,4=20@@=20= PG_RMGR(RM_COMMIT_TS_ID,=20"CommitTs",=20commit_ts_redo,=20= commit_ts_desc,=20commit_ts_i=0A=20PG_RMGR(RM_REPLORIGIN_ID,=20= "ReplicationOrigin",=20replorigin_redo,=20replorigin_desc,=20= replorigin_identify,=20NULL,=20NULL,=20NULL,=20NULL)=0A=20= PG_RMGR(RM_GENERIC_ID,=20"Generic",=20generic_redo,=20generic_desc,=20= generic_identify,=20NULL,=20NULL,=20generic_mask,=20NULL)=0A=20= PG_RMGR(RM_LOGICALMSG_ID,=20"LogicalMessage",=20logicalmsg_redo,=20= logicalmsg_desc,=20logicalmsg_identify,=20NULL,=20NULL,=20NULL,=20= logicalmsg_decode)=0A+PG_RMGR(RM_XLOG2_ID,=20"XLOG2",=20xlog2_redo,=20= xlog2_desc,=20xlog2_identify,=20NULL,=20NULL,=20NULL,=20xlog2_decode)=0A= diff=20--git=20a/src/include/access/xlog.h=20b/src/include/access/xlog.h=0A= index=20dcc12eb8cbe..5fde1e847b2=20100644=0A---=20= a/src/include/access/xlog.h=0A+++=20b/src/include/access/xlog.h=0A@@=20= -57,6=20+57,7=20@@=20extern=20PGDLLIMPORT=20int=20CommitDelay;=0A=20= extern=20PGDLLIMPORT=20int=20CommitSiblings;=0A=20extern=20PGDLLIMPORT=20= bool=20track_wal_io_timing;=0A=20extern=20PGDLLIMPORT=20int=20= wal_decode_buffer_size;=0A+extern=20PGDLLIMPORT=20int=20data_checksums;=0A= =20=0A=20extern=20PGDLLIMPORT=20int=20CheckPointSegments;=0A=20=0A@@=20= -119,7=20+120,7=20@@=20extern=20PGDLLIMPORT=20bool=20XLogLogicalInfo;=0A=20= =20*=20of=20the=20bits=20make=20it=20to=20disk,=20but=20the=20checksum=20= wouldn't=20match.=20=20Also=20WAL-log=0A=20=20*=20them=20if=20forced=20= by=20wal_log_hints=3Don.=0A=20=20*/=0A-#define=20XLogHintBitIsNeeded()=20= (DataChecksumsEnabled()=20||=20wal_log_hints)=0A+#define=20= XLogHintBitIsNeeded()=20(wal_log_hints=20||=20DataChecksumsNeedWrite())=0A= =20=0A=20/*=20Do=20we=20need=20to=20WAL-log=20information=20required=20= only=20for=20Hot=20Standby=20and=20logical=20replication?=20*/=0A=20= #define=20XLogStandbyInfoActive()=20(wal_level=20>=3D=20= WAL_LEVEL_REPLICA)=0A@@=20-229,8=20+230,11=20@@=20extern=20void=20= XLogSetReplicationSlotMinimumLSN(XLogRecPtr=20lsn);=0A=20extern=20= XLogRecPtr=20XLogGetReplicationSlotMinimumLSN(void);=0A=20=0A=20extern=20= void=20xlog_redo(struct=20XLogReaderState=20*record);=0A+extern=20void=20= xlog2_redo(struct=20XLogReaderState=20*record);=0A=20extern=20void=20= xlog_desc(StringInfo=20buf,=20struct=20XLogReaderState=20*record);=0A= +extern=20void=20xlog2_desc(StringInfo=20buf,=20struct=20XLogReaderState=20= *record);=0A=20extern=20const=20char=20*xlog_identify(uint8=20info);=0A= +extern=20const=20char=20*xlog2_identify(uint8=20info);=0A=20=0A=20= extern=20void=20issue_xlog_fsync(int=20fd,=20XLogSegNo=20segno,=20= TimeLineID=20tli);=0A=20=0A@@=20-243,7=20+247,16=20@@=20extern=20= XLogRecPtr=20GetXLogWriteRecPtr(void);=0A=20=0A=20extern=20uint64=20= GetSystemIdentifier(void);=0A=20extern=20char=20= *GetMockAuthenticationNonce(void);=0A-extern=20bool=20= DataChecksumsEnabled(void);=0A+extern=20bool=20= DataChecksumsNeedWrite(void);=0A+extern=20bool=20= DataChecksumsNeedVerify(void);=0A+extern=20bool=20= DataChecksumsInProgress(void);=0A+extern=20void=20= SetDataChecksumsOnInProgress(void);=0A+extern=20void=20= SetDataChecksumsOn(void);=0A+extern=20void=20SetDataChecksumsOff(void);=0A= +extern=20const=20char=20*show_data_checksums(void);=0A+extern=20const=20= char=20*get_checksum_state_string(uint32=20state);=0A+extern=20void=20= InitLocalDataChecksumState(void);=0A+extern=20void=20= SetLocalDataChecksumState(uint32=20data_checksum_version);=0A=20extern=20= bool=20GetDefaultCharSignedness(void);=0A=20extern=20XLogRecPtr=20= GetFakeLSNForUnloggedRel(void);=0A=20extern=20Size=20= XLOGShmemSize(void);=0Adiff=20--git=20= a/src/include/access/xlog_internal.h=20= b/src/include/access/xlog_internal.h=0Aindex=20755835d63bf..10c18d39ff8=20= 100644=0A---=20a/src/include/access/xlog_internal.h=0A+++=20= b/src/include/access/xlog_internal.h=0A@@=20-25,6=20+25,7=20@@=0A=20= #include=20"lib/stringinfo.h"=0A=20#include=20"pgtime.h"=0A=20#include=20= "storage/block.h"=0A+#include=20"storage/checksum.h"=0A=20#include=20= "storage/relfilelocator.h"=0A=20=0A=20=0A@@=20-287,6=20+288,12=20@@=20= typedef=20struct=20xl_restore_point=0A=20=09char=09=09= rp_name[MAXFNAMELEN];=0A=20}=20xl_restore_point;=0A=20=0A+/*=20= Information=20logged=20when=20data=20checksum=20level=20is=20changed=20= */=0A+typedef=20struct=20xl_checksum_state=0A+{=0A+=09ChecksumStateType=20= new_checksum_state;=0A+}=20xl_checksum_state;=0A+=0A=20/*=20Overwrite=20= of=20prior=20contrecord=20*/=0A=20typedef=20struct=20= xl_overwrite_contrecord=0A=20{=0A@@=20-307,6=20+314,7=20@@=20typedef=20= struct=20xl_end_of_recovery=0A=20typedef=20struct=20xl_checkpoint_redo=0A= =20{=0A=20=09int=09=09=09wal_level;=0A+=09uint32=09=09= data_checksum_version;=0A=20}=20xl_checkpoint_redo;=0A=20=0A=20/*=0Adiff=20= --git=20a/src/include/catalog/pg_control.h=20= b/src/include/catalog/pg_control.h=0Aindex=2077a661e818b..80b3a730e03=20= 100644=0A---=20a/src/include/catalog/pg_control.h=0A+++=20= b/src/include/catalog/pg_control.h=0A@@=20-22,7=20+22,7=20@@=0A=20=0A=20=0A= =20/*=20Version=20identifier=20for=20this=20pg_control=20format=20*/=0A= -#define=20PG_CONTROL_VERSION=091901=0A+#define=20PG_CONTROL_VERSION=09= 1902=0A=20=0A=20/*=20Nonce=20key=20length,=20see=20below=20*/=0A=20= #define=20MOCK_AUTH_NONCE_LEN=09=0932=0A@@=20-63,6=20+63,9=20@@=20= typedef=20struct=20CheckPoint=0A=20=09=20*=20set=20to=20= InvalidTransactionId.=0A=20=09=20*/=0A=20=09TransactionId=20= oldestActiveXid;=0A+=0A+=09/*=20data=20checksums=20state=20at=20the=20= time=20of=20the=20checkpoint=20=20*/=0A+=09uint32=09=09= dataChecksumState;=0A=20}=20CheckPoint;=0A=20=0A=20/*=20XLOG=20info=20= values=20for=20XLOG=20rmgr=20*/=0A@@=20-83,6=20+86,9=20@@=20typedef=20= struct=20CheckPoint=0A=20#define=20XLOG_CHECKPOINT_REDO=09=09=090xE0=0A=20= #define=20XLOG_LOGICAL_DECODING_STATUS_CHANGE=090xF0=0A=20=0A+/*=20XLOG=20= info=20values=20for=20XLOG2=20rmgr=20*/=0A+#define=20XLOG2_CHECKSUMS=09=09= =09=09=090x00=0A+=0A=20=0A=20/*=0A=20=20*=20System=20status=20indicator.=20= =20Note=20this=20is=20stored=20in=20pg_control;=20if=20you=20change=0A= diff=20--git=20a/src/include/catalog/pg_proc.dat=20= b/src/include/catalog/pg_proc.dat=0Aindex=20acf16254b21..bd177aebfcb=20= 100644=0A---=20a/src/include/catalog/pg_proc.dat=0A+++=20= b/src/include/catalog/pg_proc.dat=0A@@=20-12558,6=20+12558,20=20@@=0A=20=20= =20proname=20=3D>=20'jsonb_subscript_handler',=20prorettype=20=3D>=20= 'internal',=0A=20=20=20proargtypes=20=3D>=20'internal',=20prosrc=20=3D>=20= 'jsonb_subscript_handler'=20},=0A=20=0A+#=20data=20checksum=20management=20= functions=0A+{=20oid=20=3D>=20'9258',=0A+=20=20descr=20=3D>=20'disable=20= data=20checksums',=0A+=20=20proname=20=3D>=20= 'pg_disable_data_checksums',=20provolatile=20=3D>=20'v',=20prorettype=20= =3D>=20'void',=0A+=20=20proparallel=20=3D>=20'r',=20prosrc=20=3D>=20= 'disable_data_checksums',=20proargtypes=20=3D>=20'',=0A+=20=20proacl=20= =3D>=20'{POSTGRES=3DX}'},=0A+{=20oid=20=3D>=20'9257',=0A+=20=20descr=20= =3D>=20'enable=20data=20checksums',=0A+=20=20proname=20=3D>=20= 'pg_enable_data_checksums',=20provolatile=20=3D>=20'v',=20prorettype=20= =3D>=20'void',=0A+=20=20proparallel=20=3D>=20'r',=20proargtypes=20=3D>=20= 'int4=20int4',=20proallargtypes=20=3D>=20'{int4,int4}',=0A+=20=20= proargmodes=20=3D>=20'{i,i}',=20proargnames=20=3D>=20= '{cost_delay,cost_limit}',=0A+=20=20proargdefaults=20=3D>=20'{0,100}',=20= prosrc=20=3D>=20'enable_data_checksums',=0A+=20=20proacl=20=3D>=20= '{POSTGRES=3DX}'},=0A+=0A=20#=20collation=20management=20functions=0A=20= {=20oid=20=3D>=20'3445',=20descr=20=3D>=20'import=20collations=20from=20= operating=20system',=0A=20=20=20proname=20=3D>=20= 'pg_import_system_collations',=20procost=20=3D>=20'100',=0Adiff=20--git=20= a/src/include/commands/progress.h=20b/src/include/commands/progress.h=0A= index=209c40772706c..67948667a97=20100644=0A---=20= a/src/include/commands/progress.h=0A+++=20= b/src/include/commands/progress.h=0A@@=20-185,4=20+185,20=20@@=0A=20= #define=20PROGRESS_COPY_TYPE_PIPE=203=0A=20#define=20= PROGRESS_COPY_TYPE_CALLBACK=204=0A=20=0A+/*=20Progress=20parameters=20= for=20PROGRESS_DATACHECKSUMS=20*/=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE=09=090=0A+#define=20= PROGRESS_DATACHECKSUMS_DBS_TOTAL=091=0A+#define=20= PROGRESS_DATACHECKSUMS_DBS_DONE=09=092=0A+#define=20= PROGRESS_DATACHECKSUMS_RELS_TOTAL=093=0A+#define=20= PROGRESS_DATACHECKSUMS_RELS_DONE=094=0A+#define=20= PROGRESS_DATACHECKSUMS_BLOCKS_TOTAL=095=0A+#define=20= PROGRESS_DATACHECKSUMS_BLOCKS_DONE=096=0A+=0A+/*=20Phases=20of=20= datachecksumsworker=20operation=20*/=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_ENABLING=09=09=090=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_DISABLING=09=09=091=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_WAITING_TEMPREL=092=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_WAITING_BARRIER=093=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_DONE=09=09=09=094=0A+=0A=20#endif=0Adiff=20= --git=20a/src/include/miscadmin.h=20b/src/include/miscadmin.h=0Aindex=20= 04f29748be7..7277c37e779=20100644=0A---=20a/src/include/miscadmin.h=0A= +++=20b/src/include/miscadmin.h=0A@@=20-367,6=20+367,9=20@@=20typedef=20= enum=20BackendType=0A=20=09B_WAL_SUMMARIZER,=0A=20=09B_WAL_WRITER,=0A=20=0A= +=09B_DATACHECKSUMSWORKER_LAUNCHER,=0A+=09B_DATACHECKSUMSWORKER_WORKER,=0A= +=0A=20=09/*=0A=20=09=20*=20Logger=20is=20not=20connected=20to=20shared=20= memory=20and=20does=20not=20have=20a=20PGPROC=0A=20=09=20*=20entry.=0A@@=20= -392,6=20+395,9=20@@=20extern=20PGDLLIMPORT=20BackendType=20= MyBackendType;=0A=20#define=20AmWalSummarizerProcess()=09(MyBackendType=20= =3D=3D=20B_WAL_SUMMARIZER)=0A=20#define=20AmWalWriterProcess()=09=09= (MyBackendType=20=3D=3D=20B_WAL_WRITER)=0A=20#define=20= AmIoWorkerProcess()=09=09=09(MyBackendType=20=3D=3D=20B_IO_WORKER)=0A= +#define=20AmDataChecksumsWorkerProcess()=20\=0A+=09(MyBackendType=20=3D=3D= =20B_DATACHECKSUMSWORKER_LAUNCHER=20||=20\=0A+=09=20MyBackendType=20=3D=3D= =20B_DATACHECKSUMSWORKER_WORKER)=0A=20=0A=20#define=20= AmSpecialWorkerProcess()=20\=0A=20=09(AmAutoVacuumLauncherProcess()=20||=20= \=0Adiff=20--git=20a/src/include/postmaster/datachecksum_state.h=20= b/src/include/postmaster/datachecksum_state.h=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..343494edcc8=0A---=20/dev/null=0A+++=20= b/src/include/postmaster/datachecksum_state.h=0A@@=20-0,0=20+1,58=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20datachecksum_state.h=0A+=20*=09=20=20header=20file=20= for=20data=20checksum=20helper=20background=20worker=20and=20data=0A+=20= *=09=20=20checksum=20state=20manipulation=0A+=20*=0A+=20*=0A+=20*=20= Portions=20Copyright=20(c)=201996-2026,=20PostgreSQL=20Global=20= Development=20Group=0A+=20*=20Portions=20Copyright=20(c)=201994,=20= Regents=20of=20the=20University=20of=20California=0A+=20*=0A+=20*=20= src/include/postmaster/datachecksum_state.h=0A+=20*=0A+=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#ifndef=20DATACHECKSUM_STATE_H=0A+#define=20= DATACHECKSUM_STATE_H=0A+=0A+#include=20"storage/procsignal.h"=0A+=0A+/*=20= Shared=20memory=20*/=0A+extern=20Size=20DataChecksumsShmemSize(void);=0A= +extern=20void=20DataChecksumsShmemInit(void);=0A+=0A+/*=20Possible=20= operations=20the=20Datachecksumsworker=20can=20perform=20*/=0A+typedef=20= enum=20DataChecksumsWorkerOperation=0A+{=0A+=09ENABLE_DATACHECKSUMS,=0A+=09= DISABLE_DATACHECKSUMS,=0A+}=20DataChecksumsWorkerOperation;=0A+=0A+/*=0A= +=20*=20Possible=20states=20for=20a=20database=20entry=20which=20has=20= been=20processed.=20Exported=0A+=20*=20here=20since=20we=20want=20to=20= be=20able=20to=20reference=20this=20from=20injection=20point=20tests.=0A= +=20*/=0A+typedef=20enum=0A+{=0A+=09DATACHECKSUMSWORKER_SUCCESSFUL=20=3D=20= 0,=0A+=09DATACHECKSUMSWORKER_ABORTED,=0A+=09DATACHECKSUMSWORKER_FAILED,=0A= +=09DATACHECKSUMSWORKER_DROPDB,=0A+}=20DataChecksumsWorkerResult;=0A+=0A= +/*=20Prototypes=20for=20data=20checksum=20state=20manipulation=20*/=0A= +bool=09=09AbsorbDataChecksumsBarrier(ProcSignalBarrierType=20= target_state);=0A+void=09=09EmitAndWaitDataChecksumsBarrier(uint32=20= state);=0A+=0A+/*=20Prototypes=20for=20data=20checksum=20background=20= worker=20*/=0A+=0A+/*=20Start=20the=20background=20processes=20for=20= enabling=20or=20disabling=20checksums=20*/=0A+void=09=09= StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A+=09= =09=09=09=09=09=09=09=09=09=09=20int=20cost_delay,=0A+=09=09=09=09=09=09=09= =09=09=09=09=20int=20cost_limit);=0A+=0A+/*=20Background=20worker=20= entrypoints=20*/=0A+void=09=09DataChecksumsWorkerLauncherMain(Datum=20= arg);=0A+void=09=09DataChecksumsWorkerMain(Datum=20arg);=0A+=0A+#endif=09= =09=09=09=09=09=09/*=20DATACHECKSUM_STATE_H=20*/=0Adiff=20--git=20= a/src/include/postmaster/proctypelist.h=20= b/src/include/postmaster/proctypelist.h=0Aindex=20= feac19ba207..b3477e6f17a=20100644=0A---=20= a/src/include/postmaster/proctypelist.h=0A+++=20= b/src/include/postmaster/proctypelist.h=0A@@=20-38,6=20+38,8=20@@=20= PG_PROCTYPE(B_BACKEND,=20"backend",=20gettext_noop("client=20backend"),=20= BackendMain,=20t=0A=20PG_PROCTYPE(B_BG_WORKER,=20"bgworker",=20= gettext_noop("background=20worker"),=20BackgroundWorkerMain,=20true)=0A=20= PG_PROCTYPE(B_BG_WRITER,=20"bgwriter",=20gettext_noop("background=20= writer"),=20BackgroundWriterMain,=20true)=0A=20= PG_PROCTYPE(B_CHECKPOINTER,=20"checkpointer",=20= gettext_noop("checkpointer"),=20CheckpointerMain,=20true)=0A= +PG_PROCTYPE(B_DATACHECKSUMSWORKER_LAUNCHER,=20"checksums",=20= gettext_noop("datachecksum=20launcher"),=20NULL,=20false)=0A= +PG_PROCTYPE(B_DATACHECKSUMSWORKER_WORKER,=20"checksums",=20= gettext_noop("datachecksum=20worker"),=20NULL,=20false)=0A=20= PG_PROCTYPE(B_DEAD_END_BACKEND,=20"backend",=20gettext_noop("dead-end=20= client=20backend"),=20BackendMain,=20true)=0A=20PG_PROCTYPE(B_INVALID,=20= "postmaster",=20gettext_noop("unrecognized"),=20NULL,=20false)=0A=20= PG_PROCTYPE(B_IO_WORKER,=20"ioworker",=20gettext_noop("io=20worker"),=20= IoWorkerMain,=20true)=0Adiff=20--git=20= a/src/include/replication/decode.h=20b/src/include/replication/decode.h=0A= index=2049f00fc48b8..107e43ef750=20100644=0A---=20= a/src/include/replication/decode.h=0A+++=20= b/src/include/replication/decode.h=0A@@=20-22,6=20+22,7=20@@=20typedef=20= struct=20XLogRecordBuffer=0A=20}=20XLogRecordBuffer;=0A=20=0A=20extern=20= void=20xlog_decode(LogicalDecodingContext=20*ctx,=20XLogRecordBuffer=20= *buf);=0A+extern=20void=20xlog2_decode(LogicalDecodingContext=20*ctx,=20= XLogRecordBuffer=20*buf);=0A=20extern=20void=20= heap_decode(LogicalDecodingContext=20*ctx,=20XLogRecordBuffer=20*buf);=0A= =20extern=20void=20heap2_decode(LogicalDecodingContext=20*ctx,=20= XLogRecordBuffer=20*buf);=0A=20extern=20void=20= xact_decode(LogicalDecodingContext=20*ctx,=20XLogRecordBuffer=20*buf);=0A= diff=20--git=20a/src/include/storage/bufpage.h=20= b/src/include/storage/bufpage.h=0Aindex=20e5267b93fe6..aec539db292=20= 100644=0A---=20a/src/include/storage/bufpage.h=0A+++=20= b/src/include/storage/bufpage.h=0A@@=20-16,6=20+16,7=20@@=0A=20=0A=20= #include=20"access/xlogdefs.h"=0A=20#include=20"storage/block.h"=0A= +#include=20"storage/checksum.h"=0A=20#include=20"storage/off.h"=0A=20=0A= =20/*=20GUC=20variable=20*/=0A@@=20-230,7=20+231,6=20@@=20typedef=20= PageHeaderData=20*PageHeader;=0A=20=20*=20handling=20pages.=0A=20=20*/=0A= =20#define=20PG_PAGE_LAYOUT_VERSION=09=094=0A-#define=20= PG_DATA_CHECKSUM_VERSION=091=0A=20=0A=20/*=20= ----------------------------------------------------------------=0A=20=20= *=09=09=09=09=09=09page=20support=20functions=0A@@=20-501,6=20+501,7=20= @@=20do=20{=20\=0A=20#define=20PIV_LOG_WARNING=09=09=09(1=20<<=200)=0A=20= #define=20PIV_LOG_LOG=09=09=09=09(1=20<<=201)=0A=20#define=20= PIV_IGNORE_CHECKSUM_FAILURE=20(1=20<<=202)=0A+#define=20= PIV_ZERO_BUFFERS_ON_ERROR=20(1=20<<=203)=0A=20=0A=20#define=20= PageAddItem(page,=20item,=20size,=20offsetNumber,=20overwrite,=20= is_heap)=20\=0A=20=09PageAddItemExtended(page,=20item,=20size,=20= offsetNumber,=20\=0Adiff=20--git=20a/src/include/storage/checksum.h=20= b/src/include/storage/checksum.h=0Aindex=20ff417d5ae3e..fe5d30b4349=20= 100644=0A---=20a/src/include/storage/checksum.h=0A+++=20= b/src/include/storage/checksum.h=0A@@=20-15,6=20+15,20=20@@=0A=20=0A=20= #include=20"storage/block.h"=0A=20=0A+/*=0A+=20*=20Checksum=20state=200=20= is=20used=20for=20when=20data=20checksums=20are=20disabled=20(OFF).=0A+=20= *=20PG_DATA_CHECKSUM_INPROGRESS_{ON|OFF}=20defines=20that=20data=20= checksums=20are=20either=0A+=20*=20currently=20being=20enabled=20or=20= disabled,=20and=20PG_DATA_CHECKSUM_VERSION=20defines=0A+=20*=20that=20= data=20checksums=20are=20enabled.=0A+=20*/=0A+typedef=20enum=20= ChecksumStateType=0A+{=0A+=09PG_DATA_CHECKSUM_OFF=20=3D=200,=0A+=09= PG_DATA_CHECKSUM_VERSION,=0A+=09PG_DATA_CHECKSUM_INPROGRESS_OFF,=0A+=09= PG_DATA_CHECKSUM_INPROGRESS_ON,=0A+}=20ChecksumStateType;=0A+=0A=20/*=0A=20= =20*=20Compute=20the=20checksum=20for=20a=20Postgres=20page.=20=20The=20= page=20must=20be=20aligned=20on=20a=0A=20=20*=204-byte=20boundary.=0A= diff=20--git=20a/src/include/storage/lwlocklist.h=20= b/src/include/storage/lwlocklist.h=0Aindex=2059ee097977d..af8553bcb6c=20= 100644=0A---=20a/src/include/storage/lwlocklist.h=0A+++=20= b/src/include/storage/lwlocklist.h=0A@@=20-87,6=20+87,7=20@@=20= PG_LWLOCK(52,=20SerialControl)=0A=20PG_LWLOCK(53,=20= AioWorkerSubmissionQueue)=0A=20PG_LWLOCK(54,=20WaitLSN)=0A=20= PG_LWLOCK(55,=20LogicalDecodingControl)=0A+PG_LWLOCK(56,=20= DataChecksumsWorker)=0A=20=0A=20/*=0A=20=20*=20There=20also=20exist=20= several=20built-in=20LWLock=20tranches.=20=20As=20with=20the=20= predefined=0Adiff=20--git=20a/src/include/storage/procsignal.h=20= b/src/include/storage/procsignal.h=0Aindex=20348fba53a93..cc4f26aa33d=20= 100644=0A---=20a/src/include/storage/procsignal.h=0A+++=20= b/src/include/storage/procsignal.h=0A@@=20-48,6=20+48,10=20@@=20typedef=20= enum=0A=20=09PROCSIGNAL_BARRIER_SMGRRELEASE,=20/*=20ask=20smgr=20to=20= close=20files=20*/=0A=20=09PROCSIGNAL_BARRIER_UPDATE_XLOG_LOGICAL_INFO,=09= /*=20ask=20to=20update=0A=20=09=09=09=09=09=09=09=09=09=09=09=09=09=20*=20= XLogLogicalInfo=20*/=0A+=09PROCSIGNAL_BARRIER_CHECKSUM_OFF,=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON,=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF,=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_ON,=0A=20}=20ProcSignalBarrierType;=0A=20=0A=20= /*=0Adiff=20--git=20a/src/include/utils/backend_progress.h=20= b/src/include/utils/backend_progress.h=0Aindex=20= 6300dbd15d5..61e13c40e28=20100644=0A---=20= a/src/include/utils/backend_progress.h=0A+++=20= b/src/include/utils/backend_progress.h=0A@@=20-28,6=20+28,7=20@@=20= typedef=20enum=20ProgressCommandType=0A=20=09= PROGRESS_COMMAND_BASEBACKUP,=0A=20=09PROGRESS_COMMAND_COPY,=0A=20=09= PROGRESS_COMMAND_REPACK,=0A+=09PROGRESS_COMMAND_DATACHECKSUMS,=0A=20}=20= ProgressCommandType;=0A=20=0A=20#define=20PGSTAT_NUM_PROGRESS_PARAM=0920=0A= diff=20--git=20a/src/test/modules/Makefile=20b/src/test/modules/Makefile=0A= index=2028ce3b35eda..864b407abcf=20100644=0A---=20= a/src/test/modules/Makefile=0A+++=20b/src/test/modules/Makefile=0A@@=20= -20,6=20+20,7=20@@=20SUBDIRS=20=3D=20\=0A=20=09=09=20=20test_bitmapset=20= \=0A=20=09=09=20=20test_bloomfilter=20\=0A=20=09=09=20=20test_cloexec=20= \=0A+=09=09=20=20test_checksums=20\=0A=20=09=09=20=20test_copy_callbacks=20= \=0A=20=09=09=20=20test_custom_rmgrs=20\=0A=20=09=09=20=20= test_custom_stats=20\=0Adiff=20--git=20a/src/test/modules/meson.build=20= b/src/test/modules/meson.build=0Aindex=203ac291656c1..e5acacd5083=20= 100644=0A---=20a/src/test/modules/meson.build=0A+++=20= b/src/test/modules/meson.build=0A@@=20-20,6=20+20,7=20@@=20= subdir('test_binaryheap')=0A=20subdir('test_bitmapset')=0A=20= subdir('test_bloomfilter')=0A=20subdir('test_cloexec')=0A= +subdir('test_checksums')=0A=20subdir('test_copy_callbacks')=0A=20= subdir('test_cplusplusext')=0A=20subdir('test_custom_rmgrs')=0Adiff=20= --git=20a/src/test/modules/test_checksums/.gitignore=20= b/src/test/modules/test_checksums/.gitignore=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..871e943d50e=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/.gitignore=0A@@=20-0,0=20+1,2=20@@=0A= +#=20Generated=20by=20test=20suite=0A+/tmp_check/=0Adiff=20--git=20= a/src/test/modules/test_checksums/Makefile=20= b/src/test/modules/test_checksums/Makefile=0Anew=20file=20mode=20100644=0A= index=2000000000000..fa85b79ae57=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/Makefile=0A@@=20-0,0=20+1,40=20@@=0A= +#------------------------------------------------------------------------= -=0A+#=0A+#=20Makefile=20for=20src/test/modules/test_checksums=0A+#=0A+#=20= Portions=20Copyright=20(c)=201996-2026,=20PostgreSQL=20Global=20= Development=20Group=0A+#=20Portions=20Copyright=20(c)=201994,=20Regents=20= of=20the=20University=20of=20California=0A+#=0A+#=20= src/test/modules/test_checksums/Makefile=0A+#=0A= +#------------------------------------------------------------------------= -=0A+=0A+EXTRA_INSTALL=20=3D=20src/test/modules/injection_points=0A+=0A= +export=20enable_injection_points=0A+=0A+MODULE_big=20=3D=20= test_checksums=0A+OBJS=20=3D=20\=0A+=09$(WIN32RES)=20\=0A+=09= test_checksums.o=0A+PGFILEDESC=20=3D=20"test_checksums=20-=20test=20code=20= for=20data=20checksums"=0A+=0A+EXTENSION=20=3D=20test_checksums=0A+DATA=20= =3D=20test_checksums--1.0.sql=0A+=0A+ifdef=20USE_PGXS=0A+PG_CONFIG=20=3D=20= pg_config=0A+PGXS=20:=3D=20$(shell=20$(PG_CONFIG)=20--pgxs)=0A+include=20= $(PGXS)=0A+else=0A+subdir=20=3D=20src/test/modules/test_checksums=0A= +top_builddir=20=3D=20../../../..=0A+include=20= $(top_builddir)/src/Makefile.global=0A+include=20= $(top_srcdir)/contrib/contrib-global.mk=0A+endif=0A+=0A+check:=0A+=09= $(prove_check)=0A+=0A+installcheck:=0A+=09$(prove_installcheck)=0Adiff=20= --git=20a/src/test/modules/test_checksums/README=20= b/src/test/modules/test_checksums/README=0Anew=20file=20mode=20100644=0A= index=2000000000000..6a23e4ff0ae=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/README=0A@@=20-0,0=20+1,30=20@@=0A= +src/test/modules/test_checksums/README=0A+=0A+Regression=20tests=20for=20= data=20checksums=0A+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A+This=20directory=20= contains=20a=20test=20suite=20for=20enabling,=20and=20disabling,=20data=0A= +checksums=20both=20offline=20as=20well=20as=20in=20a=20running=20= cluster.=0A+=0A+Running=20the=20tests=20with=20autoconf=0A= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=0A+=0A+=20=20=20=20make=20check=0A+=0A+or=0A+=0A+=20=20= =20=20make=20installcheck=0A+=0A+Running=20the=20tests=20with=20meson=0A= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=0A+=46rom=20your=20build=20directory,=20issue=20the=20= following=20command:=0A+=0A+=20=20=20=20meson=20test=20-q=20= --print-errorlogs=20--suite=20setup=20--suite=20test_checksums=0A+=0A= +NOTE:=20This=20creates=20a=20temporary=20installation=20(in=20the=20= case=20of=20"make=20check"=20or=0A+"--suite=20setup"),=20with=20multiple=20= nodes,=20be=20they=20master=20or=20standby(s)=20for=20the=0A+purpose=20= of=20the=20tests.=0A+=0A+NOTE:=20This=20test=20suite=20requires=20TAP=20= tests=20to=20be=20enabled,=20a=20subset=20of=20the=20tests=0A+also=20= require=20injection=20points=20to=20function.=20=20In=20order=20to=20run=20= the=20extended=20test=0A+then=20"checksum_extended"=20must=20be=20set=20= in=20the=20PG_TEST_EXTRA=20environment=0A+variable.=0Adiff=20--git=20= a/src/test/modules/test_checksums/meson.build=20= b/src/test/modules/test_checksums/meson.build=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..9b1421a9b91=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/meson.build=0A@@=20-0,0=20+1,38=20@@=0A= +#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20Development=20Group=0A= +=0A+test_checksums_sources=20=3D=20files(=0A+=09'test_checksums.c',=0A= +)=0A+=0A+test_checksums=20=3D=20shared_module('test_checksums',=0A+=09= test_checksums_sources,=0A+=09kwargs:=20pg_test_mod_args,=0A+)=0A= +test_install_libs=20+=3D=20test_checksums=0A+=0A+test_install_data=20+=3D= =20files(=0A+=09'test_checksums.control',=0A+=09= 'test_checksums--1.0.sql',=0A+)=0A+=0A+tests=20+=3D=20{=0A+=20=20'name':=20= 'test_checksums',=0A+=20=20'sd':=20meson.current_source_dir(),=0A+=20=20= 'bd':=20meson.current_build_dir(),=0A+=20=20'tap':=20{=0A+=20=20=20=20= 'env':=20{=0A+=20=20=20=20=20=20=20'enable_injection_points':=20= get_option('injection_points')=20?=20'yes'=20:=20'no',=0A+=09},=0A+=20=20= =20=20'tests':=20[=0A+=20=20=20=20=20=20't/001_basic.pl',=0A+=20=20=20=20= =20=20't/002_restarts.pl',=0A+=20=20=20=20=20=20= 't/003_standby_restarts.pl',=0A+=20=20=20=20=20=20't/004_offline.pl',=0A= +=20=20=20=20=20=20't/005_injection.pl',=0A+=20=20=20=20=20=20= 't/006_pgbench_single.pl',=0A+=20=20=20=20=20=20= 't/007_pgbench_standby.pl',=0A+=20=20=20=20=20=20't/008_pitr.pl',=0A+=20=20= =20=20=20=20't/009_fpi.pl',=0A+=20=20=20=20],=0A+=20=20},=0A+}=0Adiff=20= --git=20a/src/test/modules/test_checksums/t/001_basic.pl=20= b/src/test/modules/test_checksums/t/001_basic.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..c008e95fbff=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/001_basic.pl=0A@@=20-0,0=20+1,63=20= @@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=0A+use=20strict;=0A+use=20= warnings=20FATAL=20=3D>=20'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A= +use=20PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20= FindBin;=0A+use=20lib=20$FindBin::RealBin;=0A+=0A+use=20= DataChecksums::Utils;=0A+=0A+#=20Initialize=20node=20with=20checksums=20= disabled.=0A+my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('basic_node');=0A= +$node->init(no_data_checksums=20=3D>=201);=0A+$node->start;=0A+=0A+#=20= Create=20some=20content=20to=20have=20un-checksummed=20data=20in=20the=20= cluster=0A+$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Ensure=20that=20= checksums=20are=20turned=20off=0A+test_checksum_state($node,=20'off');=0A= +=0A+#=20Enable=20data=20checksums=20and=20wait=20for=20the=20state=20= transition=20to=20'on'=0A+enable_data_checksums($node,=20wait=20=3D>=20= 'on');=0A+=0A+#=20Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20= can=20read=20back=20data=0A+my=20$result=20=3D=0A+=20=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201=20");=0A+is($result,=20'9999',=20'ensure=20checksummed=20pages=20= can=20be=20read=20back');=0A+=0A+#=20Enable=20data=20checksums=20again=20= which=20should=20be=20a=20no-op=20so=20we=20explicitly=20don't=0A+#=20= wait=20for=20any=20state=20transition=20as=20none=20should=20happen=20= here=0A+enable_data_checksums($node);=0A+test_checksum_state($node,=20= 'on');=0A+#=20..and=20make=20sure=20we=20can=20still=20read/write=20data=0A= +$node->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;");=0A+$result=20=3D=20$node->safe_psql('postgres',=20"SELECT=20= count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20'10000',=20= 'ensure=20checksummed=20pages=20can=20be=20read=20back');=0A+=0A+#=20= Disable=20checksums=20again=20and=20wait=20for=20the=20state=20= transition=0A+disable_data_checksums($node,=20wait=20=3D>=201);=0A+=0A+#=20= Test=20reading=20data=20again=0A+$result=20=3D=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+is($result,=20'10000',=20'ensure=20previously=20checksummed=20= pages=20can=20be=20read=20back');=0A+=0A+#=20Re-enable=20checksums=20and=20= make=20sure=20that=20the=20underlying=20data=20has=20changed=20to=0A+#=20= ensure=20that=20checksums=20will=20be=20different.=0A= +$node->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;");=0A+enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+#=20= Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20can=20read=20= back=20the=20data=0A+$result=20=3D=20$node->safe_psql('postgres',=20= "SELECT=20count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20= '10000',=20'ensure=20checksummed=20pages=20can=20be=20read=20back');=0A+=0A= +$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/002_restarts.pl=20= b/src/test/modules/test_checksums/t/002_restarts.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..bab59be82bd=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/002_restarts.pl=0A@@=20-0,0=20+1,110=20= @@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=20a=0A+#=20restart=20= which=20breaks=20processing.=0A+use=20strict;=0A+use=20warnings=20FATAL=20= =3D>=20'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A+use=20= PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20FindBin;=0A= +use=20lib=20$FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A= +#=20Initialize=20node=20with=20checksums=20disabled.=0A+my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('restarts_node');=0A= +$node->init(no_data_checksums=20=3D>=201);=0A+$node->start;=0A+=0A+#=20= Initialize=20result=20storage=20for=20queries=0A+my=20$result;=0A+=0A+#=20= Create=20some=20content=20to=20have=20un-checksummed=20data=20in=20the=20= cluster=0A+$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Ensure=20that=20= checksums=20are=20disabled=0A+test_checksum_state($node,=20'off');=0A+=0A= +SKIP:=0A+{=0A+=09skip=20'Data=20checksum=20delay=20tests=20not=20= enabled=20in=20PG_TEST_EXTRA',=206=0A+=09=20=20if=20= (!$ENV{PG_TEST_EXTRA}=0A+=09=09||=20$ENV{PG_TEST_EXTRA}=20!~=20= /\bchecksum_extended\b/);=0A+=0A+=09#=20Create=20a=20barrier=20for=20= checksum=20enablement=20to=20block=20on,=20in=20this=20case=20a=20pre-=0A= +=09#=20existing=20temporary=20table=20which=20is=20kept=20open=20while=20= processing=20is=20started.=0A+=09#=20We=20can=20accomplish=20this=20by=20= setting=20up=20an=20interactive=20psql=20process=20which=0A+=09#=20keeps=20= the=20temporary=20table=20created=20as=20we=20enable=20checksums=20in=20= another=20psql=0A+=09#=20process.=0A+=09#=0A+=09#=20This=20is=20a=20= similar=20test=20to=20the=20synthetic=20variant=20in=20005_injection.pl=0A= +=09#=20which=20fakes=20this=20scenario.=0A+=09my=20$bsession=20=3D=20= $node->background_psql('postgres');=0A+=09$bsession->query_safe('CREATE=20= TEMPORARY=20TABLE=20tt=20(a=20integer);');=0A+=0A+=09#=20In=20another=20= session,=20make=20sure=20we=20can=20see=20the=20blocking=20temp=20table=20= but=0A+=09#=20start=20processing=20anyways=20and=20check=20that=20we=20= are=20blocked=20with=20a=20proper=0A+=09#=20wait=20event.=0A+=09$result=20= =3D=20$node->safe_psql('postgres',=0A+=09=09"SELECT=20relpersistence=20= FROM=20pg_catalog.pg_class=20WHERE=20relname=20=3D=20'tt';"=0A+=09);=0A+=09= is($result,=20't',=20'ensure=20we=20can=20see=20the=20temporary=20= table');=0A+=0A+=09#=20Enabling=20data=20checksums=20shouldn't=20work=20= as=20the=20process=20is=20blocked=20on=20the=0A+=09#=20temporary=20table=20= held=20open=20by=20$bsession.=20Ensure=20that=20we=20reach=20inprogress-=0A= +=09#=20on=20before=20we=20do=20more=20tests.=0A+=09= enable_data_checksums($node,=20wait=20=3D>=20'inprogress-on');=0A+=0A+=09= #=20Wait=20for=20processing=20to=20finish=20and=20the=20worker=20waiting=20= for=20leftover=20temp=0A+=09#=20relations=20to=20be=20able=20to=20= actually=20finish=0A+=09$result=20=3D=20$node->poll_query_until(=0A+=09=09= 'postgres',=0A+=09=09"SELECT=20wait_event=20FROM=20= pg_catalog.pg_stat_activity=20"=0A+=09=09=20=20.=20"WHERE=20backend_type=20= =3D=20'datachecksum=20worker';",=0A+=09=09= 'ChecksumEnableTemptableWait');=0A+=0A+=09#=20The=20datachecksumsworker=20= waits=20for=20temporary=20tables=20to=20disappear=20for=203=0A+=09#=20= seconds=20before=20retrying,=20so=20sleep=20for=204=20seconds=20to=20be=20= guaranteed=20to=20see=0A+=09#=20a=20retry=20cycle=0A+=09sleep(4);=0A+=0A= +=09#=20Re-check=20the=20wait=20event=20to=20ensure=20we=20are=20blocked=20= on=20the=20right=20thing.=0A+=09$result=20=3D=20= $node->safe_psql('postgres',=0A+=09=09=09"SELECT=20wait_event=20FROM=20= pg_catalog.pg_stat_activity=20"=0A+=09=09=20=20.=20"WHERE=20backend_type=20= =3D=20'datachecksum=20worker';");=0A+=09is($result,=20= 'ChecksumEnableTemptableWait',=0A+=09=09'ensure=20the=20correct=20wait=20= condition=20is=20set');=0A+=09test_checksum_state($node,=20= 'inprogress-on');=0A+=0A+=09#=20Stop=20the=20cluster=20while=20bsession=20= is=20still=20attached.=20=20We=20can't=20close=20the=0A+=09#=20session=20= first=20since=20the=20brief=20period=20between=20closing=20and=20= stopping=20might=0A+=09#=20be=20enough=20for=20checksums=20to=20get=20= enabled.=0A+=09$node->stop;=0A+=09$bsession->quit;=0A+=09$node->start;=0A= +=0A+=09#=20Ensure=20the=20checksums=20aren't=20enabled=20across=20the=20= restart.=20=20This=20leaves=20the=0A+=09#=20cluster=20in=20the=20same=20= state=20as=20before=20we=20entered=20the=20SKIP=20block.=0A+=09= test_checksum_state($node,=20'off');=0A+}=0A+=0A= +enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+$result=20=3D=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+is($result,=20'9999',=20'ensure=20checksummed=20pages=20can=20= be=20read=20back');=0A+=0A+$result=20=3D=20$node->poll_query_until(=0A+=09= 'postgres',=0A+=09"SELECT=20count(*)=20FROM=20pg_stat_activity=20WHERE=20= backend_type=20LIKE=20'datachecksum%';",=0A+=09'0');=0A+is($result,=201,=20= 'await=20datachecksums=20worker/launcher=20termination');=0A+=0A= +disable_data_checksums($node,=20wait=20=3D>=201);=0A+=0A+$node->stop;=0A= +done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/003_standby_restarts.pl=20= b/src/test/modules/test_checksums/t/003_standby_restarts.pl=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..6b016925651=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/003_standby_restarts.pl=0A@@=20-0,0=20= +1,114=20@@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=0A+#=20streaming=20= replication=0A+use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20'all';=0A= +use=20PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A= +use=20Test::More;=0A+=0A+use=20FindBin;=0A+use=20lib=20= $FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A+#=20= Initialize=20primary=20node=0A+my=20$node_primary=20=3D=20= PostgreSQL::Test::Cluster->new('standby_restarts_primary');=0A= +$node_primary->init(allows_streaming=20=3D>=201,=20no_data_checksums=20= =3D>=201);=0A+$node_primary->start;=0A+=0A+my=20$slotname=20=3D=20= 'physical_slot';=0A+$node_primary->safe_psql('postgres',=0A+=09"SELECT=20= pg_create_physical_replication_slot('$slotname')");=0A+=0A+#=20Take=20= backup=0A+my=20$backup_name=20=3D=20'my_backup';=0A= +$node_primary->backup($backup_name);=0A+=0A+#=20Create=20streaming=20= standby=20linking=20to=20primary=0A+my=20$node_standby=20=3D=20= PostgreSQL::Test::Cluster->new('standby_restarts_standby');=0A= +$node_standby->init_from_backup($node_primary,=20$backup_name,=0A+=09= has_streaming=20=3D>=201);=0A+$node_standby->append_conf(=0A+=09= 'postgresql.conf',=20qq[=0A+primary_slot_name=20=3D=20'$slotname'=0A+]);=0A= +$node_standby->start;=0A+=0A+#=20Create=20some=20content=20on=20the=20= primary=20to=20have=20un-checksummed=20data=20in=20the=20cluster=0A= +$node_primary->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Wait=20for=20= standbys=20to=20catch=20up=0A= +$node_primary->wait_for_catchup($node_standby,=20'replay',=0A+=09= $node_primary->lsn('insert'));=0A+=0A+#=20Check=20that=20checksums=20are=20= turned=20off=20on=20all=20nodes=0A+test_checksum_state($node_primary,=20= 'off');=0A+test_checksum_state($node_standby,=20'off');=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Enable=20checksums=20for=20the=20cluster,=20and=20make=20sure=20= that=20both=20the=20primary=20and=0A+#=20standby=20change=20state.=0A+#=0A= +=0A+#=20Ensure=20that=20the=20primary=20switches=20to=20"inprogress-on"=0A= +enable_data_checksums($node_primary,=20wait=20=3D>=20'inprogress-on');=0A= +#=20Wait=20for=20checksum=20enable=20to=20be=20replayed=0A= +$node_primary->wait_for_catchup($node_standby,=20'replay');=0A+=0A+#=20= Ensure=20that=20the=20standby=20has=20switched=20to=20"inprogress-on"=20= or=20"on".=20=20Normally=20it=0A+#=20would=20be=20"inprogress-on",=20but=20= it=20is=20theoretically=20possible=20for=20the=20primary=20to=0A+#=20= complete=20the=20checksum=20enabling=20*and*=20have=20the=20standby=20= replay=20that=20record=0A+#=20before=20we=20reach=20the=20check=20below.=0A= +my=20$result=20=3D=20$node_standby->poll_query_until(=0A+=09'postgres',=0A= +=09"SELECT=20setting=20=3D=20'off'=20FROM=20pg_catalog.pg_settings=20= WHERE=20name=20=3D=20'data_checksums';",=0A+=09'f');=0A+is($result,=201,=20= 'ensure=20standby=20has=20absorbed=20the=20inprogress-on=20barrier');=0A= +$result=20=3D=20$node_standby->safe_psql('postgres',=0A+=09"SELECT=20= setting=20FROM=20pg_catalog.pg_settings=20WHERE=20name=20=3D=20= 'data_checksums';"=0A+);=0A+=0A+is(($result=20eq=20'inprogress-on'=20||=20= $result=20eq=20'on'),=0A+=091,=20'ensure=20checksums=20are=20on,=20or=20= in=20progress,=20on=20standby_1');=0A+=0A+#=20Insert=20some=20more=20= data=20which=20should=20be=20checksummed=20on=20INSERT=0A= +$node_primary->safe_psql('postgres',=0A+=09"INSERT=20INTO=20t=20VALUES=20= (generate_series(1,=2010000));");=0A+=0A+#=20Wait=20for=20checksums=20= enabled=20on=20the=20primary=20and=20standby=0A= +wait_for_checksum_state($node_primary,=20'on');=0A= +wait_for_checksum_state($node_standby,=20'on');=0A+=0A+$result=20=3D=0A= +=20=20$node_primary->safe_psql('postgres',=20"SELECT=20count(a)=20FROM=20= t=20WHERE=20a=20>=201");=0A+is($result,=20'19998',=20'ensure=20we=20can=20= safely=20read=20all=20data=20with=20checksums');=0A+=0A+$result=20=3D=20= $node_primary->poll_query_until(=0A+=09'postgres',=0A+=09"SELECT=20= count(*)=20FROM=20pg_stat_activity=20WHERE=20backend_type=20LIKE=20= 'datachecksum%';",=0A+=09'0');=0A+is($result,=201,=20'await=20= datachecksums=20worker/launcher=20termination');=0A+=0A+#=0A+#=20Disable=20= checksums=20and=20ensure=20it's=20propagated=20to=20standby=20and=20that=20= we=20can=0A+#=20still=20read=20all=20data=0A+#=0A+=0A+#=20Disable=20= checksums=20and=20wait=20for=20the=20operation=20to=20be=20replayed=0A= +disable_data_checksums($node_primary);=0A= +$node_primary->wait_for_catchup($node_standby,=20'replay');=0A+#=20= Ensure=20that=20the=20primary=20and=20standby=20has=20switched=20to=20= off=0A+wait_for_checksum_state($node_primary,=20'off');=0A= +wait_for_checksum_state($node_standby,=20'off');=0A+#=20Doublecheck=20= reading=20data=20without=20errors=0A+$result=20=3D=0A+=20=20= $node_primary->safe_psql('postgres',=20"SELECT=20count(a)=20FROM=20t=20= WHERE=20a=20>=201");=0A+is($result,=20"19998",=20'ensure=20we=20can=20= safely=20read=20all=20data=20without=20checksums');=0A+=0A= +$node_standby->stop;=0A+$node_primary->stop;=0A+done_testing();=0Adiff=20= --git=20a/src/test/modules/test_checksums/t/004_offline.pl=20= b/src/test/modules/test_checksums/t/004_offline.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..f1972bddff1=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/004_offline.pl=0A@@=20-0,0=20+1,82=20= @@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20offline=20from=20various=20states=0A+#=20of=20= checksum=20processing=0A+use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20= 'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A+use=20= PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20FindBin;=0A= +use=20lib=20$FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A= +#=20Initialize=20node=20with=20checksums=20disabled.=0A+my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('offline_node');=0A= +$node->init(no_data_checksums=20=3D>=201);=0A+$node->start;=0A+=0A+#=20= Create=20some=20content=20to=20have=20un-checksummed=20data=20in=20the=20= cluster=0A+$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Ensure=20that=20= checksums=20are=20disabled=0A+test_checksum_state($node,=20'off');=0A+=0A= +#=20Enable=20checksums=20offline=20using=20pg_checksums=0A+$node->stop;=0A= +$node->checksum_enable_offline;=0A+$node->start;=0A+=0A+#=20Ensure=20= that=20checksums=20are=20enabled=0A+test_checksum_state($node,=20'on');=0A= +=0A+#=20Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20can=20= read=20back=20some=20data=0A+my=20$result=20=3D=0A+=20=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+is($result,=20'9999',=20'ensure=20checksummed=20pages=20can=20= be=20read=20back');=0A+=0A+#=20Disable=20checksums=20offline=20again=20= using=20pg_checksums=0A+$node->stop;=0A+$node->checksum_disable_offline;=0A= +$node->start;=0A+=0A+#=20Ensure=20that=20checksums=20are=20disabled=0A= +test_checksum_state($node,=20'off');=0A+=0A+#=20Create=20a=20barrier=20= for=20checksum=20enablement=20to=20block=20on,=20in=20this=20case=20a=20= pre-=0A+#=20existing=20temporary=20table=20which=20is=20kept=20open=20= while=20processing=20is=20started.=20We=0A+#=20can=20accomplish=20this=20= by=20setting=20up=20an=20interactive=20psql=20process=20which=20keeps=20= the=0A+#=20temporary=20table=20created=20as=20we=20enable=20checksums=20= in=20another=20psql=20process.=0A+=0A+my=20$bsession=20=3D=20= $node->background_psql('postgres');=0A+$bsession->query_safe('CREATE=20= TEMPORARY=20TABLE=20tt=20(a=20integer);');=0A+=0A+#=20In=20another=20= session,=20make=20sure=20we=20can=20see=20the=20blocking=20temp=20table=20= but=20start=0A+#=20processing=20anyways=20and=20check=20that=20we=20are=20= blocked=20with=20a=20proper=20wait=20event.=0A+$result=20=3D=20= $node->safe_psql('postgres',=0A+=09"SELECT=20relpersistence=20FROM=20= pg_catalog.pg_class=20WHERE=20relname=20=3D=20'tt';");=0A+is($result,=20= 't',=20'ensure=20we=20can=20see=20the=20temporary=20table');=0A+=0A= +enable_data_checksums($node,=20wait=20=3D>=20'inprogress-on');=0A+=0A+#=20= Turn=20the=20cluster=20off=20and=20enable=20checksums=20offline,=20then=20= start=20back=20up=0A+$bsession->quit;=0A+$node->stop;=0A= +$node->checksum_enable_offline;=0A+$node->start;=0A+=0A+#=20Ensure=20= that=20checksums=20are=20now=20enabled=20even=20though=20processing=20= wasn't=0A+#=20restarted=0A+test_checksum_state($node,=20'on');=0A+=0A+#=20= Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20can=20read=20= back=20some=20data=0A+$result=20=3D=20$node->safe_psql('postgres',=20= "SELECT=20count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20= '9999',=20'ensure=20checksummed=20pages=20can=20be=20read=20back');=0A+=0A= +$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/005_injection.pl=20= b/src/test/modules/test_checksums/t/005_injection.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..897f282a1f2=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/005_injection.pl=0A@@=20-0,0=20+1,74=20= @@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=0A+#=20injection=20= point=20tests=20injecting=20failures=20into=20the=20processing=0A+=0A= +use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20'all';=0A+=0A+use=20= PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A+use=20= Test::More;=0A+=0A+use=20FindBin;=0A+use=20lib=20$FindBin::RealBin;=0A+=0A= +use=20DataChecksums::Utils;=0A+=0A+if=20($ENV{enable_injection_points}=20= ne=20'yes')=0A+{=0A+=09plan=20skip_all=20=3D>=20'Injection=20points=20= not=20supported=20by=20this=20build';=0A+}=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Test=20cluster=20setup=0A+#=0A+=0A+#=20Initiate=20testcluster=0A= +my=20$node=20=3D=20PostgreSQL::Test::Cluster->new('injection_node');=0A= +$node->init(no_data_checksums=20=3D>=201);=0A+$node->start;=0A+=0A+#=20= Set=20up=20test=20environment=0A+$node->safe_psql('postgres',=20'CREATE=20= EXTENSION=20test_checksums;');=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Inducing=20failures=20and=20crashes=20in=20processing=0A+=0A+#=20= Force=20enabling=20checksums=20to=20fail=20by=20marking=20one=20of=20the=20= databases=20as=20having=0A+#=20failed=20in=20processing.=0A= +disable_data_checksums($node,=20wait=20=3D>=201);=0A= +$node->safe_psql('postgres',=20'SELECT=20= dcw_inject_fail_database(true);');=0A+enable_data_checksums($node,=20= wait=20=3D>=20'off');=0A+$node->safe_psql('postgres',=20'SELECT=20= dcw_inject_fail_database(false);');=0A+=0A+#=20Make=20sure=20that=20= disabling=20after=20a=20failure=20works=0A= +disable_data_checksums($node);=0A+test_checksum_state($node,=20'off');=0A= +=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Timing=20and=20retry=20related=20tests=0A+#=0A+=0A+SKIP:=0A+{=0A= +=09skip=20'Data=20checksum=20delay=20tests=20not=20enabled=20in=20= PG_TEST_EXTRA',=204=0A+=09=20=20if=20(!$ENV{PG_TEST_EXTRA}=0A+=09=09||=20= $ENV{PG_TEST_EXTRA}=20!~=20/\bchecksum_extended\b/);=0A+=0A+=09#=20= Inject=20a=20delay=20in=20the=20barrier=20for=20enabling=20checksums=0A+=09= disable_data_checksums($node,=20wait=20=3D>=201);=0A+=09= $node->safe_psql('postgres',=20'SELECT=20dcw_inject_delay_barrier();');=0A= +=09enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+=09#=20= Fake=20the=20existence=20of=20a=20temporary=20table=20at=20the=20start=20= of=20processing,=20which=0A+=09#=20will=20force=20the=20processing=20to=20= wait=20and=20retry=20in=20order=20to=20wait=20for=20it=20to=0A+=09#=20= disappear.=0A+=09disable_data_checksums($node,=20wait=20=3D>=201);=0A+=09= $node->safe_psql('postgres',=20'SELECT=20dcw_fake_temptable(true);');=0A= +=09enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+}=0A+=0A= +$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/006_pgbench_single.pl=20= b/src/test/modules/test_checksums/t/006_pgbench_single.pl=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..0ab5b04b931=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/006_pgbench_single.pl=0A@@=20-0,0=20= +1,275=20@@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=0A+#=20concurrent=20= activity=20via=20pgbench=20runs=0A+=0A+use=20strict;=0A+use=20warnings=20= FATAL=20=3D>=20'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A+use=20= PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20FindBin;=0A= +use=20lib=20$FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A= +#=20This=20test=20suite=20is=20expensive,=20or=20very=20expensive,=20to=20= execute.=20=20There=20are=20two=0A+#=20PG_TEST_EXTRA=20options=20for=20= running=20it,=20"checksum"=20for=20a=20pared-down=20test=20suite=0A+#=20= an=20"checksum_extended"=20for=20the=20full=20suite.=20=20The=20full=20= suite=20can=20run=20for=20hours=0A+#=20on=20slow=20or=20constrained=20= systems.=0A+my=20$extended=20=3D=20undef;=0A+if=20($ENV{PG_TEST_EXTRA})=0A= +{=0A+=09$extended=20=3D=201=20if=20($ENV{PG_TEST_EXTRA}=20=3D~=20= /\bchecksum_extended\b/);=0A+=09plan=20skip_all=20=3D>=20'Expensive=20= data=20checksums=20test=20disabled'=0A+=09=20=20unless=20= ($ENV{PG_TEST_EXTRA}=20=3D~=20/\bchecksum(_extended)?\b/);=0A+}=0A+else=0A= +{=0A+=09plan=20skip_all=20=3D>=20'Expensive=20data=20checksums=20test=20= disabled';=0A+}=0A+=0A+if=20($ENV{enable_injection_points}=20ne=20'yes')=0A= +{=0A+=09plan=20skip_all=20=3D>=20'Injection=20points=20not=20supported=20= by=20this=20build';=0A+}=0A+=0A+my=20$node;=0A+my=20$node_loglocation=20= =3D=200;=0A+=0A+#=20The=20number=20of=20full=20test=20iterations=20which=20= will=20be=20performed.=20The=20exact=20number=0A+#=20of=20tests=20= performed=20and=20the=20wall=20time=20taken=20is=20non-deterministic=20= as=20the=20test=0A+#=20performs=20a=20lot=20of=20randomized=20actions,=20= but=2010=20iterations=20will=20be=20a=20long=20test=0A+#=20run=20= regardless.=0A+my=20$TEST_ITERATIONS=20=3D=201;=0A+$TEST_ITERATIONS=20=3D=20= 10=20if=20($extended);=0A+=0A+#=20Variables=20which=20record=20the=20= current=20state=20of=20the=20cluster=0A+my=20$data_checksum_state=20=3D=20= 'off';=0A+my=20$pgbench=20=3D=20undef;=0A+=0A+#=20Start=20a=20pgbench=20= run=20in=20the=20background=20against=20the=20server=20specified=20via=20= the=0A+#=20port=20passed=20as=20parameter.=0A+sub=20= background_rw_pgbench=0A+{=0A+=09my=20$port=20=3D=20shift;=0A+=0A+=09#=20= If=20a=20previous=20pgbench=20is=20still=20running,=20start=20by=20= shutting=20it=20down.=0A+=09$pgbench->finish=20if=20$pgbench;=0A+=0A+=09= my=20$clients=20=3D=201;=0A+=09my=20$runtime=20=3D=202;=0A+=0A+=09if=20= ($extended)=0A+=09{=0A+=09=09#=20Randomize=20the=20number=20of=20pgbench=20= clients=20a=20bit=20(range=201-16)=0A+=09=09$clients=20=3D=201=20+=20= int(rand(15));=0A+=09=09$runtime=20=3D=20600;=0A+=09}=0A+=09my=20@cmd=20= =3D=20('pgbench',=20'-p',=20$port,=20'-T',=20$runtime,=20'-c',=20= $clients);=0A+=0A+=09#=20Randomize=20whether=20we=20spawn=20connections=20= or=20not=0A+=09push(@cmd,=20'-C')=20if=20($extended=20&&=20cointoss);=0A= +=09#=20Finally=20add=20the=20database=20name=20to=20use=0A+=09= push(@cmd,=20'postgres');=0A+=0A+=09$pgbench=20=3D=20IPC::Run::start(=0A= +=09=09\@cmd,=0A+=09=09'<'=20=3D>=20'/dev/null',=0A+=09=09'>'=20=3D>=20= '/dev/null',=0A+=09=09'2>'=20=3D>=20'/dev/null',=0A+=09=09= IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default));=0A+}=0A+=0A= +#=20Invert=20the=20state=20of=20data=20checksums=20in=20the=20cluster,=20= if=20data=20checksums=20are=20on=0A+#=20then=20disable=20them=20and=20= vice=20versa.=20Also=20performs=20proper=20validation=20of=20the=0A+#=20= before=20and=20after=20state.=0A+sub=20flip_data_checksums=0A+{=0A+=09#=20= First,=20make=20sure=20the=20cluster=20is=20in=20the=20state=20we=20= expect=20it=20to=20be=0A+=09test_checksum_state($node,=20= $data_checksum_state);=0A+=0A+=09if=20($data_checksum_state=20eq=20= 'off')=0A+=09{=0A+=09=09#=20Coin-toss=20to=20see=20if=20we=20are=20= injecting=20a=20retry=20due=20to=20a=20temptable=0A+=09=09= $node->safe_psql('postgres',=20'SELECT=20dcw_fake_temptable();')=0A+=09=09= =20=20if=20cointoss();=0A+=0A+=09=09#=20log=20LSN=20right=20before=20we=20= start=20changing=20checksums=0A+=09=09my=20$result=20=3D=0A+=09=09=20=20= $node->safe_psql('postgres',=20"SELECT=20pg_current_wal_lsn()");=0A+=09=09= note("LSN=20before=20enabling:=20"=20.=20$result=20.=20"\n");=0A+=0A+=09=09= #=20Ensure=20that=20the=20primary=20switches=20to=20"inprogress-on"=0A+=09= =09enable_data_checksums($node,=20wait=20=3D>=20'inprogress-on');=0A+=0A= +=09=09random_sleep()=20if=20($extended);=0A+=0A+=09=09#=20Wait=20for=20= checksums=20enabled=20on=20the=20primary=0A+=09=09= wait_for_checksum_state($node,=20'on');=0A+=0A+=09=09#=20log=20LSN=20= right=20after=20the=20primary=20flips=20checksums=20to=20"on"=0A+=09=09= $result=20=3D=20$node->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=09=09note("LSN=20after=20enabling:=20"=20.=20= $result=20.=20"\n");=0A+=0A+=09=09random_sleep()=20if=20($extended);=0A+=0A= +=09=09$node->safe_psql('postgres',=20'SELECT=20= dcw_fake_temptable(false);');=0A+=09=09$data_checksum_state=20=3D=20= 'on';=0A+=09}=0A+=09elsif=20($data_checksum_state=20eq=20'on')=0A+=09{=0A= +=09=09random_sleep()=20if=20($extended);=0A+=0A+=09=09#=20log=20LSN=20= right=20before=20we=20start=20changing=20checksums=0A+=09=09my=20$result=20= =3D=0A+=09=09=20=20$node->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=09=09note("LSN=20before=20disabling:=20"=20.=20= $result=20.=20"\n");=0A+=0A+=09=09disable_data_checksums($node);=0A+=0A+=09= =09#=20Wait=20for=20checksums=20disabled=20on=20the=20primary=0A+=09=09= wait_for_checksum_state($node,=20'off');=0A+=0A+=09=09#=20log=20LSN=20= right=20after=20the=20primary=20flips=20checksums=20to=20"off"=0A+=09=09= $result=20=3D=20$node->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=09=09note("LSN=20after=20disabling:=20"=20.=20= $result=20.=20"\n");=0A+=0A+=09=09random_sleep()=20if=20($extended);=0A+=0A= +=09=09$data_checksum_state=20=3D=20'off';=0A+=09}=0A+=09else=0A+=09{=0A= +=09=09#=20This=20should=20only=20happen=20due=20to=20programmer=20error=20= when=20hacking=20on=20the=0A+=09=09#=20test=20code,=20but=20since=20that=20= might=20pass=20subtly=20we=20error=20out.=0A+=09=09= BAIL_OUT('data_checksum_state=20variable=20has=20invalid=20state:'=0A+=09= =09=09=20=20.=20$data_checksum_state);=0A+=09}=0A+}=0A+=0A+#=20Create=20= and=20start=20a=20cluster=20with=20one=20node=0A+$node=20=3D=20= PostgreSQL::Test::Cluster->new('pgbench_single_main');=0A= +$node->init(allows_streaming=20=3D>=201,=20no_data_checksums=20=3D>=20= 1);=0A+#=20max_connections=20need=20to=20be=20bumped=20in=20order=20to=20= accommodate=20for=20pgbench=20clients=0A+#=20and=20log_statement=20is=20= dialled=20down=20since=20it=20otherwise=20will=20generate=20enormous=0A= +#=20amounts=20of=20logging.=20Page=20verification=20failures=20are=20= still=20logged.=0A+$node->append_conf(=0A+=09'postgresql.conf',=0A+=09= qq[=0A+max_connections=20=3D=20100=0A+log_statement=20=3D=20none=0A+]);=0A= +$node->start;=0A+$node->safe_psql('postgres',=20'CREATE=20EXTENSION=20= test_checksums;');=0A+#=20Create=20some=20content=20to=20have=20= un-checksummed=20data=20in=20the=20cluster=0A= +$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20SELECT=20= generate_series(1,=20100000)=20AS=20a;");=0A+#=20Initialize=20pgbench=0A= +my=20$scalefactor=20=3D=20($extended=20?=2010=20:=201);=0A= +$node->command_ok(=0A+=09[=0A+=09=09'pgbench',=20'-p',=20$node->port,=20= '-i',=0A+=09=09'-s',=20$scalefactor,=20'-q',=20'postgres'=0A+=09]);=0A+#=20= Start=20the=20test=20suite=20with=20pgbench=20running.=0A= +background_rw_pgbench($node->port);=0A+=0A+#=20Main=20test=20suite.=20= This=20loop=20will=20start=20a=20pgbench=20run=20on=20the=20cluster=20= and=20while=0A+#=20that's=20running=20flip=20the=20state=20of=20data=20= checksums=20concurrently.=20It=20will=20then=0A+#=20randomly=20restart=20= the=20cluster=20and=20then=20check=20for=0A+#=20the=20desired=20state.=20= =20The=20idea=20behind=20doing=20things=20randomly=20is=20to=20stress=20= out=0A+#=20any=20timing=20related=20issues=20by=20subjecting=20the=20= cluster=20for=20varied=20workloads.=0A+#=20A=20TODO=20is=20to=20generate=20= a=20trace=20such=20that=20any=20test=20failure=20can=20be=20traced=20to=0A= +#=20its=20order=20of=20operations=20for=20debugging.=0A+for=20(my=20$i=20= =3D=200;=20$i=20<=20$TEST_ITERATIONS;=20$i++)=0A+{=0A+=09note("iteration=20= ",=20($i=20+=201),=20"=20of=20",=20$TEST_ITERATIONS);=0A+=0A+=09if=20= (!$node->is_alive)=0A+=09{=0A+=09=09#=20Start,=20to=20do=20recovery,=20= and=20stop=0A+=09=09$node->start;=0A+=09=09$node->stop('fast');=0A+=0A+=09= =09#=20Since=20the=20log=20isn't=20being=20written=20to=20now,=20parse=20= the=20log=20and=20check=0A+=09=09#=20for=20instances=20of=20checksum=20= verification=20failures.=0A+=09=09my=20$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node->logfile,=0A+=09=09=09= $node_loglocation);=0A+=09=09unlike(=0A+=09=09=09$log,=0A+=09=09=09= qr/page=20verification=20failed,.+\d$/,=0A+=09=09=09"no=20checksum=20= validation=20errors=20in=20primary=20log=20(during=20WAL=20recovery)"=0A= +=09=09);=0A+=09=09$node_loglocation=20=3D=20-s=20$node->logfile;=0A+=0A= +=09=09#=20Randomize=20the=20WAL=20size,=20to=20trigger=20checkpoints=20= less/more=20often=0A+=09=09my=20$sb=20=3D=2064=20+=20int(rand(1024));=0A= +=09=09$node->append_conf('postgresql.conf',=20qq[max_wal_size=20=3D=20= $sb]);=0A+=09=09note("changing=20max_wal_size=20to=20"=20.=20$sb);=0A+=0A= +=09=09$node->start;=0A+=0A+=09=09#=20Start=20a=20pgbench=20in=20the=20= background=20against=20the=20primary=0A+=09=09= background_rw_pgbench($node->port);=0A+=09}=0A+=0A+=09= $node->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;");=0A+=0A+=09flip_data_checksums();=0A+=09random_sleep()=20if=20= ($extended);=0A+=09my=20$result=20=3D=0A+=09=20=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+=09is($result,=20'100000',=20'ensure=20data=20pages=20can=20= be=20read=20back=20on=20primary');=0A+=0A+=09random_sleep()=20if=20= ($extended);=0A+=0A+=09#=20Potentially=20powercycle=20the=20node=0A+=09= if=20(cointoss())=0A+=09{=0A+=09=09$node->stop(stopmode());=0A+=0A+=09=09= PostgreSQL::Test::Utils::system_log("pg_controldata",=0A+=09=09=09= $node->data_dir);=0A+=0A+=09=09my=20$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node->logfile,=0A+=09=09=09= $node_loglocation);=0A+=09=09unlike(=0A+=09=09=09$log,=0A+=09=09=09= qr/page=20verification=20failed,.+\d$/,=0A+=09=09=09"no=20checksum=20= validation=20errors=20in=20primary=20log=20(outside=20WAL=20recovery)"=0A= +=09=09);=0A+=09=09$node_loglocation=20=3D=20-s=20$node->logfile;=0A+=09= }=0A+=0A+=09random_sleep()=20if=20($extended);=0A+}=0A+=0A+#=20Make=20= sure=20the=20node=20is=20running=0A+if=20(!$node->is_alive)=0A+{=0A+=09= $node->start;=0A+}=0A+=0A+#=20Testrun=20is=20over,=20ensure=20that=20= data=20reads=20back=20as=20expected=20and=20perform=20a=20final=0A+#=20= verification=20of=20the=20data=20checksum=20state.=0A+my=20$result=20=3D=0A= +=20=20$node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20= WHERE=20a=20>=201");=0A+is($result,=20'100000',=20'ensure=20data=20pages=20= can=20be=20read=20back=20on=20primary');=0A+test_checksum_state($node,=20= $data_checksum_state);=0A+=0A+#=20Perform=20one=20final=20pass=20over=20= the=20logs=20and=20hunt=20for=20unexpected=20errors=0A+my=20$log=20=3D=0A= +=20=20PostgreSQL::Test::Utils::slurp_file($node->logfile,=20= $node_loglocation);=0A+unlike(=0A+=09$log,=0A+=09qr/page=20verification=20= failed,.+\d$/,=0A+=09"no=20checksum=20validation=20errors=20in=20primary=20= log");=0A+$node_loglocation=20=3D=20-s=20$node->logfile;=0A+=0A= +$node->teardown_node;=0A+=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/007_pgbench_standby.pl=20= b/src/test/modules/test_checksums/t/007_pgbench_standby.pl=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..b0d40d24005=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/007_pgbench_standby.pl=0A@@=20-0,0=20= +1,400=20@@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster,=0A+#=20comprising=20of=20= a=20primary=20and=20a=20replicated=20standby,=20with=20concurrent=20= activity=0A+#=20via=20pgbench=20runs=0A+=0A+use=20strict;=0A+use=20= warnings=20FATAL=20=3D>=20'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A= +use=20PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20= FindBin;=0A+use=20lib=20$FindBin::RealBin;=0A+=0A+use=20= DataChecksums::Utils;=0A+=0A+#=20This=20test=20suite=20is=20expensive,=20= or=20very=20expensive,=20to=20execute.=20=20There=20are=20two=0A+#=20= PG_TEST_EXTRA=20options=20for=20running=20it,=20"checksum"=20for=20a=20= pared-down=20test=20suite=0A+#=20an=20"checksum_extended"=20for=20the=20= full=20suite.=20=20The=20full=20suite=20can=20run=20for=20hours=0A+#=20= on=20slow=20or=20constrained=20systems.=0A+my=20$extended=20=3D=20undef;=0A= +if=20($ENV{PG_TEST_EXTRA})=0A+{=0A+=09$extended=20=3D=201=20if=20= ($ENV{PG_TEST_EXTRA}=20=3D~=20/\bchecksum_extended\b/);=0A+=09plan=20= skip_all=20=3D>=20'Expensive=20data=20checksums=20test=20disabled'=0A+=09= =20=20unless=20($ENV{PG_TEST_EXTRA}=20=3D~=20= /\bchecksum(_extended)?\b/);=0A+}=0A+else=0A+{=0A+=09plan=20skip_all=20= =3D>=20'Expensive=20data=20checksums=20test=20disabled';=0A+}=0A+=0A+if=20= ($ENV{enable_injection_points}=20ne=20'yes')=0A+{=0A+=09plan=20skip_all=20= =3D>=20'Injection=20points=20not=20supported=20by=20this=20build';=0A+}=0A= +=0A+my=20$node_primary_slot=20=3D=20'physical_slot';=0A+my=20= $node_primary_backup=20=3D=20'primary_backup';=0A+my=20$node_primary;=0A= +my=20$node_primary_loglocation=20=3D=200;=0A+my=20$node_standby;=0A+my=20= $node_standby_loglocation=20=3D=200;=0A+=0A+#=20The=20number=20of=20full=20= test=20iterations=20which=20will=20be=20performed.=20The=20exact=20= number=0A+#=20of=20tests=20performed=20and=20the=20wall=20time=20taken=20= is=20non-deterministic=20as=20the=20test=0A+#=20performs=20a=20lot=20of=20= randomized=20actions,=20but=205=20iterations=20will=20be=20a=20long=20= test=0A+#=20run=20regardless.=0A+my=20$TEST_ITERATIONS=20=3D=205;=0A= +$TEST_ITERATIONS=20=3D=201=20if=20($extended);=0A+=0A+#=20Variables=20= which=20record=20the=20current=20state=20of=20the=20cluster=0A+my=20= $data_checksum_state=20=3D=20'off';=0A+=0A+my=20$pgbench_primary=20=3D=20= undef;=0A+my=20$pgbench_standby=20=3D=20undef;=0A+=0A+#=20Start=20a=20= pgbench=20run=20in=20the=20background=20against=20the=20server=20= specified=20via=20the=0A+#=20port=20passed=20as=20parameter=0A+sub=20= background_pgbench=0A+{=0A+=09my=20($port,=20$standby)=20=3D=20@_;=0A+=09= my=20$pgbench=20=3D=20($standby=20?=20\$pgbench_standby=20:=20= \$pgbench_primary);=0A+=0A+=09#=20Terminate=20any=20currently=20running=20= pgbench=20process=20before=20continuing=0A+=09$$pgbench->finish=20if=20= $$pgbench;=0A+=0A+=09my=20$clients=20=3D=201;=0A+=09my=20$runtime=20=3D=20= 5;=0A+=0A+=09if=20($extended)=0A+=09{=0A+=09=09#=20Randomize=20the=20= number=20of=20pgbench=20clients=20a=20bit=20(range=201-16)=0A+=09=09= $clients=20=3D=201=20+=20int(rand(15));=0A+=09=09$runtime=20=3D=20600;=0A= +=09}=0A+=0A+=09my=20@cmd=20=3D=20('pgbench',=20'-p',=20$port,=20'-T',=20= $runtime,=20'-c',=20$clients);=0A+=09#=20Randomize=20whether=20we=20= spawn=20connections=20or=20not=0A+=09push(@cmd,=20'-C')=20if=20= ($extended=20&&=20cointoss());=0A+=09#=20If=20we=20run=20on=20a=20= standby=20it=20needs=20to=20be=20a=20read-only=20benchmark=0A+=09= push(@cmd,=20'-S')=20if=20($standby);=0A+=09#=20Finally=20add=20the=20= database=20name=20to=20use=0A+=09push(@cmd,=20'postgres');=0A+=0A+=09= $$pgbench=20=3D=20IPC::Run::start(=0A+=09=09\@cmd,=0A+=09=09'<'=20=3D>=20= '/dev/null',=0A+=09=09'>'=20=3D>=20'/dev/null',=0A+=09=09'2>'=20=3D>=20= '/dev/null',=0A+=09=09= IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default));=0A+}=0A+=0A= +#=20Invert=20the=20state=20of=20data=20checksums=20in=20the=20cluster,=20= if=20data=20checksums=20are=20on=0A+#=20then=20disable=20them=20and=20= vice=20versa.=20Also=20performs=20proper=20validation=20of=20the=0A+#=20= before=20and=20after=20state.=0A+sub=20flip_data_checksums=0A+{=0A+=09#=20= First,=20make=20sure=20the=20cluster=20is=20in=20the=20state=20we=20= expect=20it=20to=20be=0A+=09test_checksum_state($node_primary,=20= $data_checksum_state);=0A+=09test_checksum_state($node_standby,=20= $data_checksum_state);=0A+=0A+=09if=20($data_checksum_state=20eq=20= 'off')=0A+=09{=0A+=09=09#=20Coin-toss=20to=20see=20if=20we=20are=20= injecting=20a=20retry=20due=20to=20a=20temptable=0A+=09=09= $node_primary->safe_psql('postgres',=20'SELECT=20dcw_fake_temptable();')=0A= +=09=09=20=20if=20cointoss();=0A+=0A+=09=09#=20log=20LSN=20right=20= before=20we=20start=20changing=20checksums=0A+=09=09my=20$result=20=3D=0A= +=09=09=20=20$node_primary->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=09=09note("LSN=20before=20enabling:=20"=20.=20= $result=20.=20"\n");=0A+=0A+=09=09#=20Ensure=20that=20the=20primary=20= switches=20to=20"inprogress-on"=0A+=09=09= enable_data_checksums($node_primary,=20wait=20=3D>=20'inprogress-on');=0A= +=0A+=09=09random_sleep()=20if=20($extended);=0A+=0A+=09=09#=20Wait=20= for=20checksum=20enable=20to=20be=20replayed=0A+=09=09= $node_primary->wait_for_catchup($node_standby,=20'replay');=0A+=0A+=09=09= #=20Ensure=20that=20the=20standby=20has=20switched=20to=20= "inprogress-on"=20or=20"on".=0A+=09=09#=20Normally=20it=20would=20be=20= "inprogress-on",=20but=20it=20is=20theoretically=0A+=09=09#=20possible=20= for=20the=20primary=20to=20complete=20the=20checksum=20enabling=20*and*=20= have=0A+=09=09#=20the=20standby=20replay=20that=20record=20before=20we=20= reach=20the=20check=20below.=0A+=09=09$result=20=3D=20= $node_standby->poll_query_until(=0A+=09=09=09'postgres',=0A+=09=09=09= "SELECT=20setting=20=3D=20'off'=20"=0A+=09=09=09=20=20.=20"FROM=20= pg_catalog.pg_settings=20"=0A+=09=09=09=20=20.=20"WHERE=20name=20=3D=20= 'data_checksums';",=0A+=09=09=09'f');=0A+=09=09is($result,=201,=0A+=09=09= =09'ensure=20standby=20has=20absorbed=20the=20inprogress-on=20barrier');=0A= +=09=09$result=20=3D=20$node_standby->safe_psql('postgres',=0A+=09=09=09=09= "SELECT=20setting=20"=0A+=09=09=09=20=20.=20"FROM=20= pg_catalog.pg_settings=20"=0A+=09=09=09=20=20.=20"WHERE=20name=20=3D=20= 'data_checksums';");=0A+=0A+=09=09is(($result=20eq=20'inprogress-on'=20= ||=20$result=20eq=20'on'),=0A+=09=09=091,=20'ensure=20checksums=20are=20= on,=20or=20in=20progress,=20on=20standby_1');=0A+=0A+=09=09#=20Wait=20= for=20checksums=20enabled=20on=20the=20primary=20and=20standby=0A+=09=09= wait_for_checksum_state($node_primary,=20'on');=0A+=0A+=09=09#=20log=20= LSN=20right=20after=20the=20primary=20flips=20checksums=20to=20"on"=0A+=09= =09$result=20=3D=0A+=09=09=20=20$node_primary->safe_psql('postgres',=20= "SELECT=20pg_current_wal_lsn()");=0A+=09=09note("LSN=20after=20enabling:=20= "=20.=20$result=20.=20"\n");=0A+=0A+=09=09random_sleep()=20if=20= ($extended);=0A+=09=09wait_for_checksum_state($node_standby,=20'on');=0A= +=0A+=09=09$node_primary->safe_psql('postgres',=0A+=09=09=09'SELECT=20= dcw_fake_temptable(false);');=0A+=09=09$data_checksum_state=20=3D=20= 'on';=0A+=09}=0A+=09elsif=20($data_checksum_state=20eq=20'on')=0A+=09{=0A= +=09=09random_sleep()=20if=20($extended);=0A+=0A+=09=09#=20log=20LSN=20= right=20before=20we=20start=20changing=20checksums=0A+=09=09my=20$result=20= =3D=0A+=09=09=20=20$node_primary->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=09=09note("LSN=20before=20disabling:=20"=20.=20= $result=20.=20"\n");=0A+=0A+=09=09disable_data_checksums($node_primary);=0A= +=09=09$node_primary->wait_for_catchup($node_standby,=20'replay');=0A+=0A= +=09=09#=20Wait=20for=20checksums=20disabled=20on=20the=20primary=20and=20= standby=0A+=09=09wait_for_checksum_state($node_primary,=20'off');=0A+=09=09= wait_for_checksum_state($node_standby,=20'off');=0A+=0A+=09=09#=20log=20= LSN=20right=20after=20the=20primary=20flips=20checksums=20to=20"off"=0A+=09= =09$result=20=3D=0A+=09=09=20=20$node_primary->safe_psql('postgres',=20= "SELECT=20pg_current_wal_lsn()");=0A+=09=09note("LSN=20after=20= disabling:=20"=20.=20$result=20.=20"\n");=0A+=0A+=09=09random_sleep()=20= if=20($extended);=0A+=09=09wait_for_checksum_state($node_standby,=20= 'off');=0A+=0A+=09=09$data_checksum_state=20=3D=20'off';=0A+=09}=0A+=09= else=0A+=09{=0A+=09=09#=20This=20should=20only=20happen=20due=20to=20= programmer=20error=20when=20hacking=20on=20the=0A+=09=09#=20test=20code,=20= but=20since=20that=20might=20pass=20subtly=20we=20error=20out.=0A+=09=09= BAIL_OUT('data_checksum_state=20variable=20has=20invalid=20state:'=0A+=09= =09=09=20=20.=20$data_checksum_state);=0A+=09}=0A+}=0A+=0A+#=20Create=20= and=20start=20a=20cluster=20with=20one=20primary=20and=20one=20standby=20= node,=20and=20ensure=0A+#=20they=20are=20caught=20up=20and=20in=20sync.=0A= +$node_primary=20=3D=20= PostgreSQL::Test::Cluster->new('pgbench_standby_main');=0A= +$node_primary->init(allows_streaming=20=3D>=201,=20no_data_checksums=20= =3D>=201);=0A+#=20max_connections=20need=20to=20be=20bumped=20in=20order=20= to=20accommodate=20for=20pgbench=20clients=0A+#=20and=20log_statement=20= is=20dialled=20down=20since=20it=20otherwise=20will=20generate=20= enormous=0A+#=20amounts=20of=20logging.=20Page=20verification=20failures=20= are=20still=20logged.=0A+$node_primary->append_conf(=0A+=09= 'postgresql.conf',=0A+=09qq[=0A+max_connections=20=3D=2030=0A= +log_statement=20=3D=20none=0A+]);=0A+$node_primary->start;=0A= +$node_primary->safe_psql('postgres',=20'CREATE=20EXTENSION=20= test_checksums;');=0A+#=20Create=20some=20content=20to=20have=20= un-checksummed=20data=20in=20the=20cluster=0A= +$node_primary->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,=20100000)=20AS=20a;");=0A= +$node_primary->safe_psql('postgres',=0A+=09"SELECT=20= pg_create_physical_replication_slot('$node_primary_slot');");=0A= +$node_primary->backup($node_primary_backup);=0A+=0A+$node_standby=20=3D=20= PostgreSQL::Test::Cluster->new('pgbench_standby_standby');=0A= +$node_standby->init_from_backup($node_primary,=20$node_primary_backup,=0A= +=09has_streaming=20=3D>=201);=0A+$node_standby->append_conf(=0A+=09= 'postgresql.conf',=20qq[=0A+primary_slot_name=20=3D=20= '$node_primary_slot'=0A+]);=0A+$node_standby->start;=0A+=0A+#=20= Initialize=20pgbench=20and=20wait=20for=20the=20objects=20to=20be=20= created=20on=20the=20standby=0A+my=20$scalefactor=20=3D=20($extended=20?=20= 10=20:=201);=0A+$node_primary->command_ok(=0A+=09[=0A+=09=09'pgbench',=20= '-p',=20$node_primary->port,=20'-i',=20'-s',=20$scalefactor,=20'-q',=0A+=09= =09'postgres'=0A+=09]);=0A= +$node_primary->wait_for_catchup($node_standby,=20'replay');=0A+=0A+#=20= Start=20the=20test=20suite=20with=20pgbench=20running=20on=20all=20nodes=0A= +background_pgbench($node_standby->port,=201);=0A= +background_pgbench($node_primary->port,=200);=0A+=0A+#=20Main=20test=20= suite.=20This=20loop=20will=20start=20a=20pgbench=20run=20on=20the=20= cluster=20and=20while=0A+#=20that's=20running=20flip=20the=20state=20of=20= data=20checksums=20concurrently.=20It=20will=20then=0A+#=20randomly=20= restart=20the=20cluster=20and=20then=20check=20for=0A+#=20the=20desired=20= state.=20=20The=20idea=20behind=20doing=20things=20randomly=20is=20to=20= stress=20out=0A+#=20any=20timing=20related=20issues=20by=20subjecting=20= the=20cluster=20for=20varied=20workloads.=0A+#=20A=20TODO=20is=20to=20= generate=20a=20trace=20such=20that=20any=20test=20failure=20can=20be=20= traced=20to=0A+#=20its=20order=20of=20operations=20for=20debugging.=0A= +for=20(my=20$i=20=3D=200;=20$i=20<=20$TEST_ITERATIONS;=20$i++)=0A+{=0A+=09= note("iteration=20",=20($i=20+=201),=20"=20of=20",=20$TEST_ITERATIONS);=0A= +=0A+=09if=20(!$node_primary->is_alive)=0A+=09{=0A+=09=09#=20start,=20to=20= do=20recovery,=20and=20stop=0A+=09=09$node_primary->start;=0A+=09=09= $node_primary->stop('fast');=0A+=0A+=09=09#=20Since=20the=20log=20isn't=20= being=20written=20to=20now,=20parse=20the=20log=20and=20check=0A+=09=09#=20= for=20instances=20of=20checksum=20verification=20failures.=0A+=09=09my=20= $log=20=3D=20PostgreSQL::Test::Utils::slurp_file($node_primary->logfile,=0A= +=09=09=09$node_primary_loglocation);=0A+=09=09unlike(=0A+=09=09=09$log,=0A= +=09=09=09qr/page=20verification=20failed,.+\d$/,=0A+=09=09=09"no=20= checksum=20validation=20errors=20in=20primary=20log=20(during=20WAL=20= recovery)"=0A+=09=09);=0A+=09=09$node_primary_loglocation=20=3D=20-s=20= $node_primary->logfile;=0A+=0A+=09=09#=20randomize=20the=20WAL=20size,=20= to=20trigger=20checkpoints=20less/more=20often=0A+=09=09my=20$sb=20=3D=20= 32=20+=20int(rand(960));=0A+=09=09= $node_primary->append_conf('postgresql.conf',=20qq[max_wal_size=20=3D=20= $sb]);=0A+=0A+=09=09note("changing=20primary=20max_wal_size=20to=20"=20.=20= $sb);=0A+=0A+=09=09$node_primary->start;=0A+=0A+=09=09#=20Start=20a=20= pgbench=20in=20the=20background=20against=20the=20primary=0A+=09=09= background_pgbench($node_primary->port,=200);=0A+=09}=0A+=0A+=09if=20= (!$node_standby->is_alive)=0A+=09{=0A+=09=09$node_standby->start;=0A+=09=09= $node_standby->stop('fast');=0A+=0A+=09=09#=20Since=20the=20log=20isn't=20= being=20written=20to=20now,=20parse=20the=20log=20and=20check=0A+=09=09#=20= for=20instances=20of=20checksum=20verification=20failures.=0A+=09=09my=20= $log=20=3D=0A+=09=09=20=20= PostgreSQL::Test::Utils::slurp_file($node_standby->logfile,=0A+=09=09=09= $node_standby_loglocation);=0A+=09=09unlike(=0A+=09=09=09$log,=0A+=09=09=09= qr/page=20verification=20failed,.+\d$/,=0A+=09=09=09"no=20checksum=20= validation=20errors=20in=20standby_1=20log=20(during=20WAL=20recovery)"=0A= +=09=09);=0A+=09=09$node_standby_loglocation=20=3D=20-s=20= $node_standby->logfile;=0A+=0A+=09=09#=20randomize=20the=20WAL=20size,=20= to=20trigger=20checkpoints=20less/more=20often=0A+=09=09my=20$sb=20=3D=20= 32=20+=20int(rand(960));=0A+=09=09= $node_standby->append_conf('postgresql.conf',=20qq[max_wal_size=20=3D=20= $sb]);=0A+=0A+=09=09note("changing=20standby=20max_wal_size=20to=20"=20.=20= $sb);=0A+=0A+=09=09$node_standby->start;=0A+=0A+=09=09#=20Start=20a=20= read-only=20pgbench=20in=20the=20background=20on=20the=20standby=0A+=09=09= background_pgbench($node_standby->port,=201);=0A+=09}=0A+=0A+=09= $node_primary->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20= +=201;");=0A+=09$node_primary->wait_for_catchup($node_standby,=20= 'write');=0A+=0A+=09flip_data_checksums();=0A+=09random_sleep()=20if=20= ($extended);=0A+=09my=20$result=20=3D=20= $node_primary->safe_psql('postgres',=0A+=09=09"SELECT=20count(*)=20FROM=20= t=20WHERE=20a=20>=201");=0A+=09is($result,=20'100000',=20'ensure=20data=20= pages=20can=20be=20read=20back=20on=20primary');=0A+=09random_sleep();=0A= +=0A+=09#=20Potentially=20powercycle=20the=20cluster=20(the=20nodes=20= independently).=20A=20TODO=20is=0A+=09#=20to=20randomly=20stop=20the=20= nodes=20in=20the=20opposite=20order=20too.=0A+=09if=20($extended=20&&=20= cointoss())=0A+=09{=0A+=09=09$node_primary->stop(stopmode());=0A+=0A+=09=09= #=20print=20the=20contents=20of=20the=20control=20file=20on=20the=20= primary=0A+=09=09PostgreSQL::Test::Utils::system_log("pg_controldata",=0A= +=09=09=09$node_primary->data_dir);=0A+=0A+=09=09#=20slurp=20the=20file=20= after=20shutdown,=20so=20that=20it=20doesn't=20interfere=20with=20the=20= recovery=0A+=09=09my=20$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node_primary->logfile,=0A+=09=09=09= $node_primary_loglocation);=0A+=09=09unlike(=0A+=09=09=09$log,=0A+=09=09=09= qr/page=20verification=20failed,.+\d$/,=0A+=09=09=09"no=20checksum=20= validation=20errors=20in=20primary=20log=20(outside=20WAL=20recovery)"=0A= +=09=09);=0A+=09=09$node_primary_loglocation=20=3D=20-s=20= $node_primary->logfile;=0A+=09}=0A+=0A+=09random_sleep()=20if=20= ($extended);=0A+=0A+=09if=20($extended=20&&=20cointoss())=0A+=09{=0A+=09=09= $node_standby->stop(stopmode());=0A+=0A+=09=09#=20print=20the=20contents=20= of=20the=20control=20file=20on=20the=20standby=0A+=09=09= PostgreSQL::Test::Utils::system_log("pg_controldata",=0A+=09=09=09= $node_standby->data_dir);=0A+=0A+=09=09#=20slurp=20the=20file=20after=20= shutdown,=20so=20that=20it=20doesn't=20interfere=20with=20the=20recovery=0A= +=09=09my=20$log=20=3D=0A+=09=09=20=20= PostgreSQL::Test::Utils::slurp_file($node_standby->logfile,=0A+=09=09=09= $node_standby_loglocation);=0A+=09=09unlike(=0A+=09=09=09$log,=0A+=09=09=09= qr/page=20verification=20failed,.+\d$/,=0A+=09=09=09"no=20checksum=20= validation=20errors=20in=20standby_1=20log=20(outside=20WAL=20recovery)"=0A= +=09=09);=0A+=09=09$node_standby_loglocation=20=3D=20-s=20= $node_standby->logfile;=0A+=09}=0A+}=0A+=0A+#=20make=20sure=20the=20= nodes=20are=20running=0A+if=20(!$node_primary->is_alive)=0A+{=0A+=09= $node_primary->start;=0A+}=0A+=0A+if=20(!$node_standby->is_alive)=0A+{=0A= +=09$node_standby->start;=0A+}=0A+=0A+#=20Testrun=20is=20over,=20ensure=20= that=20data=20reads=20back=20as=20expected=20and=20perform=20a=20final=0A= +#=20verification=20of=20the=20data=20checksum=20state.=0A+my=20$result=20= =3D=0A+=20=20$node_primary->safe_psql('postgres',=20"SELECT=20count(*)=20= FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20'100000',=20'ensure=20= data=20pages=20can=20be=20read=20back=20on=20primary');=0A= +test_checksum_state($node_primary,=20$data_checksum_state);=0A= +test_checksum_state($node_standby,=20$data_checksum_state);=0A+=0A+#=20= Perform=20one=20final=20pass=20over=20the=20logs=20and=20hunt=20for=20= unexpected=20errors=0A+my=20$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node_primary->logfile,=0A+=09= $node_primary_loglocation);=0A+unlike(=0A+=09$log,=0A+=09qr/page=20= verification=20failed,.+\d$/,=0A+=09"no=20checksum=20validation=20errors=20= in=20primary=20log");=0A+$node_primary_loglocation=20=3D=20-s=20= $node_primary->logfile;=0A+$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node_standby->logfile,=0A+=09= $node_standby_loglocation);=0A+unlike(=0A+=09$log,=0A+=09qr/page=20= verification=20failed,.+\d$/,=0A+=09"no=20checksum=20validation=20errors=20= in=20standby_1=20log");=0A+$node_standby_loglocation=20=3D=20-s=20= $node_standby->logfile;=0A+=0A+$node_standby->teardown_node;=0A= +$node_primary->teardown_node;=0A+=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/008_pitr.pl=20= b/src/test/modules/test_checksums/t/008_pitr.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..b9b89f414ab=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/008_pitr.pl=0A@@=20-0,0=20+1,189=20= @@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20= 'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A+use=20= PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20FindBin;=0A= +use=20lib=20$FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A= +#=20This=20test=20suite=20is=20expensive,=20or=20very=20expensive,=20to=20= execute.=20=20There=20are=20two=0A+#=20PG_TEST_EXTRA=20options=20for=20= running=20it,=20"checksum"=20for=20a=20pared-down=20test=20suite=0A+#=20= an=20"checksum_extended"=20for=20the=20full=20suite.=0A+my=20$extended=20= =3D=20undef;=0A+if=20($ENV{PG_TEST_EXTRA})=0A+{=0A+=09$extended=20=3D=20= 1=20if=20($ENV{PG_TEST_EXTRA}=20=3D~=20/\bchecksum_extended\b/);=0A+=09= plan=20skip_all=20=3D>=20'Expensive=20data=20checksums=20test=20= disabled'=0A+=09=20=20unless=20($ENV{PG_TEST_EXTRA}=20=3D~=20= /\bchecksum(_extended)?\b/);=0A+}=0A+else=0A+{=0A+=09plan=20skip_all=20= =3D>=20'Expensive=20data=20checksums=20test=20disabled';=0A+}=0A+=0A+=0A= +my=20$pgbench=20=3D=20undef;=0A+my=20$data_checksum_state=20=3D=20= 'off';=0A+=0A+my=20$node_primary;=0A+=0A+#=20Invert=20the=20state=20of=20= data=20checksums=20in=20the=20cluster,=20if=20data=20checksums=20are=20= on=0A+#=20then=20disable=20them=20and=20vice=20versa.=20Also=20performs=20= proper=20validation=20of=20the=0A+#=20before=20and=20after=20state.=0A= +sub=20flip_data_checksums=0A+{=0A+=09my=20$lsn_pre=20=3D=20undef;=0A+=09= my=20$lsn_post=20=3D=20undef;=0A+=0A+=09#=20First,=20make=20sure=20the=20= cluster=20is=20in=20the=20state=20we=20expect=20it=20to=20be=0A+=09= test_checksum_state($node_primary,=20$data_checksum_state);=0A+=0A+=09if=20= ($data_checksum_state=20eq=20'off')=0A+=09{=0A+=09=09#=20log=20LSN=20= right=20before=20we=20start=20changing=20checksums=0A+=09=09$lsn_pre=20=3D= =0A+=09=09=20=20$node_primary->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=09=09note("LSN=20before=20enabling:=20"=20.=20= $lsn_pre=20.=20"\n");=0A+=0A+=09=09#=20Wait=20for=20checksums=20enabled=20= on=20the=20primary=0A+=09=09enable_data_checksums($node_primary,=20wait=20= =3D>=20'on');=0A+=0A+=09=09#=20log=20LSN=20right=20after=20the=20primary=20= flips=20checksums=20to=20"on"=0A+=09=09$lsn_post=20=3D=0A+=09=09=20=20= $node_primary->safe_psql('postgres',=20"SELECT=20pg_current_wal_lsn()");=0A= +=09=09note("LSN=20after=20enabling:=20"=20.=20$lsn_post=20.=20"\n");=0A= +=0A+=09=09$data_checksum_state=20=3D=20'on';=0A+=09}=0A+=09elsif=20= ($data_checksum_state=20eq=20'on')=0A+=09{=0A+=09=09#=20log=20LSN=20= right=20before=20we=20start=20changing=20checksums=0A+=09=09$lsn_pre=20=3D= =0A+=09=09=20=20$node_primary->safe_psql('postgres',=20"SELECT=20= pg_current_wal_lsn()");=0A+=0A+=09=09= disable_data_checksums($node_primary);=0A+=0A+=09=09#=20Wait=20for=20= checksums=20disabled=20on=20the=20primary=0A+=09=09= wait_for_checksum_state($node_primary,=20'off');=0A+=0A+=09=09#=20log=20= LSN=20right=20after=20the=20primary=20flips=20checksums=20to=20"off"=0A+=09= =09$lsn_post=20=3D=0A+=09=09=20=20$node_primary->safe_psql('postgres',=20= "SELECT=20pg_current_wal_lsn()");=0A+=0A+=09=09$data_checksum_state=20=3D=20= 'off';=0A+=09}=0A+=09else=0A+=09{=0A+=09=09#=20This=20should=20only=20= happen=20due=20to=20programmer=20error=20when=20hacking=20on=20the=0A+=09= =09#=20test=20code,=20but=20since=20that=20might=20pass=20subtly=20we=20= error=20out.=0A+=09=09BAIL_OUT('data_checksum_state=20variable=20has=20= invalid=20state:'=0A+=09=09=09=20=20.=20$data_checksum_state);=0A+=09}=0A= +=0A+=09return=20($lsn_pre,=20$lsn_post);=0A+}=0A+#=20Start=20a=20= pgbench=20run=20in=20the=20background=20against=20the=20server=20= specified=20via=20the=0A+#=20port=20passed=20as=20parameter.=0A+sub=20= background_rw_pgbench=0A+{=0A+=09my=20$port=20=3D=20shift;=0A+=0A+=09#=20= If=20a=20previous=20pgbench=20is=20still=20running,=20start=20by=20= shutting=20it=20down.=0A+=09$pgbench->finish=20if=20$pgbench;=0A+=0A+=09= #=20Randomize=20the=20number=20of=20pgbench=20clients=20in=20extended=20= mode,=20else=201=20client=0A+=09my=20$clients=20=3D=20($extended=20?=201=20= +=20int(rand(15))=20:=201);=0A+=09my=20$runtime=20=3D=20($extended=20?=20= 600=20:=205);=0A+=0A+=09my=20@cmd=20=3D=20('pgbench',=20'-p',=20$port,=20= '-T',=20$runtime,=20'-c',=20$clients);=0A+=0A+=09#=20Randomize=20whether=20= we=20spawn=20connections=20or=20not=0A+=09push(@cmd,=20'-C')=20if=20= ($extended=20&&=20cointoss());=0A+=09#=20Finally=20add=20the=20database=20= name=20to=20use=0A+=09push(@cmd,=20'postgres');=0A+=0A+=09$pgbench=20=3D=20= IPC::Run::start(=0A+=09=09\@cmd,=0A+=09=09'<'=20=3D>=20'/dev/null',=0A+=09= =09'>'=20=3D>=20'/dev/null',=0A+=09=09'2>'=20=3D>=20'/dev/null',=0A+=09=09= IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default));=0A+}=0A+=0A= +#=20Start=20a=20primary=20node=20with=20WAL=20archiving=20enabled=20and=20= with=20enough=20connections=0A+#=20available=20to=20handle=20pgbench=20= clients.=0A+$node_primary=20=3D=20= PostgreSQL::Test::Cluster->new('pitr_main');=0A+$node_primary->init(=0A+=09= has_archiving=20=3D>=201,=0A+=09allows_streaming=20=3D>=201,=0A+=09= no_data_checksums=20=3D>=201);=0A+$node_primary->append_conf(=0A+=09= 'postgresql.conf',=0A+=09qq[=0A+max_connections=20=3D=20100=0A= +log_statement=20=3D=20none=0A+]);=0A+$node_primary->start;=0A+=0A+#=20= Prime=20the=20cluster=20with=20a=20bit=20of=20known=20data=20which=20we=20= can=20read=20back=20to=20check=0A+#=20for=20data=20consistency=20as=20= well=20as=20page=20verification=20faults=20in=20the=20logfile.=0A= +$node_primary->safe_psql('postgres',=0A+=09'CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,=20100000)=20AS=20a;');=0A+#=20Initialize=20= and=20start=20pgbench=20in=20read/write=20mode=20against=20the=20cluster=0A= +my=20$scalefactor=20=3D=20($extended=20?=2010=20:=201);=0A= +$node_primary->command_ok(=0A+=09[=0A+=09=09'pgbench',=20'-p',=20= $node_primary->port,=20'-i',=20'-s',=20$scalefactor,=20'-q',=0A+=09=09= 'postgres'=0A+=09]);=0A+background_rw_pgbench($node_primary->port);=0A+=0A= +#=20Take=20a=20backup=20to=20use=20for=20PITR=0A+my=20$backup_name=20=3D=20= 'my_backup';=0A+$node_primary->backup($backup_name);=0A+=0A+my=20= ($pre_lsn,=20$post_lsn)=20=3D=20flip_data_checksums();=0A+=0A= +$node_primary->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20= +=201;");=0A+$node_primary->safe_psql('postgres',=20"SELECT=20= pg_create_restore_point('a');");=0A+$node_primary->safe_psql('postgres',=20= "UPDATE=20t=20SET=20a=20=3D=20a=20+=201;");=0A= +$node_primary->stop('immediate');=0A+=0A+my=20$node_pitr=20=3D=20= PostgreSQL::Test::Cluster->new('pitr_backup');=0A= +$node_pitr->init_from_backup(=0A+=09$node_primary,=20$backup_name,=0A+=09= standby=20=3D>=200,=0A+=09has_restoring=20=3D>=201);=0A= +$node_pitr->append_conf(=0A+=09'postgresql.conf',=20qq{=0A= +recovery_target_lsn=20=3D=20'$post_lsn'=0A+recovery_target_action=20=3D=20= 'promote'=0A+recovery_target_inclusive=20=3D=20on=0A+});=0A+=0A= +$node_pitr->start;=0A+=0A+$node_pitr->poll_query_until('postgres',=20= "SELECT=20pg_is_in_recovery()=20=3D=20'f';")=0A+=20=20or=20die=20"Timed=20= out=20while=20waiting=20for=20PITR=20promotion";=0A+=0A= +test_checksum_state($node_pitr,=20$data_checksum_state);=0A+my=20= $result=20=3D=0A+=20=20$node_pitr->safe_psql('postgres',=20"SELECT=20= count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20'99999',=20= 'ensure=20data=20pages=20can=20be=20read=20back=20on=20primary');=0A+=0A= +$node_pitr->stop;=0A+=0A+my=20$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node_pitr->logfile,=200);=0A= +unlike(=0A+=09$log,=0A+=09qr/page=20verification=20failed,.+\d$/,=0A+=09= "no=20checksum=20validation=20errors=20in=20pitr=20log");=0A+=0A= +done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/009_fpi.pl=20= b/src/test/modules/test_checksums/t/009_fpi.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..a1cea91f787=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/009_fpi.pl=0A@@=20-0,0=20+1,64=20@@=0A= +=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20Development=20= Group=0A+=0A+use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20'all';=0A+=0A= +use=20PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A= +use=20Test::More;=0A+=0A+use=20FindBin;=0A+use=20lib=20= $FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A+#=20Create=20= and=20start=20a=20cluster=20with=20one=20node=0A+my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('fpi_node');=0A= +$node->init(allows_streaming=20=3D>=201,=20no_data_checksums=20=3D>=20= 1);=0A+#=20max_connections=20need=20to=20be=20bumped=20in=20order=20to=20= accommodate=20for=20pgbench=20clients=0A+#=20and=20log_statement=20is=20= dialled=20down=20since=20it=20otherwise=20will=20generate=20enormous=0A= +#=20amounts=20of=20logging.=20Page=20verification=20failures=20are=20= still=20logged.=0A+$node->append_conf(=0A+=09'postgresql.conf',=0A+=09= qq[=0A+max_connections=20=3D=20100=0A+log_statement=20=3D=20none=0A+]);=0A= +$node->start;=0A+$node->safe_psql('postgres',=20'CREATE=20EXTENSION=20= test_checksums;');=0A+#=20Create=20some=20content=20to=20have=20= un-checksummed=20data=20in=20the=20cluster=0A= +$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20SELECT=20= generate_series(1,=201000000)=20AS=20a;");=0A+=0A+#=20Enable=20data=20= checksums=20and=20wait=20for=20the=20state=20transition=20to=20'on'=0A= +enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A= +$node->safe_psql('postgres',=20'UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;');=0A+=0A+disable_data_checksums($node,=20wait=20=3D>=201);=0A+=0A= +$node->append_conf('postgresql.conf',=20'full_page_writes=20=3D=20= off');=0A+$node->restart;=0A+test_checksum_state($node,=20'off');=0A+=0A= +$node->safe_psql('postgres',=20'UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;');=0A+$node->safe_psql('postgres',=20'DELETE=20FROM=20t=20WHERE=20a=20= <=2010000;');=0A+=0A+$node->adjust_conf('postgresql.conf',=20= 'full_page_writes',=20'on');=0A+$node->restart;=0A= +test_checksum_state($node,=20'off');=0A+=0A= +enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+my=20$result=20= =3D=20$node->safe_psql('postgres',=20'SELECT=20count(*)=20FROM=20t;');=0A= +is($result,=20'990003',=20'Reading=20back=20all=20data=20from=20table=20= t');=0A+=0A+$node->stop;=0A+my=20$log=20=3D=20= PostgreSQL::Test::Utils::slurp_file($node->logfile,=200);=0A+unlike(=0A+=09= $log,=0A+=09qr/page=20verification=20failed,.+\d$/,=0A+=09"no=20checksum=20= validation=20errors=20in=20server=20log");=0A+=0A+done_testing();=0Adiff=20= --git=20a/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=20= b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..9a2269e8a92=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0A@@=20-0,0=20= +1,262=20@@=0A+=0A+#=20Copyright=20(c)=202026,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+=3Dpod=0A+=0A+=3Dhead1=20NAME=0A+=0A= +DataChecksums::Utils=20-=20Utility=20functions=20for=20testing=20data=20= checksums=20in=20a=20running=20cluster=0A+=0A+=3Dhead1=20SYNOPSIS=0A+=0A= +=20=20use=20PostgreSQL::Test::Cluster;=0A+=20=20use=20= DataChecksums::Utils=20qw(=20..=20);=0A+=0A+=20=20#=20Create,=20and=20= start,=20a=20new=20cluster=0A+=20=20my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('primary');=0A+=20=20$node->init;=0A+=20=20= $node->start;=0A+=0A+=20=20test_checksum_state($node,=20'off');=0A+=0A+=20= =20enable_data_checksums($node);=0A+=0A+=20=20= wait_for_checksum_state($node,=20'on');=0A+=0A+=0A+=3Dcut=0A+=0A+package=20= DataChecksums::Utils;=0A+=0A+use=20strict;=0A+use=20warnings=20FATAL=20= =3D>=20'all';=0A+use=20Exporter=20'import';=0A+use=20= PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A+use=20= Test::More;=0A+=0A+our=20@EXPORT=20=3D=20qw(=0A+=20=20cointoss=0A+=20=20= disable_data_checksums=0A+=20=20enable_data_checksums=0A+=20=20= random_sleep=0A+=20=20stopmode=0A+=20=20test_checksum_state=0A+=20=20= wait_for_checksum_state=0A+=20=20wait_for_cluster_crash=0A+);=0A+=0A= +=3Dpod=0A+=0A+=3Dhead1=20METHODS=0A+=0A+=3Dover=0A+=0A+=3Ditem=20= test_checksum_state(node,=20state)=0A+=0A+Test=20that=20the=20current=20= value=20of=20the=20data=20checksum=20GUC=20in=20the=20server=20running=0A= +at=20B=20matches=20B.=20=20If=20the=20values=20differ,=20a=20= test=20failure=20is=20logged.=0A+Returns=20True=20if=20the=20values=20= match,=20otherwise=20False.=0A+=0A+=3Dcut=0A+=0A+sub=20= test_checksum_state=0A+{=0A+=09my=20($postgresnode,=20$state)=20=3D=20= @_;=0A+=0A+=09my=20$result=20=3D=20$postgresnode->safe_psql('postgres',=0A= +=09=09"SELECT=20setting=20FROM=20pg_catalog.pg_settings=20WHERE=20name=20= =3D=20'data_checksums';"=0A+=09);=0A+=09is($result,=20$state,=20'ensure=20= checksums=20are=20set=20to=20'=20.=20$state);=0A+=09return=20$result=20= eq=20$state;=0A+}=0A+=0A+=3Ditem=20wait_for_checksum_state(node,=20= state)=0A+=0A+Test=20the=20value=20of=20the=20data=20checksum=20GUC=20in=20= the=20server=20running=20at=20B=0A+repeatedly=20until=20it=20= matches=20B=20or=20times=20out.=20=20Processing=20will=20run=20= for=0A+$PostgreSQL::Test::Utils::timeout_default=20seconds=20before=20= timing=20out.=20=20If=20the=0A+values=20differ=20when=20the=20process=20= times=20out,=20False=20is=20returned=20and=20a=20test=20failure=0A+is=20= logged,=20otherwise=20True.=0A+=0A+=3Dcut=0A+=0A+sub=20= wait_for_checksum_state=0A+{=0A+=09my=20($postgresnode,=20$state)=20=3D=20= @_;=0A+=0A+=09my=20$res=20=3D=20$postgresnode->poll_query_until(=0A+=09=09= 'postgres',=0A+=09=09"SELECT=20setting=20FROM=20pg_catalog.pg_settings=20= WHERE=20name=20=3D=20'data_checksums';",=0A+=09=09$state);=0A+=09= is($res,=201,=20'ensure=20data=20checksums=20are=20transitioned=20to=20'=20= .=20$state);=0A+=09return=20$res=20=3D=3D=201;=0A+}=0A+=0A+=3Ditem=20= wait_for_cluster_crash(node,=20params)=0A+=0A+Repeatedly=20test=20if=20= the=20cluster=20running=20at=20B=20responds=20to=20connections=0A= +and=20return=20when=20it=20no=20longer=20does=20so,=20or=20when=20it=20= times=20out.=20=20Processing=20will=0A+run=20for=20= $PostgreSQL::Test::Utils::timeout_default=20seconds=20unless=20a=20= timeout=0A+value=20is=20specified=20as=20a=20parameter.=20=20Returns=20= True=20if=20the=20cluster=20crashed,=20else=0A+False=20if=20the=20= process=20timed=20out.=0A+=0A+=3Dover=0A+=0A+=3Ditem=20timeout=0A+=0A= +Approximate=20number=20of=20seconds=20to=20wait=20for=20cluster=20to=20= crash,=20default=20is=0A+$PostgreSQL::Test::Utils::timeout_default.=20=20= There=20are=20no=20real-time=20guarantees=0A+that=20the=20total=20= process=20time=20won't=20exceed=20the=20timeout.=0A+=0A+=3Dback=0A+=0A= +=3Dcut=0A+=0A+sub=20wait_for_cluster_crash=0A+{=0A+=09my=20= $postgresnode=20=3D=20shift;=0A+=09my=20%params=20=3D=20@_;=0A+=09my=20= $crash=20=3D=200;=0A+=0A+=09$params{timeout}=20=3D=20= $PostgreSQL::Test::Utils::timeout_default=0A+=09=20=20unless=20= (defined($params{timeout}));=0A+=0A+=09for=20(my=20$naps=20=3D=200;=20= $naps=20<=20$params{timeout};=20$naps++)=0A+=09{=0A+=09=09if=20= (!$postgresnode->is_alive)=0A+=09=09{=0A+=09=09=09$crash=20=3D=201;=0A+=09= =09=09last;=0A+=09=09}=0A+=09=09sleep(1);=0A+=09}=0A+=0A+=09return=20= $crash=20=3D=3D=201;=0A+}=0A+=0A+=3Ditem=20enable_data_checksums($node,=20= %params)=0A+=0A+Function=20for=20enabling=20data=20checksums=20in=20the=20= cluster=20running=20at=20B.=0A+=0A+=3Dover=0A+=0A+=3Ditem=20= cost_delay=0A+=0A+The=20B=20to=20use=20when=20enabling=20= data=20checksums,=20default=20is=200.=0A+=0A+=3Ditem=20cost_limit=0A+=0A= +The=20B=20to=20use=20when=20enabling=20data=20checksums,=20= default=20is=20100.=0A+=0A+=3Ditem=20wait=0A+=0A+If=20defined,=20the=20= function=20will=20wait=20for=20the=20state=20defined=20in=20this=20= parameter,=0A+waiting=20timing=20out,=20before=20returning.=20=20The=20= function=20will=20wait=20for=0A= +$PostgreSQL::Test::Utils::timeout_default=20seconds=20before=20timing=20= out.=0A+=0A+=3Dback=0A+=0A+=3Dcut=0A+=0A+sub=20enable_data_checksums=0A= +{=0A+=09my=20$postgresnode=20=3D=20shift;=0A+=09my=20%params=20=3D=20= @_;=0A+=0A+=09#=20Set=20sane=20defaults=20for=20the=20parameters=0A+=09= $params{cost_delay}=20=3D=200=20unless=20(defined($params{cost_delay}));=0A= +=09$params{cost_limit}=20=3D=20100=20unless=20= (defined($params{cost_limit}));=0A+=0A+=09my=20$query=20=3D=20<<'EOQ';=0A= +SELECT=20pg_enable_data_checksums(%s,=20%s);=0A+EOQ=0A+=0A+=09= $postgresnode->safe_psql('postgres',=0A+=09=09sprintf($query,=20= $params{cost_delay},=20$params{cost_limit}));=0A+=0A+=09= wait_for_checksum_state($postgresnode,=20$params{wait})=0A+=09=20=20if=20= (defined($params{wait}));=0A+}=0A+=0A+=3Ditem=20= disable_data_checksums($node,=20%params)=0A+=0A+Function=20for=20= disabling=20data=20checksums=20in=20the=20cluster=20running=20at=20= B.=0A+=0A+=3Dover=0A+=0A+=3Ditem=20wait=0A+=0A+If=20defined,=20the=20= function=20will=20wait=20for=20the=20state=20to=20turn=20to=20B,=20= or=0A+waiting=20timing=20out,=20before=20returning.=20=20The=20function=20= will=20wait=20for=0A+$PostgreSQL::Test::Utils::timeout_default=20seconds=20= before=20timing=20out.=0A+Unlike=20in=20C=20the=20= value=20of=20the=20parameter=20is=20discarded.=0A+=0A+=3Dback=0A+=0A= +=3Dcut=0A+=0A+sub=20disable_data_checksums=0A+{=0A+=09my=20= $postgresnode=20=3D=20shift;=0A+=09my=20%params=20=3D=20@_;=0A+=0A+=09= $postgresnode->safe_psql('postgres',=0A+=09=09'SELECT=20= pg_disable_data_checksums();');=0A+=0A+=09= wait_for_checksum_state($postgresnode,=20'off')=20if=20= (defined($params{wait}));=0A+}=0A+=0A+=3Ditem=20cointoss=0A+=0A+Helper=20= for=20retrieving=20a=20binary=20value=20with=20random=20distribution=20= for=20deciding=0A+whether=20to=20turn=20things=20off=20during=20testing.=0A= +=0A+=3Dback=0A+=0A+=3Dcut=0A+=0A+sub=20cointoss=0A+{=0A+=09return=20= int(rand()=20<=200.5);=0A+}=0A+=0A+=3Ditem=20random_sleep(max)=0A+=0A= +Helper=20for=20injecting=20random=20sleeps=20here=20and=20there=20in=20= the=20testrun.=20The=20sleep=0A+duration=20will=20be=20in=20the=20range=20= (0,B),=20but=20won't=20be=20predictable=20in=20order=20to=0A+avoid=20= sleep=20patterns=20that=20manage=20to=20avoid=20race=20conditions=20and=20= timing=20bugs.=0A+The=20default=20B=20is=203=20seconds.=0A+=0A= +=3Dback=0A+=0A+=3Dcut=0A+=0A+sub=20random_sleep=0A+{=0A+=09my=20$max=20= =3D=20shift;=0A+=09return=20if=20(defined($max)=20&&=20($max=20=3D=3D=20= 0));=0A+=09sleep(int(rand(defined($max)=20?=20$max=20:=203)))=20if=20= cointoss;=0A+}=0A+=0A+=3Ditem=20stopmode=0A+=0A+Small=20helper=20= function=20for=20randomly=20selecting=20a=20valid=20stopmode.=0A+=0A= +=3Dback=0A+=0A+=3Dcut=0A+=0A+sub=20stopmode=0A+{=0A+=09return=20= 'immediate'=20if=20(cointoss);=0A+=09return=20'fast';=0A+}=0A+=0A+=3Dpod=0A= +=0A+=3Dback=0A+=0A+=3Dcut=0A+=0A+1;=0Adiff=20--git=20= a/src/test/modules/test_checksums/test_checksums--1.0.sql=20= b/src/test/modules/test_checksums/test_checksums--1.0.sql=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..90642d247fa=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/test_checksums--1.0.sql=0A@@=20-0,0=20= +1,24=20@@=0A+/*=20= src/test/modules/test_checksums/test_checksums--1.0.sql=20*/=0A+=0A+--=20= complain=20if=20script=20is=20sourced=20in=20psql,=20rather=20than=20via=20= CREATE=20EXTENSION=0A+\echo=20Use=20"CREATE=20EXTENSION=20= test_checksums"=20to=20load=20this=20file.=20\quit=0A+=0A+CREATE=20= FUNCTION=20dcw_inject_delay_barrier(attach=20boolean=20DEFAULT=20true)=0A= +=09RETURNS=20pg_catalog.void=0A+=09AS=20'MODULE_PATHNAME'=20LANGUAGE=20= C;=0A+=0A+CREATE=20FUNCTION=20dcw_inject_launcher_delay(attach=20boolean=20= DEFAULT=20true)=0A+=09RETURNS=20pg_catalog.void=0A+=09AS=20= 'MODULE_PATHNAME'=20LANGUAGE=20C;=0A+=0A+CREATE=20FUNCTION=20= dcw_inject_startup_delay(attach=20boolean=20DEFAULT=20true)=0A+=09= RETURNS=20pg_catalog.void=0A+=09AS=20'MODULE_PATHNAME'=20LANGUAGE=20C;=0A= +=0A+CREATE=20FUNCTION=20dcw_inject_fail_database(attach=20boolean=20= DEFAULT=20true)=0A+=09RETURNS=20pg_catalog.void=0A+=09AS=20= 'MODULE_PATHNAME'=20LANGUAGE=20C;=0A+=0A+CREATE=20FUNCTION=20= dcw_fake_temptable(attach=20boolean=20DEFAULT=20true)=0A+=09RETURNS=20= pg_catalog.void=0A+=09AS=20'MODULE_PATHNAME'=20LANGUAGE=20C;=0Adiff=20= --git=20a/src/test/modules/test_checksums/test_checksums.c=20= b/src/test/modules/test_checksums/test_checksums.c=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..b087a3b4664=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/test_checksums.c=0A@@=20-0,0=20+1,184=20= @@=0A= +/*-----------------------------------------------------------------------= ---=0A+=20*=0A+=20*=20test_checksums.c=0A+=20*=09=09Test=20data=20= checksums=0A+=20*=0A+=20*=20Copyright=20(c)=202026,=20PostgreSQL=20= Global=20Development=20Group=0A+=20*=0A+=20*=20IDENTIFICATION=0A+=20*=09=09= src/test/modules/test_checksums/test_checksums.c=0A+=20*=0A+=20*=20= -------------------------------------------------------------------------=0A= +=20*/=0A+#include=20"postgres.h"=0A+=0A+#include=20"funcapi.h"=0A= +#include=20"miscadmin.h"=0A+#include=20= "postmaster/datachecksum_state.h"=0A+#include=20"storage/latch.h"=0A= +#include=20"utils/injection_point.h"=0A+#include=20"utils/wait_event.h"=0A= +=0A+PG_MODULE_MAGIC;=0A+=0A+extern=20PGDLLEXPORT=20void=20= dc_delay_barrier(const=20char=20*name,=20const=20void=20*private_data,=20= void=20*arg);=0A+extern=20PGDLLEXPORT=20void=20dc_modify_db_result(const=20= char=20*name,=20const=20void=20*private_data,=20void=20*arg);=0A+extern=20= PGDLLEXPORT=20void=20dc_fake_temptable(const=20char=20*name,=20const=20= void=20*private_data,=20void=20*arg);=0A+=0A+extern=20PGDLLEXPORT=20void=20= crash(const=20char=20*name,=20const=20void=20*private_data,=20void=20= *arg);=0A+=0A+/*=0A+=20*=20Test=20for=20delaying=20emission=20of=20= procsignalbarriers.=0A+=20*/=0A+void=0A+dc_delay_barrier(const=20char=20= *name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09(void)=20= name;=0A+=09(void)=20private_data;=0A+=0A+=09(void)=20WaitLatch(MyLatch,=0A= +=09=09=09=09=09=20WL_LATCH_SET=20|=20WL_TIMEOUT=20|=20= WL_EXIT_ON_PM_DEATH,=0A+=09=09=09=09=09=20(3=20*=201000),=0A+=09=09=09=09= =09=20WAIT_EVENT_PG_SLEEP);=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_inject_delay_barrier);=0A+Datum=0A= +dcw_inject_delay_barrier(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksums-enable-checksums-delay",=0A+=09=09=09= =09=09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_delay_barrier",=0A+=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09= =09=200);=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksums-enable-checksums-delay");=0A+#else=0A= +=09elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20intended=20= when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_inject_launcher_delay);=0A+Datum=0A= +dcw_inject_launcher_delay(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-launcher-delay",=0A+=09=09=09=09= =09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_delay_barrier",=0A+=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09= =09=200);=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksumsworker-launcher-delay");=0A+#else=0A+=09= elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20intended=20= when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_inject_startup_delay);=0A+Datum=0A= +dcw_inject_startup_delay(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-startup-delay",=0A+=09=09=09=09= =09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_delay_barrier",=0A+=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09= =09=200);=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksumsworker-startup-delay");=0A+#else=0A+=09= elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20intended=20= when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+static=20uint32=20db_fail=20=3D=20= DATACHECKSUMSWORKER_FAILED;=0A+=0A+void=0A+dc_modify_db_result(const=20= char=20*name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09= DataChecksumsWorkerResult=20*res=20=3D=20(DataChecksumsWorkerResult=20*)=20= arg;=0A+=09uint32=09=09new_res=20=3D=20*(uint32=20*)=20private_data;=0A+=0A= +=09*res=20=3D=20new_res;=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_inject_fail_database);=0A+Datum=0A= +dcw_inject_fail_database(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-modify-db-result",=0A+=09=09=09= =09=09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_modify_db_result",=0A+=09=09=09=09=09=09=09=20&db_fail,=0A+=09=09=09=09= =09=09=09=20sizeof(uint32));=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksumsworker-modify-db-result");=0A+#else=0A= +=09elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20intended=20= when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+/*=0A+=20*=20Test=20to=20force=20waiting=20= for=20existing=20temptables.=0A+=20*/=0A+void=0A+dc_fake_temptable(const=20= char=20*name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09= static=20bool=20first_pass=20=3D=20true;=0A+=09int=09=09=20=20=20= *numleft=20=3D=20(int=20*)=20arg;=0A+=0A+=09if=20(first_pass)=0A+=09=09= *numleft=20=3D=201;=0A+=09first_pass=20=3D=20false;=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_fake_temptable);=0A+Datum=0A= +dcw_fake_temptable(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-fake-temptable-wait",=0A+=09=09= =09=09=09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_fake_temptable",=0A+=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09= =09=200);=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksumsworker-fake-temptable-wait");=0A= +#else=0A+=09elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20= intended=20when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+void=0A+crash(const=20char=20*name,=20= const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09abort();=0A+}=0A= diff=20--git=20a/src/test/modules/test_checksums/test_checksums.control=20= b/src/test/modules/test_checksums/test_checksums.control=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..84b4cc035a7=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/test_checksums.control=0A@@=20-0,0=20= +1,4=20@@=0A+comment=20=3D=20'Test=20code=20for=20data=20checksums'=0A= +default_version=20=3D=20'1.0'=0A+module_pathname=20=3D=20= '$libdir/test_checksums'=0A+relocatable=20=3D=20true=0Adiff=20--git=20= a/src/test/perl/PostgreSQL/Test/Cluster.pm=20= b/src/test/perl/PostgreSQL/Test/Cluster.pm=0Aindex=20= f8dc732e66e..54e6b646e8f=20100644=0A---=20= a/src/test/perl/PostgreSQL/Test/Cluster.pm=0A+++=20= b/src/test/perl/PostgreSQL/Test/Cluster.pm=0A@@=20-3898,6=20+3898,42=20= @@=20sub=20advance_wal=0A=20=09}=0A=20}=0A=20=0A+=3Ditem=20= $node->checksum_enable_offline()=0A+=0A+Enable=20data=20page=20checksums=20= in=20an=20offline=20cluster=20with=20B.=20The=0A+caller=20= is=20responsible=20for=20ensuring=20that=20the=20cluster=20is=20in=20the=20= right=20state=20for=0A+this=20operation.=0A+=0A+=3Dcut=0A+=0A+sub=20= checksum_enable_offline=0A+{=0A+=09my=20($self)=20=3D=20@_;=0A+=0A+=09= print=20"#=20Enabling=20checksums=20in=20\"$self->data_dir\"\n";=0A+=09= PostgreSQL::Test::Utils::system_or_bail('pg_checksums',=20'-D',=0A+=09=09= $self->data_dir,=20'-e');=0A+=09return;=0A+}=0A+=0A+=3Ditem=20= $node->checksum_disable_offline()=0A+=0A+Disable=20data=20page=20= checksums=20in=20an=20offline=20cluster=20with=20B.=20The=0A= +caller=20is=20responsible=20for=20ensuring=20that=20the=20cluster=20is=20= in=20the=20right=20state=20for=0A+this=20operation.=0A+=0A+=3Dcut=0A+=0A= +sub=20checksum_disable_offline=0A+{=0A+=09my=20($self)=20=3D=20@_;=0A+=0A= +=09print=20"#=20Disabling=20checksums=20in=20\"$self->data_dir\"\n";=0A= +=09PostgreSQL::Test::Utils::system_or_bail('pg_checksums',=20'-D',=0A+=09= =09$self->data_dir,=20'-d');=0A+=09return;=0A+}=0A+=0A=20=3Dpod=0A=20=0A=20= =3Dback=0Adiff=20--git=20a/src/test/regress/expected/rules.out=20= b/src/test/regress/expected/rules.out=0Aindex=202b3cf6d8569..81a73c426d2=20= 100644=0A---=20a/src/test/regress/expected/rules.out=0A+++=20= b/src/test/regress/expected/rules.out=0A@@=20-2085,6=20+2085,41=20@@=20= pg_stat_progress_create_index|=20SELECT=20s.pid,=0A=20=20=20=20=20= s.param15=20AS=20partitions_done=0A=20=20=20=20FROM=20= (pg_stat_get_progress_info('CREATE=20INDEX'::text)=20s(pid,=20datid,=20= relid,=20param1,=20param2,=20param3,=20param4,=20param5,=20param6,=20= param7,=20param8,=20param9,=20param10,=20param11,=20param12,=20param13,=20= param14,=20param15,=20param16,=20param17,=20param18,=20param19,=20= param20)=0A=20=20=20=20=20=20LEFT=20JOIN=20pg_database=20d=20ON=20= ((s.datid=20=3D=20d.oid)));=0A+pg_stat_progress_data_checksums|=20SELECT=20= s.pid,=0A+=20=20=20=20s.datid,=0A+=20=20=20=20d.datname,=0A+=20=20=20=20=20= =20=20=20CASE=20s.param1=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=200=20= THEN=20'enabling'::text=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=201=20= THEN=20'disabling'::text=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=202=20= THEN=20'waiting=20on=20temporary=20tables'::text=0A+=20=20=20=20=20=20=20= =20=20=20=20=20WHEN=203=20THEN=20'waiting=20on=20barrier'::text=0A+=20=20= =20=20=20=20=20=20=20=20=20=20WHEN=204=20THEN=20'done'::text=0A+=20=20=20= =20=20=20=20=20=20=20=20=20ELSE=20NULL::text=0A+=20=20=20=20=20=20=20=20= END=20AS=20phase,=0A+=20=20=20=20=20=20=20=20CASE=20s.param2=0A+=20=20=20= =20=20=20=20=20=20=20=20=20WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A= +=20=20=20=20=20=20=20=20=20=20=20=20ELSE=20s.param2=0A+=20=20=20=20=20=20= =20=20END=20AS=20databases_total,=0A+=20=20=20=20s.param3=20AS=20= databases_done,=0A+=20=20=20=20=20=20=20=20CASE=20s.param4=0A+=20=20=20=20= =20=20=20=20=20=20=20=20WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A+=20= =20=20=20=20=20=20=20=20=20=20=20ELSE=20s.param4=0A+=20=20=20=20=20=20=20= =20END=20AS=20relations_total,=0A+=20=20=20=20=20=20=20=20CASE=20= s.param5=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=20'-1'::integer=20= THEN=20NULL::bigint=0A+=20=20=20=20=20=20=20=20=20=20=20=20ELSE=20= s.param5=0A+=20=20=20=20=20=20=20=20END=20AS=20relations_done,=0A+=20=20=20= =20=20=20=20=20CASE=20s.param6=0A+=20=20=20=20=20=20=20=20=20=20=20=20= WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A+=20=20=20=20=20=20=20=20=20= =20=20=20ELSE=20s.param6=0A+=20=20=20=20=20=20=20=20END=20AS=20= blocks_total,=0A+=20=20=20=20=20=20=20=20CASE=20s.param7=0A+=20=20=20=20=20= =20=20=20=20=20=20=20WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A+=20=20= =20=20=20=20=20=20=20=20=20=20ELSE=20s.param7=0A+=20=20=20=20=20=20=20=20= END=20AS=20blocks_done=0A+=20=20=20FROM=20= (pg_stat_get_progress_info('DATACHECKSUMS'::text)=20s(pid,=20datid,=20= relid,=20param1,=20param2,=20param3,=20param4,=20param5,=20param6,=20= param7,=20param8,=20param9,=20param10,=20param11,=20param12,=20param13,=20= param14,=20param15,=20param16,=20param17,=20param18,=20param19,=20= param20)=0A+=20=20=20=20=20LEFT=20JOIN=20pg_database=20d=20ON=20= ((s.datid=20=3D=20d.oid)))=0A+=20=20ORDER=20BY=20s.datid;=0A=20= pg_stat_progress_repack|=20SELECT=20s.pid,=0A=20=20=20=20=20s.datid,=0A=20= =20=20=20=20d.datname,=0Adiff=20--git=20= a/src/test/regress/expected/stats.out=20= b/src/test/regress/expected/stats.out=0Aindex=20ea7f7846895..35632f83052=20= 100644=0A---=20a/src/test/regress/expected/stats.out=0A+++=20= b/src/test/regress/expected/stats.out=0A@@=20-51,6=20+51,22=20@@=20= client=20backend|relation|vacuum=0A=20client=20backend|temp=20= relation|normal=0A=20client=20backend|wal|init=0A=20client=20= backend|wal|normal=0A+datachecksum=20launcher|relation|bulkread=0A= +datachecksum=20launcher|relation|bulkwrite=0A+datachecksum=20= launcher|relation|init=0A+datachecksum=20launcher|relation|normal=0A= +datachecksum=20launcher|relation|vacuum=0A+datachecksum=20launcher|temp=20= relation|normal=0A+datachecksum=20launcher|wal|init=0A+datachecksum=20= launcher|wal|normal=0A+datachecksum=20worker|relation|bulkread=0A= +datachecksum=20worker|relation|bulkwrite=0A+datachecksum=20= worker|relation|init=0A+datachecksum=20worker|relation|normal=0A= +datachecksum=20worker|relation|vacuum=0A+datachecksum=20worker|temp=20= relation|normal=0A+datachecksum=20worker|wal|init=0A+datachecksum=20= worker|wal|normal=0A=20io=20worker|relation|bulkread=0A=20io=20= worker|relation|bulkwrite=0A=20io=20worker|relation|init=0A@@=20-95,7=20= +111,7=20@@=20walsummarizer|wal|init=0A=20walsummarizer|wal|normal=0A=20= walwriter|wal|init=0A=20walwriter|wal|normal=0A-(79=20rows)=0A+(95=20= rows)=0A=20\a=0A=20--=20ensure=20that=20both=20seqscan=20and=20indexscan=20= plans=20are=20allowed=0A=20SET=20enable_seqscan=20TO=20on;=0Adiff=20= --git=20a/src/tools/pgindent/typedefs.list=20= b/src/tools/pgindent/typedefs.list=0Aindex=2091b1225da82..ad999aa48dd=20= 100644=0A---=20a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-438,6=20+438,8=20@@=20= CheckPointStmt=0A=20CheckpointStatsData=0A=20CheckpointerRequest=0A=20= CheckpointerShmemStruct=0A+ChecksumBarrierCondition=0A+ChecksumStateType=0A= =20Chromosome=0A=20CkptSortItem=0A=20CkptTsStatus=0A@@=20-610,6=20+612,7=20= @@=20CustomScan=0A=20CustomScanMethods=0A=20CustomScanState=0A=20= CycleCtr=0A+DataChecksumsWorkerOperation=0A=20DBState=0A=20DbOidName=0A=20= DCHCacheEntry=0A@@=20-628,6=20+631,9=20@@=20DSMREntryType=0A=20= DSMRegistryCtxStruct=0A=20DSMRegistryEntry=0A=20DWORD=0A= +DataChecksumsWorkerDatabase=0A+DataChecksumsWorkerResult=0A= +DataChecksumsStateStruct=0A=20DataDirSyncMethod=0A=20DataDumperPtr=0A=20= DataPageDeleteStack=0A@@=20-4405,6=20+4411,7=20@@=20xl_btree_unlink_page=0A= =20xl_btree_update=0A=20xl_btree_vacuum=0A=20xl_checkpoint_redo=0A= +xl_checksum_state=0A=20xl_clog_truncate=0A=20xl_commit_ts_truncate=0A=20= xl_dbase_create_file_copy_rec=0A--=20=0A2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_E4DE5431-D9EB-4BD5-9E42-507D14F7721A--