Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1unMfX-0065OG-11 for pgsql-hackers@arkaria.postgresql.org; Sat, 16 Aug 2025 19:34:25 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1unMfW-006qp2-7W for pgsql-hackers@arkaria.postgresql.org; Sat, 16 Aug 2025 19:34:22 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1unMfV-006qot-Iz for pgsql-hackers@lists.postgresql.org; Sat, 16 Aug 2025 19:34:22 +0000 Received: from smtp.outgoing.loopia.se ([93.188.3.37]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1unMfT-000D98-18 for pgsql-hackers@lists.postgresql.org; Sat, 16 Aug 2025 19:34:21 +0000 Received: from s807.loopia.se (localhost [127.0.0.1]) by s807.loopia.se (Postfix) with ESMTP id E79ED40215C for ; Sat, 16 Aug 2025 21:34:17 +0200 (CEST) Received: from s980.loopia.se (unknown [172.22.191.6]) by s807.loopia.se (Postfix) with ESMTP id D896E401E52; Sat, 16 Aug 2025 21:34:17 +0200 (CEST) Received: from s470.loopia.se (unknown [172.22.191.6]) by s980.loopia.se (Postfix) with ESMTP id D3F61220164A; Sat, 16 Aug 2025 21:34:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at amavis.loopia.se X-Spam-Flag: NO X-Spam-Score: -1.2 X-Spam-Level: X-Spam-Status: No, score=-1.2 tagged_above=-999 required=6.2 tests=[ALL_TRUSTED=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1] autolearn=disabled Authentication-Results: s470.loopia.se (amavisd-new); dkim=pass (2048-bit key) header.d=yesql.se Received: from s980.loopia.se ([172.22.191.6]) by s470.loopia.se (s470.loopia.se [172.22.190.34]) (amavisd-new, port 10024) with LMTP id alJF6KmY1lbg; Sat, 16 Aug 2025 21:34:14 +0200 (CEST) X-Loopia-Auth: user X-Loopia-User: daniel@yesql.se X-Loopia-Originating-IP: 89.255.232.236 Received: from smtpclient.apple (customer-89-255-232-236.stosn.net [89.255.232.236]) (Authenticated sender: daniel@yesql.se) by s980.loopia.se (Postfix) with ESMTPSA id 318522201622; Sat, 16 Aug 2025 21:34:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yesql.se; s=loopiadkim1707475645; t=1755372854; bh=VfxpmFsF8OoFU+8/F9Fr/WuN22PfO++yDcjueFCE0gg=; h=From:Subject:Date:In-Reply-To:Cc:To:References; b=VNTvXQDy/w1smjqlyPsXsepAz7gx1ufcFR3/GKIKx2GWVBBkqR+4H5kGW+4R/6uFX QupkvGA5qVeXSAeNdYTo2CGwNaqV1gd72S6tbue3mpgU/V9HJFQdh2BRlZI6q89Waa QFvj9nMJ0E9eJlc7lbcMuwp1JdGq1EKq6WoXM6i2sfZX6vGqgHS/XZYLNgLCT1G4MS Inux57xWAU1KoNCeTO+NfXc3TW/ODxH3mylMQzaBnk05DDN3keV5ezHNn9quCUoAxp Cr2nFdSERJ9t91nNQD5mY3oSkjje5O5ktkqFU8niKxAmf8Wirr88pzxMFYtF+9rW/n 4PB5IbmX99Kwg== From: Daniel Gustafsson Message-Id: <2CD83EC2-C971-4440-89F7-BCBAE375F908@yesql.se> Content-Type: multipart/mixed; boundary="Apple-Mail=_3DBBE924-A497-4C66-A4E1-9A3177F6ED66" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.2\)) Subject: Re: Changing the state of data checksums in a running cluster Date: Sat, 16 Aug 2025 21:34:03 +0200 In-Reply-To: <2864a79c1ab628f609104a955a725879fdf5486f.camel@oopsware.de> Cc: Tomas Vondra , Michael Paquier , Michael Banck , PostgreSQL Hackers To: Bernd Helmle References: <8B317721-1CB7-4F82-B4DA-6B15AB1725D1@yesql.se> <25edd1d1-e693-4386-bbcc-765ea1820044@vondra.me> <49f6df22-7a40-4ce1-a651-111ac033b2bb@vondra.me> <82E20987-0A25-489C-935B-B54ABF165542@yesql.se> <9a2f67a1-faba-423d-bd3d-08a01ca35494@vondra.me> <40eb5e02-e715-43d3-a618-f60555bb407c@vondra.me> <9591d5d0-1fb1-4b43-bf09-501b6df5d227@vondra.me> <5AED03AE-C15F-43DD-93AE-CCD6F3DC6A07@yesql.se> <854de38e-8899-4095-8245-6531d86060c9@vondra.me> <87ce475d-1a21-4aa9-914f-7f37159ef010@vondra.me> <3372a09c-d1f6-4974-ad60-eec15ee0c734@vondra.me> <3E9E1AF5-5344-45C5-9B38-A166CE57472E@yesql.se> <442AB85E-028E-419E-A330-40F7D3445871@yesql.se> <2864a79c1ab628f609104a955a725879fdf5486f.camel@oopsware.de> X-Mailer: Apple Mail (2.3776.700.51.11.2) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_3DBBE924-A497-4C66-A4E1-9A3177F6ED66 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Attached is a rebase on top of the func.sgml changes which caused this = to no longer apply. This version is also substantially updated with a new injection point = based test suite, fixed a few bugs (found by said test suite), added = checkpoint to disabling checksums, code cleanup, more granular wait events, comment = rewrites and additions and more smaller cleanups. -- Daniel Gustafsson --Apple-Mail=_3DBBE924-A497-4C66-A4E1-9A3177F6ED66 Content-Disposition: attachment; filename=v20250816-0001-Online-enabling-and-disabling-of-data-chec.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v20250816-0001-Online-enabling-and-disabling-of-data-chec.patch" Content-Transfer-Encoding: quoted-printable =46rom=20298a9390b81749fef66bcba74ff93a4fadca5b66=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20= =0ADate:=20Fri,=2015=20Aug=202025=2014:48:02=20= +0200=0ASubject:=20[PATCH=20v20250816]=20Online=20enabling=20and=20= disabling=20of=20data=20checksums=0A=0AThis=20allows=20data=20checksums=20= to=20be=20enabled,=20or=20disabled,=20in=20a=20running=0Acluster=20= without=20restricting=20access=20to=20the=20cluster=20during=20= processing.=0A=0AData=20checksums=20could=20prior=20to=20this=20only=20= be=20enabled=20during=20initdb=20or=0Awhen=20the=20cluster=20is=20= offline=20using=20the=20pg_checksums=20app.=20This=20commit=0Aintroduce=20= functionality=20to=20enable,=20or=20disable,=20data=20checksums=20while=0A= the=20cluster=20is=20running=20regardless=20of=20how=20it=20was=20= initialized.=0A=0AA=20background=20worker=20launcher=20process=20is=20= responsible=20for=20launching=20a=0Adynamic=20per-database=20background=20= worker=20which=20will=20mark=20all=20buffers=0Adirty=20for=20all=20= relation=20with=20storage=20in=20order=20for=20them=20to=20have=20data=0A= checksums=20calcuated=20on=20write.=20=20Once=20all=20relations=20in=20= all=20databases=0Ahave=20been=20processed,=20the=20data_checksums=20= state=20will=20be=20set=20to=20on=20and=0Athe=20cluster=20will=20at=20= that=20point=20be=20identical=20to=20one=20which=20had=20data=0A= checksums=20enabled=20during=20initialization=20or=20via=20offline=20= processing.=0A=0AWhen=20data=20checksums=20are=20being=20enabled,=20= concurrent=20I/O=20operations=0Afrom=20backends=20other=20than=20the=20= data=20checksums=20worker=20will=20write=20the=0Achecksums=20but=20not=20= verify=20them=20on=20reading.=20=20Only=20when=20all=20backends=0Ahave=20= absorbed=20the=20procsignalbarrier=20for=20setting=20data_checksums=20to=0A= on=20will=20they=20also=20start=20verifying=20checksums=20on=20reading.=20= =20The=20same=0Aprocess=20is=20repeated=20during=20disabling;=20all=20= backends=20write=20checksums=0Abut=20do=20not=20verify=20them=20until=20= the=20barrier=20for=20setting=20the=20state=20to=0Aoff=20has=20been=20= absorbed=20by=20all.=20=20This=20in-progress=20state=20is=20used=20to=0A= ensure=20there=20are=20no=20false=20negatives=20(or=20positives)=20due=20= to=20reading=0Aa=20checksum=20which=20is=20not=20in=20sync=20with=20the=20= page.=0A=0AThis=20work=20is=20based=20on=20an=20earlier=20version=20of=20= this=20patch=20which=20was=0Areviewed=20by=20among=20others=20Heikki=20= Linnakangas,=20Robert=20Haas,=20Andres=0AFreund,=20Tomas=20Vondra,=20= Michael=20Banck=20and=20Andrey=20Borodin.=20=20Tomas=0AVondra=20has=20= given=20invaluable=20assistance=20with=20not=20only=20review=20but=0A= very=20in-depth=20testing.=0A=0AAuthor:=20Daniel=20Gustafsson=20= =0AAuthor:=20Magnus=20Hagander=20=0A= Reviewed-by:=20Tomas=20Vondra=20=0ADiscussion:=20= https://postgr.es/m/CABUevExz9hUUOLnJVr2kpw9Cx=3Do4MCr1SVKwbupzuxP7ckNutA@= mail.gmail.com=0ADiscussion:=20= https://postgr.es/m/20181030051643.elbxjww5jjgnjaxg@alap3.anarazel.de=0A= Discussion:=20= https://postgr.es/m/CABUevEwE3urLtwxxqdgd5O2oQz9J717ZzMbh+ziCSa5YLLU_BA@ma= il.gmail.com=0A---=0A=20doc/src/sgml/func/func-admin.sgml=20=20=20=20=20=20= =20=20=20=20=20=20=20|=20=20=2071=20+=0A=20doc/src/sgml/glossary.sgml=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=2023=20+=0A= =20doc/src/sgml/monitoring.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20208=20++-=0A=20doc/src/sgml/ref/pg_checksums.sgml=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=20=206=20+=0A=20= doc/src/sgml/regress.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=2011=20+=0A=20doc/src/sgml/wal.sgml=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=2059=20= +-=0A=20src/backend/access/rmgrdesc/xlogdesc.c=20=20=20=20=20=20=20=20|=20= =20=2024=20+=0A=20src/backend/access/transam/xlog.c=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20547=20+++++-=0A=20= src/backend/access/transam/xlogfuncs.c=20=20=20=20=20=20=20=20|=20=20=20= 43=20+=0A=20src/backend/backup/basebackup.c=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=206=20+-=0A=20= src/backend/catalog/system_functions.sql=20=20=20=20=20=20|=20=20=2016=20= +=0A=20src/backend/catalog/system_views.sql=20=20=20=20=20=20=20=20=20=20= |=20=20=2020=20+=0A=20src/backend/postmaster/Makefile=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/backend/postmaster/auxprocess.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =2019=20+=0A=20src/backend/postmaster/bgworker.c=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20=20=207=20+=0A=20= src/backend/postmaster/datachecksumsworker.c=20=20|=201463=20= +++++++++++++++++=0A=20src/backend/postmaster/launch_backend.c=20=20=20=20= =20=20=20|=20=20=20=203=20+=0A=20src/backend/postmaster/meson.build=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/backend/postmaster/postmaster.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =20=205=20+=0A=20src/backend/replication/logical/decode.c=20=20=20=20=20=20= |=20=20=20=201=20+=0A=20src/backend/storage/ipc/ipci.c=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=20=20=20=204=20+=0A=20= src/backend/storage/ipc/procsignal.c=20=20=20=20=20=20=20=20=20=20|=20=20= =2013=20+=0A=20src/backend/storage/page/README=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=20=20=204=20+-=0A=20= src/backend/storage/page/bufpage.c=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=20=206=20+-=0A=20src/backend/utils/activity/pgstat_backend.c=20=20=20= |=20=20=20=202=20+=0A=20src/backend/utils/activity/pgstat_io.c=20=20=20=20= =20=20=20=20|=20=20=20=202=20+=0A=20= .../utils/activity/wait_event_names.txt=20=20=20=20=20=20=20|=20=20=20=20= 4=20+=0A=20src/backend/utils/adt/pgstatfuncs.c=20=20=20=20=20=20=20=20=20= =20=20|=20=20=20=208=20+-=0A=20src/backend/utils/init/miscinit.c=20=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=2012=20+-=0A=20= src/backend/utils/init/postinit.c=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=2020=20+-=0A=20src/backend/utils/misc/guc_tables.c=20=20=20=20=20= =20=20=20=20=20=20|=20=20=2030=20+-=0A=20= src/bin/pg_checksums/pg_checksums.c=20=20=20=20=20=20=20=20=20=20=20|=20=20= =20=202=20+-=0A=20src/bin/pg_controldata/pg_controldata.c=20=20=20=20=20=20= =20|=20=20=20=202=20+=0A=20src/bin/pg_upgrade/controldata.c=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=20=20=20=209=20+=0A=20= src/include/access/xlog.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=2017=20+-=0A=20src/include/access/xlog_internal.h=20= =20=20=20=20=20=20=20=20=20=20=20|=20=20=20=207=20+=0A=20= src/include/catalog/pg_control.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=206=20+-=0A=20src/include/catalog/pg_proc.dat=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=20=20=2019=20+=0A=20= src/include/commands/progress.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20|=20=20=2016=20+=0A=20src/include/miscadmin.h=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=206=20+=0A=20= src/include/postmaster/datachecksumsworker.h=20=20|=20=20=2051=20+=0A=20= src/include/storage/bufpage.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20|=20=20=20=202=20+-=0A=20src/include/storage/checksum.h=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20|=20=20=2014=20+=0A=20= src/include/storage/lwlocklist.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=201=20+=0A=20src/include/storage/proc.h=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=206=20+-=0A=20= src/include/storage/procsignal.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=205=20+=0A=20src/include/utils/backend_progress.h=20=20=20=20=20= =20=20=20=20=20|=20=20=20=201=20+=0A=20src/test/Makefile=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=2011=20+-=0A=20src/test/modules/meson.build=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= src/test/modules/test_checksums/.gitignore=20=20=20=20|=20=20=20=202=20+=0A= =20src/test/modules/test_checksums/Makefile=20=20=20=20=20=20|=20=20=20= 36=20+=0A=20src/test/modules/test_checksums/README=20=20=20=20=20=20=20=20= |=20=20=2022=20+=0A=20src/test/modules/test_checksums/meson.build=20=20=20= |=20=20=2034=20+=0A=20.../modules/test_checksums/t/001_basic.pl=20=20=20=20= =20|=20=20=2063=20+=0A=20.../modules/test_checksums/t/002_restarts.pl=20=20= |=20=20110=20++=0A=20.../test_checksums/t/003_standby_restarts.pl=20=20|=20= =20114=20++=0A=20.../modules/test_checksums/t/004_offline.pl=20=20=20|=20= =20=2082=20+=0A=20.../modules/test_checksums/t/005_injection.pl=20|=20=20= =2076=20+=0A=20.../test_checksums/t/DataChecksums/Utils.pm=20=20=20|=20=20= 185=20+++=0A=20.../test_checksums/test_checksums--1.0.sql=20=20=20=20|=20= =20=2020=20+=0A=20.../modules/test_checksums/test_checksums.c=20=20=20|=20= =20173=20++=0A=20.../test_checksums/test_checksums.control=20=20=20=20=20= |=20=20=20=204=20+=0A=20src/test/perl/PostgreSQL/Test/Cluster.pm=20=20=20= =20=20=20|=20=20=2036=20+=0A=20src/test/regress/expected/rules.out=20=20=20= =20=20=20=20=20=20=20=20|=20=20=2036=20+=0A=20= src/test/regress/expected/stats.out=20=20=20=20=20=20=20=20=20=20=20|=20=20= =2018=20+-=0A=20src/tools/pgindent/typedefs.list=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=206=20+=0A=2066=20files=20changed,=203777=20= insertions(+),=2055=20deletions(-)=0A=20create=20mode=20100644=20= src/backend/postmaster/datachecksumsworker.c=0A=20create=20mode=20100644=20= src/include/postmaster/datachecksumsworker.h=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/.gitignore=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/Makefile=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/README=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/meson.build=0A=20create=20mode=20100644=20= src/test/modules/test_checksums/t/001_basic.pl=0A=20create=20mode=20= 100644=20src/test/modules/test_checksums/t/002_restarts.pl=0A=20create=20= mode=20100644=20= src/test/modules/test_checksums/t/003_standby_restarts.pl=0A=20create=20= mode=20100644=20src/test/modules/test_checksums/t/004_offline.pl=0A=20= create=20mode=20100644=20= src/test/modules/test_checksums/t/005_injection.pl=0A=20create=20mode=20= 100644=20src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0A=20= create=20mode=20100644=20= src/test/modules/test_checksums/test_checksums--1.0.sql=0A=20create=20= mode=20100644=20src/test/modules/test_checksums/test_checksums.c=0A=20= create=20mode=20100644=20= src/test/modules/test_checksums/test_checksums.control=0A=0Adiff=20--git=20= a/doc/src/sgml/func/func-admin.sgml=20= b/doc/src/sgml/func/func-admin.sgml=0Aindex=20446fdfe56f4..69afb61ceb8=20= 100644=0A---=20a/doc/src/sgml/func/func-admin.sgml=0A+++=20= b/doc/src/sgml/func/func-admin.sgml=0A@@=20-2959,4=20+2959,75=20@@=20= SELECT=20convert_from(pg_read_binary_file('file_in_utf8.txt'),=20= 'UTF8');=0A=20=0A=20=20=20=0A=20=0A+=20=20=0A+=20=20=20Data=20Checksum=20= Functions=0A+=0A+=20=20=20=0A+=20=20=20=20The=20functions=20= shown=20in=20=20can=0A= +=20=20=20=20be=20used=20to=20enable=20or=20disable=20data=20checksums=20= in=20a=20running=20cluster.=0A+=20=20=20=20See=20=20for=20details.=0A+=20=20=20=0A+=0A+=20= =20=20=0A+=20=20=20=20= Data=20Checksum=20Functions=0A+=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20Function=0A+=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20Description=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20= pg_enable_data_checksums=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20= pg_enable_data_checksums=20(=20= cost_delay=20int,=20= cost_limit=20int=20)=0A+=20= =20=20=20=20=20=20=20void=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Initiates=20data=20checksums=20for=20the=20cluster.=20This=20will=20= switch=20the=20data=0A+=20=20=20=20=20=20=20=20checksums=20mode=20to=20= inprogress-on=20as=20well=20as=20start=20a=0A+=20=20=20= =20=20=20=20=20background=20worker=20that=20will=20process=20all=20pages=20= in=20the=20database=20and=0A+=20=20=20=20=20=20=20=20enable=20checksums=20= on=20them.=20When=20all=20data=20pages=20have=20had=20checksums=0A+=20=20= =20=20=20=20=20=20enabled,=20the=20cluster=20will=20automatically=20= switch=20data=20checksums=20mode=20to=0A+=20=20=20=20=20=20=20=20= on.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20=20If=20= cost_delay=20and=20= cost_limit=20are=0A+=20=20=20=20=20=20=20=20= specified,=20the=20speed=20of=20the=20process=20is=20throttled=20using=20= the=20same=20principles=20as=0A+=20=20=20=20=20=20=20=20Cost-based=20Vacuum=20= Delay.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20= pg_disable_data_checksums=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20= pg_disable_data_checksums=20()=0A+=20=20=20=20=20=20= =20=20void=0A+=20=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20Disables=20data=20= checksum=20validation=20and=20calculation=20for=20the=20cluster.=20This=0A= +=20=20=20=20=20=20=20=20will=20switch=20the=20data=20checksum=20mode=20= to=20inprogress-off=0A+=20=20=20=20=20=20=20=20while=20= data=20checksums=20are=20being=20disabled.=20When=20all=20active=20= backends=20have=0A+=20=20=20=20=20=20=20=20stopped=20validating=20data=20= checksums,=20the=20data=20checksum=20mode=20will=20be=0A+=20=20=20=20=20=20= =20=20changed=20to=20off.=20=20At=20this=20point=20= the=20data=20pages=20will=0A+=20=20=20=20=20=20=20=20still=20have=20= checksums=20recorded=20but=20they=20are=20not=20updated=20when=20pages=20= are=0A+=20=20=20=20=20=20=20=20modified.=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=20=20=20=20=0A+=20=20=20=0A+=0A+=20=20=0A+=0A=20= =20=20=0Adiff=20--git=20a/doc/src/sgml/glossary.sgml=20= b/doc/src/sgml/glossary.sgml=0Aindex=20b88cac598e9..a4e16d03aae=20100644=0A= ---=20a/doc/src/sgml/glossary.sgml=0A+++=20b/doc/src/sgml/glossary.sgml=0A= @@=20-184,6=20+184,8=20@@=0A=20=20=20=20=20=20(but=20not=20the=20= autovacuum=20workers),=0A=20=20=20=20=20=20the=20background=20writer,=0A= =20=20=20=20=20=20the=20checkpointer,=0A+=20=20=20=20= =20the=20data=20= checksums=20worker,=0A+=20=20=20=20=20the=20data=20checksums=20= worker=20launcher,=0A=20=20=20=20=20=20the=20logger,=0A=20=20=20=20=20=20the=20= startup=20= process,=0A=20=20=20=20=20=20the=20WAL=20archiver,=0A@@=20= -573,6=20+575,27=20@@=0A=20=20=20=20=0A=20=20=20=0A= =20=0A+=20=20=0A+=20=20= =20Data=20Checksums=20Worker=0A+=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=20An=20auxiliary=20process=0A+=20= =20=20=20=20which=20enables=20or=20disables=20data=20checksums=20in=20a=20= specific=20database.=0A+=20=20=20=20=0A+=20=20=20=0A+=20= =20=0A+=0A+=20=20=0A+=20=20=20= Data=20Checksums=20Worker=20Launcher=0A+=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=20An=20auxiliary=20process=0A+=20= =20=20=20=20which=20starts=20=20processes=0A+=20= =20=20=20=20for=20each=20database.=0A+=20=20=20=20=0A+=20=20=20= =0A+=20=20=0A+=0A=20=20=20=0A=20=20=20=20Database=20= cluster=0A=20=20=20=20=0Adiff=20--git=20= a/doc/src/sgml/monitoring.sgml=20b/doc/src/sgml/monitoring.sgml=0Aindex=20= 3f4a27a736e..6082d991497=20100644=0A---=20a/doc/src/sgml/monitoring.sgml=0A= +++=20b/doc/src/sgml/monitoring.sgml=0A@@=20-3527,8=20+3527,9=20@@=20= description=20|=20Waiting=20for=20a=20newly=20initialized=20WAL=20file=20= to=20reach=20durable=20storage=0A=20=20=20=20=20=20=20=0A=20=20=20= =20=20=20=20=0A=20=20=20=20=20=20=20=20Number=20of=20data=20page=20= checksum=20failures=20detected=20in=20this=0A-=20=20=20=20=20=20=20= database=20(or=20on=20a=20shared=20object),=20or=20NULL=20if=20data=20= checksums=20are=0A-=20=20=20=20=20=20=20disabled.=0A+=20=20=20=20=20=20=20= database=20(or=20on=20a=20shared=20object).=0A+=20=20=20=20=20=20=20= Detected=20failures=20are=20reported=20regardless=20of=20the=0A+=20=20=20= =20=20=20=20=20setting.=0A=20=20=20= =20=20=20=20=0A=20=20=20=20=20=20=0A=20=0A@@=20= -3538,8=20+3539,8=20@@=20description=20|=20Waiting=20for=20a=20newly=20= initialized=20WAL=20file=20to=20reach=20durable=20storage=0A=20=20=20=20=20= =20=20=0A=20=20=20=20=20=20=20=0A=20=20=20=20=20=20=20=20= Time=20at=20which=20the=20last=20data=20page=20checksum=20failure=20was=20= detected=20in=0A-=20=20=20=20=20=20=20this=20database=20(or=20on=20a=20= shared=20object),=20or=20NULL=20if=20data=20checksums=20are=0A-=20=20=20=20= =20=20=20disabled.=0A+=20=20=20=20=20=20=20this=20database=20(or=20on=20= a=20shared=20object).=20Last=20failure=20is=20reported=0A+=20=20=20=20=20= =20=20regardless=20of=20the=20=20= setting.=0A=20=20=20=20=20=20=20=0A=20=20=20=20=20=20= =0A=20=0A@@=20-6877,6=20+6878,205=20@@=20FROM=20= pg_stat_get_backend_idset()=20AS=20backendid;=0A=20=0A=20=20=0A=20= =0A+=20=0A+=20=20= Data=20Checksum=20Progress=20Reporting=0A+=0A+=20=20= =0A+=20=20=20= pg_stat_progress_data_checksums=0A+=20=20=0A= +=0A+=20=20=0A+=20=20=20When=20data=20checksums=20are=20being=20= enabled=20on=20a=20running=20cluster,=20the=0A+=20=20=20= pg_stat_progress_data_checksums=20view=20will=20= contain=0A+=20=20=20a=20row=20for=20the=20launcher=20process,=20and=20= one=20row=20for=20each=20worker=20process=20which=0A+=20=20=20is=20= currently=20calculating=20checksums=20for=20the=20data=20pages=20in=20= one=20database.=0A+=20=20=0A+=0A+=20=20=0A+=20=20=20= <structname>pg_stat_progress_data_checksums</structname>=20= View=0A+=20=20=20=0A+=20=20=20=20=0A= +=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20Column=20Type=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=20Description>=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= pid=20integer=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Process=20ID=20of=20a=20datachecksumworker=20process.=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=0A= +=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20datid=20oid=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= OID=20of=20this=20database,=20or=200=20for=20the=20launcher=20process=0A= +=20=20=20=20=20=20=20relation=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A= +=20=20=20=20=20=20=20datname=20= name=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20Name=20of=20this=20database,=20or=20= NULL=20for=20the=0A+=20=20=20=20=20=20=20launcher=20= process.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= phase=20text=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= Current=20processing=20phase,=20see=20=0A+=20=20=20=20=20=20=20=20for=20= description=20of=20the=20phases.=0A+=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20databases_total=20integer=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=20The=20total=20number=20of=20databases=20which=20will=20be=20= processed.=20Only=20the=0A+=20=20=20=20=20=20=20=20launcher=20worker=20= has=20this=20value=20set,=20the=20other=20worker=20processes=0A+=20=20=20= =20=20=20=20=20have=20this=20set=20to=20NULL.=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= databases_done=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20number=20of=20databases=20which=20have=20been=20processed.=20= Only=20the=0A+=20=20=20=20=20=20=20=20launcher=20worker=20has=20this=20= value=20set,=20the=20other=20worker=20processes=0A+=20=20=20=20=20=20=20=20= have=20this=20set=20to=20NULL.=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= relations_total=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20total=20number=20of=20relations=20which=20will=20be=20= processed,=20or=0A+=20=20=20=20=20=20=20=20NULL=20if=20= the=20data=20checksums=20worker=20process=20hasn't=0A+=20=20=20=20=20=20=20= =20calculated=20the=20number=20of=20relations=20yet.=20The=20launcher=20= process=20has=0A+=20=20=20=20=20=20=20=20this=20NULL.=0A= +=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= relations_done=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20number=20of=20relations=20which=20have=20been=20processed.=20= The=20launcher=0A+=20=20=20=20=20=20=20=20process=20has=20this=20= NULL.=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20= blocks_total=20integer=0A+=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20The=20number=20of=20blocks=20in=20the=20current=20relation=20which=20= will=20be=20processed,=0A+=20=20=20=20=20=20=20=20or=20= NULL=20if=20the=20data=20checksums=20worker=20process=20= hasn't=0A+=20=20=20=20=20=20=20=20calculated=20the=20number=20of=20= blocks=20yet.=20The=20launcher=20process=20has=0A+=20=20=20=20=20=20=20=20= this=20NULL.=0A+=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20blocks_done=20integer=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20The=20number=20of=20blocks=20in=20the=20current=20relation=20which=20= have=20been=20processed.=0A+=20=20=20=20=20=20=20=20The=20launcher=20= process=20has=20this=20NULL.=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=0A+=20= =20=20=20=0A+=20=20=20=0A+=20=20=0A+=0A+=20=20= =0A+=20=20=20Data=20Checksum=20= Phases=0A+=20=20=20=0A+=20=20=20=20=0A+=20=20=20=20=0A+=20=20=20=20=0A+=20=20=20=20= =20=0A+=20=20=20=20=20=20Phase=0A+=20=20=20=20=20=20= Description=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20= =20enabling=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20The=20command=20is=20currently=20= enabling=20data=20checksums=20on=20the=20cluster.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =20=20disabling=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20The=20command=20is=20currently=20= disabling=20data=20checksums=20on=20the=20cluster.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =20=20waiting=20on=20temporary=20= tables=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20The=20command=20is=20currently=20waiting=20for=20all=20temporary=20= tables=20which=20existed=0A+=20=20=20=20=20=20=20at=20the=20time=20the=20= command=20was=20started=20to=20be=20removed.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =20=20waiting=20on=20checkpoint=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20The=20command=20is=20= currently=20waiting=20for=20a=20checkpoint=20to=20update=20the=20= checksum=0A+=20=20=20=20=20=20=20state=20before=20finishing.=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=0A+=20= =20=20=0A+=20=20=0A+=20=0A+=0A=20=20=0A=20= =0A=20=20=0Adiff=20--git=20= a/doc/src/sgml/ref/pg_checksums.sgml=20= b/doc/src/sgml/ref/pg_checksums.sgml=0Aindex=2095043aa329c..0343710af53=20= 100644=0A---=20a/doc/src/sgml/ref/pg_checksums.sgml=0A+++=20= b/doc/src/sgml/ref/pg_checksums.sgml=0A@@=20-45,6=20+45,12=20@@=20= PostgreSQL=20documentation=0A=20=20=20=20exit=20status=20is=20nonzero=20= if=20the=20operation=20failed.=0A=20=20=20=0A=20=0A+=20=20=0A= +=20=20=20When=20enabling=20checksums,=20if=20checksums=20were=20in=20= the=20process=20of=20being=20enabled=0A+=20=20=20when=20the=20cluster=20= was=20shut=20down,=20pg_checksums=0A+=20=20=20= will=20still=20process=20all=20relations=20regardless=20of=20the=20= online=20processing.=0A+=20=20=0A+=0A=20=20=20=0A=20=20=20=20= When=20verifying=20checksums,=20every=20file=20in=20the=20cluster=20is=20= scanned.=20When=0A=20=20=20=20enabling=20checksums,=20each=20relation=20= file=20block=20with=20a=20changed=20checksum=20is=0Adiff=20--git=20= a/doc/src/sgml/regress.sgml=20b/doc/src/sgml/regress.sgml=0Aindex=20= 8838fe7f022..efa3f5b0bcc=20100644=0A---=20a/doc/src/sgml/regress.sgml=0A= +++=20b/doc/src/sgml/regress.sgml=0A@@=20-263,6=20+263,17=20@@=20make=20= check-world=20PG_TEST_EXTRA=3D'kerberos=20ldap=20ssl=20load_balance=20= libpq_encryption'=0A=20=0A=20=20=20=20The=20following=20= values=20are=20currently=20supported:=0A=20=20=20=20=0A+=20= =20=20=20=0A+=20=20=20=20=20= checksum_delays=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Runs=20= additional=20tests=20for=20enabling=20data=20checksums=20which=20inject=20= delays=0A+=20=20=20=20=20=20=20and=20re-tries=20in=20the=20processing.=20= =20Some=20of=20these=20test=20suites=20requires=0A+=20=20=20=20=20=20=20= injection=20points=20enabled=20in=20the=20installation.=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=0A= +=0A=20=20=20=20=20=0A=20=20=20=20=20=20= kerberos=0A=20=20=20=20=20=20=0A= diff=20--git=20a/doc/src/sgml/wal.sgml=20b/doc/src/sgml/wal.sgml=0Aindex=20= f3b86b26be9..0ada90ca0b1=20100644=0A---=20a/doc/src/sgml/wal.sgml=0A+++=20= b/doc/src/sgml/wal.sgml=0A@@=20-246,9=20+246,10=20@@=0A=20=20=20=0A= =20=20=20=20Checksums=20can=20be=20disabled=20when=20the=20cluster=20is=20= initialized=20using=20initdb.=0A-=20=20=20They=20can=20also=20be=20enabled=20or=20disabled=20at=20= a=20later=20time=20as=20an=20offline=0A-=20=20=20operation.=20Data=20= checksums=20are=20enabled=20or=20disabled=20at=20the=20full=20cluster=0A= -=20=20=20level,=20and=20cannot=20be=20specified=20individually=20for=20= databases=20or=20tables.=0A+=20=20=20They=20can=20also=20be=20enabled=20= or=20disabled=20at=20a=20later=20time=20either=20as=20an=20offline=0A+=20= =20=20operation=20or=20online=20in=20a=20running=20cluster=20allowing=20= concurrent=20access.=20Data=0A+=20=20=20checksums=20are=20enabled=20or=20= disabled=20at=20the=20full=20cluster=20level,=20and=20cannot=20be=0A+=20=20= =20specified=20individually=20for=20databases=20or=20tables.=0A=20=20=20= =0A=20=0A=20=20=20=0A@@=20-265,7=20+266,7=20@@=0A=20=20=20= =0A=20=0A=20=20=20= =0A-=20=20=20Off-line=20Enabling=20of=20Checksums=0A+=20=20= =20Offline=20Enabling=20of=20Checksums=0A=20=0A=20=20=20=20= =0A=20=20=20=20=20The=20pg_checksums= =0A@@=20-274,6=20+275,56=20@@=0A=20=20=20=20=0A=20=0A=20=20=20= =0A+=0A+=20=20=0A= +=20=20=20Online=20Enabling=20of=20Checksums=0A+=0A+=20=20= =20=0A+=20=20=20=20Checksums=20can=20be=20enabled=20or=20disabled=20= online,=20by=20calling=20the=20appropriate=0A+=20=20=20=20functions.=0A+=20=20=20= =0A+=0A+=20=20=20=0A+=20=20=20=20Enabling=20checksums=20= will=20put=20the=20cluster=20checksum=20mode=20in=0A+=20=20=20=20= inprogress-on=20mode.=20=20During=20this=20time,=20= checksums=20will=20be=0A+=20=20=20=20written=20but=20not=20verified.=20= In=20addition=20to=20this,=20a=20background=20worker=20process=0A+=20=20=20= =20is=20started=20that=20enables=20checksums=20on=20all=20existing=20= data=20in=20the=20cluster.=20Once=0A+=20=20=20=20this=20worker=20has=20= completed=20processing=20all=20databases=20in=20the=20cluster,=20the=0A+=20= =20=20=20checksum=20mode=20will=20automatically=20switch=20to=20= on.=20The=0A+=20=20=20=20processing=20will=20consume=20= two=20background=20worker=20processes,=20make=20sure=20that=0A+=20=20=20=20= max_worker_processes=20allows=20for=20at=20least=20= two=20more=0A+=20=20=20=20additional=20processes.=0A+=20=20=20=0A= +=0A+=20=20=20=0A+=20=20=20=20The=20process=20will=20initially=20= wait=20for=20all=20open=20transactions=20to=20finish=20before=0A+=20=20=20= =20it=20starts,=20so=20that=20it=20can=20be=20certain=20that=20there=20= are=20no=20tables=20that=20have=20been=0A+=20=20=20=20created=20inside=20= a=20transaction=20that=20has=20not=20committed=20yet=20and=20thus=20= would=20not=0A+=20=20=20=20be=20visible=20to=20the=20process=20enabling=20= checksums.=20It=20will=20also,=20for=20each=20database,=0A+=20=20=20=20= wait=20for=20all=20pre-existing=20temporary=20tables=20to=20get=20= removed=20before=20it=20finishes.=0A+=20=20=20=20If=20long-lived=20= temporary=20tables=20are=20used=20in=20the=20application=20it=20may=20be=20= necessary=0A+=20=20=20=20to=20terminate=20these=20application=20= connections=20to=20allow=20the=20process=20to=20complete.=0A+=20=20=20= =0A+=0A+=20=20=20=0A+=20=20=20=20If=20the=20cluster=20is=20= stopped=20while=20in=20inprogress-on=20mode,=20for=0A= +=20=20=20=20any=20reason,=20then=20this=20process=20must=20be=20= restarted=20manually.=20To=20do=20this,=0A+=20=20=20=20re-execute=20the=20= function=20pg_enable_data_checksums()=0A+=20=20=20=20= once=20the=20cluster=20has=20been=20restarted.=20The=20process=20will=20= start=20over,=20there=20is=0A+=20=20=20=20no=20support=20for=20resuming=20= work=20from=20where=20it=20was=20interrupted.=0A+=20=20=20=0A+=0A= +=20=20=20=0A+=20=20=20=20=0A+=20=20=20=20=20Enabling=20= checksums=20can=20cause=20significant=20I/O=20to=20the=20system,=20as=20= most=20of=20the=0A+=20=20=20=20=20database=20pages=20will=20need=20to=20= be=20rewritten,=20and=20will=20be=20written=20both=20to=20the=0A+=20=20=20= =20=20data=20files=20and=20the=20WAL.=20The=20impact=20may=20be=20= limited=20by=20throttling=20using=20the=0A+=20=20=20=20=20= cost_delay=20and=20= cost_limit=0A+=20=20=20=20=20parameters=20of=20= the=20pg_enable_data_checksums=20function.=0A+=20=20= =20=20=0A+=20=20=20=0A+=0A+=20=20=0A=20=20=0A= =20=0A=20=20=20=0Adiff=20--git=20= a/src/backend/access/rmgrdesc/xlogdesc.c=20= b/src/backend/access/rmgrdesc/xlogdesc.c=0Aindex=20= cd6c2a2f650..c50d654db30=20100644=0A---=20= a/src/backend/access/rmgrdesc/xlogdesc.c=0A+++=20= b/src/backend/access/rmgrdesc/xlogdesc.c=0A@@=20-18,6=20+18,7=20@@=0A=20= #include=20"access/xlog.h"=0A=20#include=20"access/xlog_internal.h"=0A=20= #include=20"catalog/pg_control.h"=0A+#include=20"storage/bufpage.h"=0A=20= #include=20"utils/guc.h"=0A=20#include=20"utils/timestamp.h"=0A=20=0A@@=20= -167,6=20+168,26=20@@=20xlog_desc(StringInfo=20buf,=20XLogReaderState=20= *record)=0A=20=09=09memcpy(&wal_level,=20rec,=20sizeof(int));=0A=20=09=09= appendStringInfo(buf,=20"wal_level=20%s",=20= get_wal_level_string(wal_level));=0A=20=09}=0A+=09else=20if=20(info=20=3D=3D= =20XLOG_CHECKSUMS)=0A+=09{=0A+=09=09xl_checksum_state=20xlrec;=0A+=0A+=09= =09memcpy(&xlrec,=20rec,=20sizeof(xl_checksum_state));=0A+=09=09switch=20= (xlrec.new_checksumtype)=0A+=09=09{=0A+=09=09=09case=20= PG_DATA_CHECKSUM_VERSION:=0A+=09=09=09=09appendStringInfoString(buf,=20= "on");=0A+=09=09=09=09break;=0A+=09=09=09case=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION:=0A+=09=09=09=09= appendStringInfoString(buf,=20"inprogress-off");=0A+=09=09=09=09break;=0A= +=09=09=09case=20PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION:=0A+=09=09=09=09= appendStringInfoString(buf,=20"inprogress-on");=0A+=09=09=09=09break;=0A= +=09=09=09default:=0A+=09=09=09=09appendStringInfoString(buf,=20"off");=0A= +=09=09}=0A+=09}=0A=20}=0A=20=0A=20const=20char=20*=0A@@=20-218,6=20= +239,9=20@@=20xlog_identify(uint8=20info)=0A=20=09=09case=20= XLOG_CHECKPOINT_REDO:=0A=20=09=09=09id=20=3D=20"CHECKPOINT_REDO";=0A=20=09= =09=09break;=0A+=09=09case=20XLOG_CHECKSUMS:=0A+=09=09=09id=20=3D=20= "CHECKSUMS";=0A+=09=09=09break;=0A=20=09}=0A=20=0A=20=09return=20id;=0A= diff=20--git=20a/src/backend/access/transam/xlog.c=20= b/src/backend/access/transam/xlog.c=0Aindex=20e8909406686..914359c8a65=20= 100644=0A---=20a/src/backend/access/transam/xlog.c=0A+++=20= b/src/backend/access/transam/xlog.c=0A@@=20-561,6=20+561,9=20@@=20= typedef=20struct=20XLogCtlData=0A=20=09=20*/=0A=20=09XLogRecPtr=09= lastFpwDisableRecPtr;=0A=20=0A+=09/*=20last=20data_checksum_version=20= we've=20seen=20*/=0A+=09uint32=09=09data_checksum_version;=0A+=0A=20=09= slock_t=09=09info_lck;=09=09/*=20locks=20shared=20variables=20shown=20= above=20*/=0A=20}=20XLogCtlData;=0A=20=0A@@=20-658,6=20+661,36=20@@=20= static=20XLogRecPtr=20LocalMinRecoveryPoint;=0A=20static=20TimeLineID=20= LocalMinRecoveryPointTLI;=0A=20static=20bool=20updateMinRecoveryPoint=20= =3D=20true;=0A=20=0A+/*=0A+=20*=20Local=20state=20fror=20Controlfile=20= data_checksum_version.=20=20After=20initialization=0A+=20*=20this=20is=20= only=20updated=20when=20absorbing=20a=20procsignal=20barrier=20during=20= interrupt=0A+=20*=20processing.=20=20The=20reason=20for=20keeping=20a=20= copy=20in=20backend-private=20memory=20is=20to=0A+=20*=20avoid=20locking=20= for=20interrogating=20checksum=20state.=20=20Possible=20values=20are=20= the=0A+=20*=20checksum=20versions=20defined=20in=20storage/bufpage.h=20= as=20well=20as=20zero=20when=20data=0A+=20*=20checksums=20are=20= disabled.=0A+=20*/=0A+static=20uint32=20LocalDataChecksumVersion=20=3D=20= 0;=0A+=0A+/*=0A+=20*=20Flag=20to=20remember=20if=20the=20= procsignalbarrier=20being=20absorbed=20for=20checksums=20is=0A+=20*=20= the=20first=20one.=20=20The=20first=20procsignalbarrier=20can=20in=20= rare=20cases=20be=20for=20the=0A+=20*=20state=20we've=20initialized,=20= i.e.=20a=20duplicate.=20=20This=20may=20happen=20for=20any=0A+=20*=20= data_checksum_version=20value,=20but=20for=20PG_DATA_CHECKSUM_ON_VERSION=20= this=20would=0A+=20*=20trigger=20an=20assert=20failure=20(this=20is=20= the=20only=20transition=20with=20an=20assert)=20when=0A+=20*=20= processing=20the=20barrier.=20=20This=20may=20happen=20if=20the=20= process=20is=20spawned=20between=0A+=20*=20the=20update=20of=20= XLogCtl->data_checksum_version=20and=20the=20barrier=20being=20emitted.=0A= +=20*=20This=20can=20only=20happen=20on=20the=20very=20first=20barrier=20= so=20mark=20that=20with=20this=20flag.=0A+=20*/=0A+static=20bool=20= InitialDataChecksumTransition=20=3D=20true;=0A+=0A+/*=0A+=20*=20Variable=20= backing=20the=20GUC,=20keep=20it=20in=20sync=20with=20= LocalDataChecksumVersion.=0A+=20*=20See=20SetLocalDataChecksumVersion().=0A= +=20*/=0A+int=09=09=09data_checksums=20=3D=200;=0A+=0A+static=20void=20= SetLocalDataChecksumVersion(uint32=20data_checksum_version);=0A+=0A=20/*=20= For=20WALInsertLockAcquire/Release=20functions=20*/=0A=20static=20int=09= MyLockNo=20=3D=200;=0A=20static=20bool=20holdingAllLocks=20=3D=20false;=0A= @@=20-726,6=20+759,8=20@@=20static=20void=20= WALInsertLockAcquireExclusive(void);=0A=20static=20void=20= WALInsertLockRelease(void);=0A=20static=20void=20= WALInsertLockUpdateInsertingAt(XLogRecPtr=20insertingAt);=0A=20=0A= +static=20void=20XLogChecksums(uint32=20new_type);=0A+=0A=20/*=0A=20=20*=20= Insert=20an=20XLOG=20record=20represented=20by=20an=20= already-constructed=20chain=20of=20data=0A=20=20*=20chunks.=20=20This=20= is=20a=20low-level=20routine;=20to=20construct=20the=20WAL=20record=20= header=0A@@=20-839,9=20+874,10=20@@=20XLogInsertRecord(XLogRecData=20= *rdata,=0A=20=09=09=20*=20only=20happen=20just=20after=20a=20checkpoint,=20= so=20it's=20better=20to=20be=20slow=20in=0A=20=09=09=20*=20this=20case=20= and=20fast=20otherwise.=0A=20=09=09=20*=0A-=09=09=20*=20Also=20check=20= to=20see=20if=20fullPageWrites=20was=20just=20turned=20on=20or=20there's=20= a=0A-=09=09=20*=20running=20backup=20(which=20forces=20full-page=20= writes);=20if=20we=20weren't=0A-=09=09=20*=20already=20doing=20full-page=20= writes=20then=20go=20back=20and=20recompute.=0A+=09=09=20*=20Also=20= check=20to=20see=20if=20fullPageWrites=20was=20just=20turned=20on,=20= there's=20a=0A+=09=09=20*=20running=20backup=20or=20if=20checksums=20are=20= enabled=20(all=20of=20which=20forces=0A+=09=09=20*=20full-page=20= writes);=20if=20we=20weren't=20already=20doing=20full-page=20writes=0A+=09= =09=20*=20then=20go=20back=20and=20recompute.=0A=20=09=09=20*=0A=20=09=09= =20*=20If=20we=20aren't=20doing=20full-page=20writes=20then=20RedoRecPtr=20= doesn't=0A=20=09=09=20*=20actually=20affect=20the=20contents=20of=20the=20= XLOG=20record,=20so=20we'll=20update=0A@@=20-854,7=20+890,9=20@@=20= XLogInsertRecord(XLogRecData=20*rdata,=0A=20=09=09=09Assert(RedoRecPtr=20= <=20Insert->RedoRecPtr);=0A=20=09=09=09RedoRecPtr=20=3D=20= Insert->RedoRecPtr;=0A=20=09=09}=0A-=09=09doPageWrites=20=3D=20= (Insert->fullPageWrites=20||=20Insert->runningBackups=20>=200);=0A+=09=09= doPageWrites=20=3D=20(Insert->fullPageWrites=20||=0A+=09=09=09=09=09=09= Insert->runningBackups=20>=200=20||=0A+=09=09=09=09=09=09= DataChecksumsNeedWrite());=0A=20=0A=20=09=09if=20(doPageWrites=20&&=0A=20= =09=09=09(!prevDoPageWrites=20||=0A@@=20-4362,6=20+4400,12=20@@=20= InitControlFile(uint64=20sysidentifier,=20uint32=20= data_checksum_version)=0A=20=09ControlFile->wal_log_hints=20=3D=20= wal_log_hints;=0A=20=09ControlFile->track_commit_timestamp=20=3D=20= track_commit_timestamp;=0A=20=09ControlFile->data_checksum_version=20=3D=20= data_checksum_version;=0A+=0A+=09/*=0A+=09=20*=20Set=20the=20= data_checksum_version=20value=20into=20XLogCtl,=20which=20is=20where=20= all=0A+=09=20*=20processes=20get=20the=20current=20value=20from.=20= (Maybe=20it=20should=20go=20just=20there?)=0A+=09=20*/=0A+=09= XLogCtl->data_checksum_version=20=3D=20data_checksum_version;=0A=20}=0A=20= =0A=20static=20void=0A@@=20-4685,10=20+4729,6=20@@=20= ReadControlFile(void)=0A=20=09=09(SizeOfXLogLongPHD=20-=20= SizeOfXLogShortPHD);=0A=20=0A=20=09CalculateCheckpointSegments();=0A-=0A= -=09/*=20Make=20the=20initdb=20settings=20visible=20as=20GUC=20= variables,=20too=20*/=0A-=09SetConfigOption("data_checksums",=20= DataChecksumsEnabled()=20?=20"yes"=20:=20"no",=0A-=09=09=09=09=09= PGC_INTERNAL,=20PGC_S_DYNAMIC_DEFAULT);=0A=20}=0A=20=0A=20/*=0A@@=20= -4722,13=20+4762,374=20@@=20GetMockAuthenticationNonce(void)=0A=20}=0A=20= =0A=20/*=0A-=20*=20Are=20checksums=20enabled=20for=20data=20pages?=0A+=20= *=20DataChecksumsNeedWrite=0A+=20*=09=09Returns=20whether=20data=20= checksums=20must=20be=20written=20or=20not=0A+=20*=0A+=20*=20Returns=20= true=20iff=20data=20checksums=20are=20enabled=20or=20are=20in=20the=20= process=20of=20being=0A+=20*=20enabled.=20During=20"inprogress-on"=20and=20= "inprogress-off"=20states=20checksums=20must=0A+=20*=20be=20written=20= even=20though=20they=20are=20not=20verified=20(see=20= datachecksumsworker.c=20for=0A+=20*=20a=20longer=20discussion).=0A+=20*=0A= +=20*=20This=20function=20is=20intended=20for=20callsites=20which=20are=20= about=20to=20write=20a=20data=20page=0A+=20*=20to=20storage,=20and=20= need=20to=20know=20whether=20to=20re-calculate=20the=20checksum=20for=20= the=0A+=20*=20page=20header.=20Calling=20this=20function=20must=20be=20= performed=20as=20close=20to=20the=20write=0A+=20*=20operation=20as=20= possible=20to=20keep=20the=20critical=20section=20short.=0A+=20*/=0A= +bool=0A+DataChecksumsNeedWrite(void)=0A+{=0A+=09return=20= (LocalDataChecksumVersion=20=3D=3D=20PG_DATA_CHECKSUM_VERSION=20||=0A+=09= =09=09LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION=20||=0A+=09=09=09= LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION);=0A+}=0A+=0A+/*=0A+=20*=20= DataChecksumsNeedVerify=0A+=20*=09=09Returns=20whether=20data=20= checksums=20must=20be=20verified=20or=20not=0A+=20*=0A+=20*=20Data=20= checksums=20are=20only=20verified=20if=20they=20are=20fully=20enabled=20= in=20the=20cluster.=0A+=20*=20During=20the=20"inprogress-on"=20and=20= "inprogress-off"=20states=20they=20are=20only=0A+=20*=20updated,=20not=20= verified=20(see=20datachecksumsworker.c=20for=20a=20longer=20= discussion).=0A+=20*=0A+=20*=20This=20function=20is=20intended=20for=20= callsites=20which=20have=20read=20data=20and=20are=20about=0A+=20*=20to=20= perform=20checksum=20validation=20based=20on=20the=20result=20of=20this.=20= =20Calling=20this=0A+=20*=20function=20must=20be=20performed=20as=20= close=20to=20the=20validation=20call=20as=20possible=20to=0A+=20*=20keep=20= the=20critical=20section=20short.=20This=20is=20in=20order=20to=20= protect=20against=20time=20of=0A+=20*=20check/time=20of=20use=20= situations=20around=20data=20checksum=20validation.=0A+=20*/=0A+bool=0A= +DataChecksumsNeedVerify(void)=0A+{=0A+=09return=20= (LocalDataChecksumVersion=20=3D=3D=20PG_DATA_CHECKSUM_VERSION);=0A+}=0A+=0A= +/*=0A+=20*=20DataChecksumsOnInProgress=0A+=20*=09=09Returns=20whether=20= data=20checksums=20are=20being=20enabled=0A+=20*=0A+=20*=20Most=20= operations=20don't=20need=20to=20worry=20about=20the=20"inprogress"=20= states,=20and=0A+=20*=20should=20use=20DataChecksumsNeedVerify()=20or=20= DataChecksumsNeedWrite().=20The=0A+=20*=20"inprogress-on"=20state=20for=20= enabling=20checksums=20is=20used=20when=20the=20checksum=0A+=20*=20= worker=20is=20setting=20checksums=20on=20all=20pages,=20it=20can=20thus=20= be=20used=20to=20check=20for=0A+=20*=20aborted=20checksum=20processing=20= which=20need=20to=20be=20restarted.=0A+=20*/=0A+inline=20bool=0A= +DataChecksumsOnInProgress(void)=0A+{=0A+=09return=20= (LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION);=0A+}=0A+=0A+/*=0A+=20*=20= DataChecksumsOffInProgress=0A+=20*=09=09Returns=20whether=20data=20= checksums=20are=20being=20disabled=0A+=20*=0A+=20*=20The=20= "inprogress-off"=20state=20for=20disabling=20checksums=20is=20used=20for=20= when=20the=0A+=20*=20worker=20resets=20the=20catalog=20state.=20=20= DataChecksumsNeedVerify()=20or=0A+=20*=20DataChecksumsNeedWrite()=20= should=20be=20used=20for=20deciding=20whether=20to=20read/write=0A+=20*=20= checksums.=0A=20=20*/=0A=20bool=0A-DataChecksumsEnabled(void)=0A= +DataChecksumsOffInProgress(void)=0A+{=0A+=09return=20= (LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION);=0A+}=0A+=0A+/*=0A+=20*=20= SetDataChecksumsOnInProgress=0A+=20*=09=09Sets=20the=20data=20checksum=20= state=20to=20"inprogress-on"=20to=20enable=20checksums=0A+=20*=0A+=20*=20= To=20start=20the=20process=20of=20enabling=20data=20checksums=20in=20a=20= running=20cluster=20the=0A+=20*=20data_checksum_version=20state=20must=20= be=20changed=20to=20"inprogress-on".=20See=0A+=20*=20SetDataChecksumsOn=20= below=20for=20a=20description=20on=20how=20this=20state=20change=20= works.=0A+=20*=20This=20function=20blocks=20until=20all=20backends=20in=20= the=20cluster=20have=20acknowledged=20the=0A+=20*=20state=20transition.=0A= +=20*/=0A+void=0A+SetDataChecksumsOnInProgress(void)=0A+{=0A+=09uint64=09= =09barrier;=0A+=0A+=09Assert(ControlFile=20!=3D=20NULL);=0A+=0A+=09/*=0A= +=09=20*=20The=20state=20transition=20is=20performed=20in=20a=20critical=20= section=20with=0A+=09=20*=20checkpoints=20held=20off=20to=20provide=20= crash=20safety.=0A+=09=20*/=0A+=09MyProc->delayChkptFlags=20|=3D=20= DELAY_CHKPT_START;=0A+=09START_CRIT_SECTION();=0A+=0A+=09= XLogChecksums(PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION);=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= XLogCtl->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON);=0A+=0A= +=09END_CRIT_SECTION();=0A+=09MyProc->delayChkptFlags=20&=3D=20= ~DELAY_CHKPT_START;=0A+=0A+=09/*=0A+=09=20*=20Await=20state=20change=20= in=20all=20backends=20to=20ensure=20that=20all=20backends=20are=20in=0A+=09= =20*=20"inprogress-on".=20Once=20done=20we=20know=20that=20all=20= backends=20are=20writing=20data=0A+=09=20*=20checksums.=0A+=09=20*/=0A+=09= WaitForProcSignalBarrier(barrier);=0A+}=0A+=0A+/*=0A+=20*=20= SetDataChecksumsOn=0A+=20*=09=09Enables=20data=20checksums=20= cluster-wide=0A+=20*=0A+=20*=20Enabling=20data=20checksums=20is=20= performed=20using=20two=20barriers,=20the=20first=20one=20to=0A+=20*=20= set=20the=20state=20to=20"inprogress-on"=20(done=20by=20= SetDataChecksumsOnInProgress())=0A+=20*=20and=20the=20second=20one=20to=20= set=20the=20state=20to=20"on"=20(done=20here).=20Below=20is=20a=20short=0A= +=20*=20description=20of=20the=20processing,=20a=20more=20detailed=20= write-up=20can=20be=20found=20in=0A+=20*=20datachecksumsworker.c.=0A+=20= *=0A+=20*=20To=20start=20the=20process=20of=20enabling=20data=20= checksums=20in=20a=20running=20cluster=20the=0A+=20*=20= data_checksum_version=20state=20must=20be=20changed=20to=20= "inprogress-on".=20=20This=20state=0A+=20*=20requires=20data=20checksums=20= to=20be=20written=20but=20not=20verified.=20This=20ensures=20that=0A+=20= *=20all=20data=20pages=20can=20be=20checksummed=20without=20the=20risk=20= of=20false=20negatives=20in=0A+=20*=20validation=20during=20the=20= process.=20=20When=20all=20existing=20pages=20are=20guaranteed=20to=0A+=20= *=20have=20checksums,=20and=20all=20new=20pages=20will=20be=20initiated=20= with=20checksums,=20the=0A+=20*=20state=20can=20be=20changed=20to=20= "on".=20Once=20the=20state=20is=20"on"=20checksums=20will=20be=20both=0A= +=20*=20written=20and=20verified.=20See=20datachecksumsworker.c=20for=20= a=20longer=20discussion=20on=0A+=20*=20how=20data=20checksums=20can=20be=20= enabled=20in=20a=20running=20cluster.=0A+=20*=0A+=20*=20This=20function=20= blocks=20until=20all=20backends=20in=20the=20cluster=20have=20= acknowledged=20the=0A+=20*=20state=20transition.=0A+=20*/=0A+void=0A= +SetDataChecksumsOn(void)=0A=20{=0A+=09uint64=09=09barrier;=0A+=0A=20=09= Assert(ControlFile=20!=3D=20NULL);=0A-=09return=20= (ControlFile->data_checksum_version=20>=200);=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=0A+=09/*=0A+=09=20*=20The=20= only=20allowed=20state=20transition=20to=20"on"=20is=20from=20= "inprogress-on"=20since=0A+=09=20*=20that=20state=20ensures=20that=20all=20= pages=20will=20have=20data=20checksums=20written.=0A+=09=20*/=0A+=09if=20= (XLogCtl->data_checksum_version=20!=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION)=0A+=09{=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=09=09elog(ERROR,=20"checksums=20= not=20in=20\"inprogress-on\"=20mode");=0A+=09}=0A+=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09MyProc->delayChkptFlags=20= |=3D=20DELAY_CHKPT_START;=0A+=09= INJECTION_POINT("datachecksums-enable-checksums-delay",=20NULL);=0A+=09= START_CRIT_SECTION();=0A+=0A+=09XLogChecksums(PG_DATA_CHECKSUM_VERSION);=0A= +=0A+=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= XLogCtl->data_checksum_version=20=3D=20PG_DATA_CHECKSUM_VERSION;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_ON);=0A+=0A+=09= END_CRIT_SECTION();=0A+=09MyProc->delayChkptFlags=20&=3D=20= ~DELAY_CHKPT_START;=0A+=0A+=09/*=0A+=09=20*=20Await=20state=20transition=20= of=20"on"=20in=20all=20backends.=20When=20done=20we=20know=20that=0A+=09=20= *=20data=20checksums=20are=20enabled=20in=20all=20backends=20and=20data=20= checksums=20are=20both=0A+=09=20*=20written=20and=20verified.=0A+=09=20= */=0A+=09WaitForProcSignalBarrier(barrier);=0A+}=0A+=0A+/*=0A+=20*=20= SetDataChecksumsOff=0A+=20*=09=09Disables=20data=20checksums=20= cluster-wide=0A+=20*=0A+=20*=20Disabling=20data=20checksums=20must=20be=20= performed=20with=20two=20sets=20of=20barriers,=20each=0A+=20*=20carrying=20= a=20different=20state.=20The=20state=20is=20first=20set=20to=20= "inprogress-off"=0A+=20*=20during=20which=20checksums=20are=20still=20= written=20but=20not=20verified.=20This=20ensures=20that=0A+=20*=20= backends=20which=20have=20yet=20to=20observe=20the=20state=20change=20= from=20"on"=20won't=20get=0A+=20*=20validation=20errors=20on=20= concurrently=20modified=20pages.=20Once=20all=20backends=20have=0A+=20*=20= changed=20to=20"inprogress-off",=20the=20barrier=20for=20moving=20to=20= "off"=20can=20be=20emitted.=0A+=20*=20This=20function=20blocks=20until=20= all=20backends=20in=20the=20cluster=20have=20acknowledged=20the=0A+=20*=20= state=20transition.=0A+=20*/=0A+void=0A+SetDataChecksumsOff(void)=0A+{=0A= +=09uint64=09=09barrier;=0A+=0A+=09Assert(ControlFile);=0A+=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=0A+=09/*=20If=20data=20= checksums=20are=20already=20disabled=20there=20is=20nothing=20to=20do=20= */=0A+=09if=20(XLogCtl->data_checksum_version=20=3D=3D=200)=0A+=09{=0A+=09= =09SpinLockRelease(&XLogCtl->info_lck);=0A+=09=09return;=0A+=09}=0A+=0A+=09= /*=0A+=09=20*=20If=20data=20checksums=20are=20currently=20enabled=20we=20= first=20transition=20to=20the=0A+=09=20*=20"inprogress-off"=20state=20= during=20which=20backends=20continue=20to=20write=0A+=09=20*=20checksums=20= without=20verifying=20them.=20When=20all=20backends=20are=20in=0A+=09=20= *=20"inprogress-off"=20the=20next=20transition=20to=20"off"=20can=20be=20= performed,=20after=0A+=09=20*=20which=20all=20data=20checksum=20= processing=20is=20disabled.=0A+=09=20*/=0A+=09if=20= (XLogCtl->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_VERSION)=0A+=09= {=0A+=09=09SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09= MyProc->delayChkptFlags=20|=3D=20DELAY_CHKPT_START;=0A+=09=09= START_CRIT_SECTION();=0A+=0A+=09=09= XLogChecksums(PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION);=0A+=0A+=09=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION;=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF);=0A+=0A= +=09=09END_CRIT_SECTION();=0A+=09=09MyProc->delayChkptFlags=20&=3D=20= ~DELAY_CHKPT_START;=0A+=0A+=09=09/*=0A+=09=09=20*=20Update=20local=20= state=20in=20all=20backends=20to=20ensure=20that=20any=20backend=20in=0A= +=09=09=20*=20"on"=20state=20is=20changed=20to=20"inprogress-off".=0A+=09= =09=20*/=0A+=09=09WaitForProcSignalBarrier(barrier);=0A+=0A+=09=09/*=0A+=09= =09=20*=20At=20this=20point=20we=20know=20that=20no=20backends=20are=20= verifying=20data=20checksums=0A+=09=09=20*=20during=20reading.=20Next,=20= we=20can=20safely=20move=20to=20state=20"off"=20to=20also=0A+=09=09=20*=20= stop=20writing=20checksums.=0A+=09=09=20*/=0A+=09}=0A+=09else=0A+=09{=0A= +=09=09/*=0A+=09=09=20*=20Ending=20up=20here=20implies=20that=20the=20= checksums=20state=20is=20"inprogress-on"=0A+=09=09=20*=20or=20= "inprogress-off"=20and=20we=20can=20transition=20directly=20to=20"off"=20= from=0A+=09=09=20*=20there.=0A+=09=09=20*/=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20= Ensure=20that=20we=20don't=20incur=20a=20checkpoint=20during=20disabling=20= checksums.=0A+=09=20*/=0A+=09MyProc->delayChkptFlags=20|=3D=20= DELAY_CHKPT_START;=0A+=09START_CRIT_SECTION();=0A+=0A+=09= XLogChecksums(0);=0A+=0A+=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= XLogCtl->data_checksum_version=20=3D=200;=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_OFF);=0A+=0A+=09= END_CRIT_SECTION();=0A+=09MyProc->delayChkptFlags=20&=3D=20= ~DELAY_CHKPT_START;=0A+=0A+=09WaitForProcSignalBarrier(barrier);=0A+}=0A= +=0A+/*=0A+=20*=20ProcSignalBarrier=20absorption=20functions=20for=20= enabling=20and=20disabling=20data=0A+=20*=20checksums=20in=20a=20running=20= cluster.=20The=20procsignalbarriers=20are=20emitted=20in=20the=0A+=20*=20= SetDataChecksums*=20functions.=0A+=20*/=0A+bool=0A= +AbsorbChecksumsOnInProgressBarrier(void)=0A+{=0A+=09= Assert(LocalDataChecksumVersion=20!=3D=20PG_DATA_CHECKSUM_VERSION);=0A+=09= SetLocalDataChecksumVersion(PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION);=0A+=09= return=20true;=0A+}=0A+=0A+bool=0A+AbsorbChecksumsOnBarrier(void)=0A+{=0A= +=09/*=0A+=09=20*=20If=20the=20process=20was=20spawned=20between=20= updating=20XLogCtl=20and=20emitting=20the=0A+=09=20*=20barrier=20it=20= will=20have=20seen=20the=20updated=20value,=20so=20for=20the=20first=20= barrier=0A+=09=20*=20we=20accept=20both=20"on"=20and=20"inprogress-on".=0A= +=09=20*/=0A+=09Assert((LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION)=20||=0A+=09=09=20=20=20= (InitialDataChecksumTransition=20&&=0A+=09=09=09= (LocalDataChecksumVersion=20=3D=3D=20PG_DATA_CHECKSUM_VERSION)));=0A+=0A= +=09SetLocalDataChecksumVersion(PG_DATA_CHECKSUM_VERSION);=0A+=09= InitialDataChecksumTransition=20=3D=20false;=0A+=09return=20true;=0A+}=0A= +=0A+bool=0A+AbsorbChecksumsOffInProgressBarrier(void)=0A+{=0A+=09= Assert(LocalDataChecksumVersion=20=3D=3D=20PG_DATA_CHECKSUM_VERSION);=0A= +=09= SetLocalDataChecksumVersion(PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION);=0A= +=09return=20true;=0A+}=0A+=0A+bool=0A+AbsorbChecksumsOffBarrier(void)=0A= +{=0A+=09/*=0A+=09=20*=20We=20should=20never=20get=20here=20directly=20= from=20a=20cluster=20with=20data=20checksums=0A+=09=20*=20enabled,=20an=20= inprogress=20state=20should=20be=20in=20between.=20=20When=20there=20are=20= no=0A+=09=20*=20failures=20the=20inprogress-off=20state=20should=20= preceed,=20but=20in=20case=20of=20error=0A+=09=20*=20in=20processing=20= we=20can=20also=20reach=20here=20from=20the=20inprogress-on=20state.=0A+=09= =20*/=0A+=09Assert((LocalDataChecksumVersion=20!=3D=20= PG_DATA_CHECKSUM_VERSION)=20&&=0A+=09=09=20=20=20= (LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION=20||=0A+=09=09=09= LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION));=0A+=09= SetLocalDataChecksumVersion(PG_DATA_CHECKSUM_OFF);=0A+=09return=20true;=0A= +}=0A+=0A+/*=0A+=20*=20InitLocalControlData=0A+=20*=0A+=20*=20Set=20up=20= backend=20local=20caches=20of=20controldata=20variables=20which=20may=20= change=20at=0A+=20*=20any=20point=20during=20runtime=20and=20thus=20= require=20special=20cased=20locking.=20So=20far=0A+=20*=20this=20only=20= applies=20to=20data_checksum_version,=20but=20it's=20intended=20to=20be=20= general=0A+=20*=20purpose=20enough=20to=20handle=20future=20cases.=0A+=20= */=0A+void=0A+InitLocalDataChecksumVersion(void)=0A+{=0A+=09= SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= SetLocalDataChecksumVersion(XLogCtl->data_checksum_version);=0A+=09= SpinLockRelease(&XLogCtl->info_lck);=0A+}=0A+=0A+void=0A= +SetLocalDataChecksumVersion(uint32=20data_checksum_version)=0A+{=0A+=09= LocalDataChecksumVersion=20=3D=20data_checksum_version;=0A+=0A+=09= data_checksums=20=3D=20data_checksum_version;=0A+}=0A+=0A+/*=20guc=20= hook=20*/=0A+const=20char=20*=0A+show_data_checksums(void)=0A+{=0A+=09if=20= (LocalDataChecksumVersion=20=3D=3D=20PG_DATA_CHECKSUM_VERSION)=0A+=09=09= return=20"on";=0A+=09else=20if=20(LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION)=0A+=09=09return=20= "inprogress-on";=0A+=09else=20if=20(LocalDataChecksumVersion=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION)=0A+=09=09return=20= "inprogress-off";=0A+=09else=0A+=09=09return=20"off";=0A=20}=0A=20=0A=20= /*=0A@@=20-5003,6=20+5404,7=20@@=20LocalProcessControlFile(bool=20reset)=0A= =20=09Assert(reset=20||=20ControlFile=20=3D=3D=20NULL);=0A=20=09= ControlFile=20=3D=20palloc(sizeof(ControlFileData));=0A=20=09= ReadControlFile();=0A+=09= SetLocalDataChecksumVersion(ControlFile->data_checksum_version);=0A=20}=0A= =20=0A=20/*=0A@@=20-5172,6=20+5574,11=20@@=20XLOGShmemInit(void)=0A=20=09= XLogCtl->InstallXLogFileSegmentActive=20=3D=20false;=0A=20=09= XLogCtl->WalWriterSleeping=20=3D=20false;=0A=20=0A+=09/*=20Use=20the=20= checksum=20info=20from=20control=20file=20*/=0A+=09= XLogCtl->data_checksum_version=20=3D=20= ControlFile->data_checksum_version;=0A+=0A+=09= SetLocalDataChecksumVersion(XLogCtl->data_checksum_version);=0A+=0A=20=09= SpinLockInit(&XLogCtl->Insert.insertpos_lck);=0A=20=09= SpinLockInit(&XLogCtl->info_lck);=0A=20=09= pg_atomic_init_u64(&XLogCtl->logInsertResult,=20InvalidXLogRecPtr);=0A@@=20= -6320,6=20+6727,47=20@@=20StartupXLOG(void)=0A=20=09= pfree(endOfRecoveryInfo->recoveryStopReason);=0A=20=09= pfree(endOfRecoveryInfo);=0A=20=0A+=09/*=0A+=09=20*=20If=20we=20reach=20= this=20point=20with=20checksums=20in=20the=20state=20inprogress-on,=20it=0A= +=09=20*=20means=20that=20data=20checksums=20were=20in=20the=20process=20= of=20being=20enabled=20when=20the=0A+=09=20*=20cluster=20shut=20down.=20= Since=20processing=20didn't=20finish,=20the=20operation=20will=0A+=09=20= *=20have=20to=20be=20restarted=20from=20scratch=20since=20there=20is=20= no=20capability=20to=0A+=09=20*=20continue=20where=20it=20was=20when=20= the=20cluster=20shut=20down.=20Thus,=20revert=20the=0A+=09=20*=20state=20= back=20to=20off,=20and=20inform=20the=20user=20with=20a=20warning=20= message.=20Being=0A+=09=20*=20able=20to=20restart=20processing=20is=20a=20= TODO,=20but=20it=20wouldn't=20be=20possible=20to=0A+=09=20*=20restart=20= here=20since=20we=20cannot=20launch=20a=20dynamic=20background=20worker=0A= +=09=20*=20directly=20from=20here=20(it=20has=20to=20be=20from=20a=20= regular=20backend).=0A+=09=20*/=0A+=09if=20= (XLogCtl->data_checksum_version=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION)=0A+=09{=0A+=09=09= XLogChecksums(0);=0A+=0A+=09=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= =09XLogCtl->data_checksum_version=20=3D=200;=0A+=09=09= SetLocalDataChecksumVersion(XLogCtl->data_checksum_version);=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09ereport(WARNING,=0A+=09= =09=09=09(errmsg("data=20checksums=20state=20has=20been=20set=20of=20= off"),=0A+=09=09=09=09=20errhint("If=20checksums=20were=20being=20= enabled=20during=20shutdown=20then=20processing=20must=20be=20manually=20= restarted.")));=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20If=20data=20checksums=20= were=20being=20disabled=20when=20the=20cluster=20was=20shut=20down,=0A+=09= =20*=20we=20know=20that=20we=20have=20a=20state=20where=20all=20backends=20= have=20stopped=20validating=0A+=09=20*=20checksums=20and=20we=20can=20= move=20to=20off=20instead=20of=20prompting=20the=20user=20to=0A+=09=20*=20= perform=20any=20action.=0A+=09=20*/=0A+=09if=20= (XLogCtl->data_checksum_version=20=3D=3D=20= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION)=0A+=09{=0A+=09=09= XLogChecksums(0);=0A+=0A+=09=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09= =09XLogCtl->data_checksum_version=20=3D=200;=0A+=09=09= SetLocalDataChecksumVersion(XLogCtl->data_checksum_version);=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=09}=0A+=0A=20=09/*=0A=20=09=20*=20= All=20done=20with=20end-of-recovery=20actions.=0A=20=09=20*=0A@@=20= -6611,7=20+7059,7=20@@=20GetRedoRecPtr(void)=0A=20=09XLogRecPtr=09ptr;=0A= =20=0A=20=09/*=0A-=09=20*=20The=20possibly=20not=20up-to-date=20copy=20= in=20XlogCtl=20is=20enough.=20Even=20if=20we=0A+=09=20*=20The=20possibly=20= not=20up-to-date=20copy=20in=20XLogCtl=20is=20enough.=20Even=20if=20we=0A= =20=09=20*=20grabbed=20a=20WAL=20insertion=20lock=20to=20read=20the=20= authoritative=20value=20in=0A=20=09=20*=20Insert->RedoRecPtr,=20someone=20= might=20update=20it=20just=20after=20we've=20released=0A=20=09=20*=20the=20= lock.=0A@@=20-7175,6=20+7623,12=20@@=20CreateCheckPoint(int=20flags)=0A=20= =09checkPoint.fullPageWrites=20=3D=20Insert->fullPageWrites;=0A=20=09= checkPoint.wal_level=20=3D=20wal_level;=0A=20=0A+=09/*=0A+=09=20*=20Get=20= the=20current=20data_checksum_version=20value=20from=20xlogctl,=20valid=20= at=20the=0A+=09=20*=20time=20of=20the=20checkpoint.=0A+=09=20*/=0A+=09= checkPoint.data_checksum_version=20=3D=20XLogCtl->data_checksum_version;=0A= +=0A=20=09if=20(shutdown)=0A=20=09{=0A=20=09=09XLogRecPtr=09curInsert=20= =3D=20XLogBytePosToRecPtr(Insert->CurrBytePos);=0A@@=20-7430,6=20+7884,9=20= @@=20CreateCheckPoint(int=20flags)=0A=20=09ControlFile->minRecoveryPoint=20= =3D=20InvalidXLogRecPtr;=0A=20=09ControlFile->minRecoveryPointTLI=20=3D=20= 0;=0A=20=0A+=09/*=20make=20sure=20we=20start=20with=20the=20checksum=20= version=20as=20of=20the=20checkpoint=20*/=0A+=09= ControlFile->data_checksum_version=20=3D=20= checkPoint.data_checksum_version;=0A+=0A=20=09/*=0A=20=09=20*=20Persist=20= unloggedLSN=20value.=20It's=20reset=20on=20crash=20recovery,=20so=20this=20= goes=0A=20=09=20*=20unused=20on=20non-shutdown=20checkpoints,=20but=20= seems=20useful=20to=20store=20it=20always=0A@@=20-7575,6=20+8032,10=20@@=20= CreateEndOfRecoveryRecord(void)=0A=20=09LWLockAcquire(ControlFileLock,=20= LW_EXCLUSIVE);=0A=20=09ControlFile->minRecoveryPoint=20=3D=20recptr;=0A=20= =09ControlFile->minRecoveryPointTLI=20=3D=20xlrec.ThisTimeLineID;=0A+=0A= +=09/*=20start=20with=20the=20latest=20checksum=20version=20(as=20of=20= the=20end=20of=20recovery)=20*/=0A+=09ControlFile->data_checksum_version=20= =3D=20XLogCtl->data_checksum_version;=0A+=0A=20=09UpdateControlFile();=0A= =20=09LWLockRelease(ControlFileLock);=0A=20=0A@@=20-7916,6=20+8377,10=20= @@=20CreateRestartPoint(int=20flags)=0A=20=09=09=09if=20(flags=20&=20= CHECKPOINT_IS_SHUTDOWN)=0A=20=09=09=09=09ControlFile->state=20=3D=20= DB_SHUTDOWNED_IN_RECOVERY;=0A=20=09=09}=0A+=0A+=09=09/*=20we=20shall=20= start=20with=20the=20latest=20checksum=20version=20*/=0A+=09=09= ControlFile->data_checksum_version=20=3D=20= lastCheckPoint.data_checksum_version;=0A+=0A=20=09=09= UpdateControlFile();=0A=20=09}=0A=20=09LWLockRelease(ControlFileLock);=0A= @@=20-8327,6=20+8792,24=20@@=20XLogReportParameters(void)=0A=20=09}=0A=20= }=0A=20=0A+/*=0A+=20*=20Log=20the=20new=20state=20of=20checksums=0A+=20= */=0A+static=20void=0A+XLogChecksums(uint32=20new_type)=0A+{=0A+=09= xl_checksum_state=20xlrec;=0A+=09XLogRecPtr=09recptr;=0A+=0A+=09= xlrec.new_checksumtype=20=3D=20new_type;=0A+=0A+=09XLogBeginInsert();=0A= +=09XLogRegisterData((char=20*)=20&xlrec,=20sizeof(xl_checksum_state));=0A= +=0A+=09recptr=20=3D=20XLogInsert(RM_XLOG_ID,=20XLOG_CHECKSUMS);=0A+=09= XLogFlush(recptr);=0A+}=0A+=0A=20/*=0A=20=20*=20Update=20= full_page_writes=20in=20shared=20memory,=20and=20write=20an=0A=20=20*=20= XLOG_FPW_CHANGE=20record=20if=20necessary.=0A@@=20-8745,6=20+9228,46=20= @@=20xlog_redo(XLogReaderState=20*record)=0A=20=09{=0A=20=09=09/*=20= nothing=20to=20do=20here,=20just=20for=20informational=20purposes=20*/=0A= =20=09}=0A+=09else=20if=20(info=20=3D=3D=20XLOG_CHECKSUMS)=0A+=09{=0A+=09= =09xl_checksum_state=20state;=0A+=09=09uint64=09=09barrier;=0A+=0A+=09=09= memcpy(&state,=20XLogRecGetData(record),=20sizeof(xl_checksum_state));=0A= +=0A+=09=09SpinLockAcquire(&XLogCtl->info_lck);=0A+=09=09= XLogCtl->data_checksum_version=20=3D=20state.new_checksumtype;=0A+=09=09= SpinLockRelease(&XLogCtl->info_lck);=0A+=0A+=09=09/*=0A+=09=09=20*=20= Block=20on=20a=20procsignalbarrier=20to=20await=20all=20processes=20= having=20seen=20the=0A+=09=09=20*=20change=20to=20checksum=20status.=20= Once=20the=20barrier=20has=20been=20passed=20we=20can=0A+=09=09=20*=20= initiate=20the=20corresponding=20processing.=0A+=09=09=20*/=0A+=09=09= switch=20(state.new_checksumtype)=0A+=09=09{=0A+=09=09=09case=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION:=0A+=09=09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON);=0A+=09=09= =09=09WaitForProcSignalBarrier(barrier);=0A+=09=09=09=09break;=0A+=0A+=09= =09=09case=20PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION:=0A+=09=09=09=09= barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF);=0A+=09= =09=09=09WaitForProcSignalBarrier(barrier);=0A+=09=09=09=09break;=0A+=0A= +=09=09=09case=20PG_DATA_CHECKSUM_VERSION:=0A+=09=09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_ON);=0A+=09=09=09=09= WaitForProcSignalBarrier(barrier);=0A+=09=09=09=09break;=0A+=0A+=09=09=09= default:=0A+=09=09=09=09Assert(state.new_checksumtype=20=3D=3D=200);=0A+=09= =09=09=09barrier=20=3D=20= EmitProcSignalBarrier(PROCSIGNAL_BARRIER_CHECKSUM_OFF);=0A+=09=09=09=09= WaitForProcSignalBarrier(barrier);=0A+=09=09=09=09break;=0A+=09=09}=0A+=09= }=0A=20}=0A=20=0A=20/*=0Adiff=20--git=20= a/src/backend/access/transam/xlogfuncs.c=20= b/src/backend/access/transam/xlogfuncs.c=0Aindex=20= 8c3090165f0..337932a89e5=20100644=0A---=20= a/src/backend/access/transam/xlogfuncs.c=0A+++=20= b/src/backend/access/transam/xlogfuncs.c=0A@@=20-26,6=20+26,7=20@@=0A=20= #include=20"funcapi.h"=0A=20#include=20"miscadmin.h"=0A=20#include=20= "pgstat.h"=0A+#include=20"postmaster/datachecksumsworker.h"=0A=20= #include=20"replication/walreceiver.h"=0A=20#include=20"storage/fd.h"=0A=20= #include=20"storage/latch.h"=0A@@=20-748,3=20+749,45=20@@=20= pg_promote(PG_FUNCTION_ARGS)=0A=20=09=09=09=09=09=09=20=20=20= wait_seconds)));=0A=20=09PG_RETURN_BOOL(false);=0A=20}=0A+=0A+/*=0A+=20*=20= Disables=20data=20checksums=20for=20the=20cluster,=20if=20applicable.=20= Starts=20a=20background=0A+=20*=20worker=20which=20turns=20off=20the=20= data=20checksums.=0A+=20*/=0A+Datum=0A= +disable_data_checksums(PG_FUNCTION_ARGS)=0A+{=0A+=09bool=09=09fast=20=3D=20= PG_GETARG_BOOL(0);=0A+=0A+=09if=20(!superuser())=0A+=09=09ereport(ERROR,=20= errmsg("must=20be=20superuser"));=0A+=0A+=09= StartDataChecksumsWorkerLauncher(DISABLE_DATACHECKSUMS,=200,=200,=20= fast);=0A+=09PG_RETURN_VOID();=0A+}=0A+=0A+/*=0A+=20*=20Enables=20data=20= checksums=20for=20the=20cluster,=20if=20applicable.=20=20Supports=20= vacuum-=0A+=20*=20like=20cost=20based=20throttling=20to=20limit=20system=20= load.=20Starts=20a=20background=20worker=0A+=20*=20which=20updates=20= data=20checksums=20on=20existing=20data.=0A+=20*/=0A+Datum=0A= +enable_data_checksums(PG_FUNCTION_ARGS)=0A+{=0A+=09int=09=09=09= cost_delay=20=3D=20PG_GETARG_INT32(0);=0A+=09int=09=09=09cost_limit=20=3D=20= PG_GETARG_INT32(1);=0A+=09bool=09=09fast=20=3D=20PG_GETARG_BOOL(2);=0A+=0A= +=09if=20(!superuser())=0A+=09=09ereport(ERROR,=20errmsg("must=20be=20= superuser"));=0A+=0A+=09if=20(cost_delay=20<=200)=0A+=09=09= ereport(ERROR,=20errmsg("cost=20delay=20cannot=20be=20a=20negative=20= value"));=0A+=0A+=09if=20(cost_limit=20<=3D=200)=0A+=09=09ereport(ERROR,=20= errmsg("cost=20limit=20must=20be=20greater=20than=20zero"));=0A+=0A+=09= StartDataChecksumsWorkerLauncher(ENABLE_DATACHECKSUMS,=20cost_delay,=20= cost_limit,=20fast);=0A+=0A+=09PG_RETURN_VOID();=0A+}=0Adiff=20--git=20= a/src/backend/backup/basebackup.c=20b/src/backend/backup/basebackup.c=0A= index=20bb7d90aa5d9..54dcfbcb333=20100644=0A---=20= a/src/backend/backup/basebackup.c=0A+++=20= b/src/backend/backup/basebackup.c=0A@@=20-1613,7=20+1613,8=20@@=20= sendFile(bbsink=20*sink,=20const=20char=20*readfilename,=20const=20char=20= *tarfilename,=0A=20=09=20*=20enabled=20for=20this=20cluster,=20and=20if=20= this=20is=20a=20relation=20file,=20then=20verify=0A=20=09=20*=20the=20= checksum.=0A=20=09=20*/=0A-=09if=20(!noverify_checksums=20&&=20= DataChecksumsEnabled()=20&&=0A+=09if=20(!noverify_checksums=20&&=0A+=09=09= DataChecksumsNeedWrite()=20&&=0A=20=09=09= RelFileNumberIsValid(relfilenumber))=0A=20=09=09verify_checksum=20=3D=20= true;=0A=20=0A@@=20-2007,6=20+2008,9=20@@=20verify_page_checksum(Page=20= page,=20XLogRecPtr=20start_lsn,=20BlockNumber=20blkno,=0A=20=09if=20= (PageIsNew(page)=20||=20PageGetLSN(page)=20>=3D=20start_lsn)=0A=20=09=09= return=20true;=0A=20=0A+=09if=20(!DataChecksumsNeedVerify())=0A+=09=09= return=20true;=0A+=0A=20=09/*=20Perform=20the=20actual=20checksum=20= calculation.=20*/=0A=20=09checksum=20=3D=20pg_checksum_page(page,=20= blkno);=0A=20=0Adiff=20--git=20= a/src/backend/catalog/system_functions.sql=20= b/src/backend/catalog/system_functions.sql=0Aindex=20= 566f308e443..dea7ad3cf30=20100644=0A---=20= a/src/backend/catalog/system_functions.sql=0A+++=20= b/src/backend/catalog/system_functions.sql=0A@@=20-650,6=20+650,18=20@@=20= LANGUAGE=20INTERNAL=0A=20CALLED=20ON=20NULL=20INPUT=20VOLATILE=20= PARALLEL=20SAFE=0A=20AS=20'pg_stat_reset_slru';=0A=20=0A+CREATE=20OR=20= REPLACE=20FUNCTION=0A+=20=20pg_enable_data_checksums(cost_delay=20= integer=20DEFAULT=200,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20cost_limit=20integer=20DEFAULT=20100,=0A= +=09=09=09=09=09=09=20=20=20fast=20boolean=20DEFAULT=20false)=0A+=20=20= RETURNS=20void=20STRICT=20VOLATILE=20LANGUAGE=20internal=20AS=20= 'enable_data_checksums'=0A+=20=20PARALLEL=20RESTRICTED;=0A+=0A+CREATE=20= OR=20REPLACE=20FUNCTION=0A+=20=20pg_disable_data_checksums(fast=20= boolean=20DEFAULT=20false)=0A+=20=20RETURNS=20void=20STRICT=20VOLATILE=20= LANGUAGE=20internal=20AS=20'disable_data_checksums'=0A+=20=20PARALLEL=20= RESTRICTED;=0A+=0A=20--=0A=20--=20The=20default=20permissions=20for=20= functions=20mean=20that=20anyone=20can=20execute=20them.=0A=20--=20A=20= number=20of=20functions=20shouldn't=20be=20executable=20by=20just=20= anyone,=20but=20rather=0A@@=20-775,6=20+787,10=20@@=20REVOKE=20EXECUTE=20= ON=20FUNCTION=20pg_ls_logicalmapdir()=20FROM=20PUBLIC;=0A=20=0A=20REVOKE=20= EXECUTE=20ON=20FUNCTION=20pg_ls_replslotdir(text)=20FROM=20PUBLIC;=0A=20=0A= +REVOKE=20EXECUTE=20ON=20FUNCTION=20pg_enable_data_checksums(integer,=20= integer,=20boolean)=20FROM=20public;=0A+=0A+REVOKE=20EXECUTE=20ON=20= FUNCTION=20pg_disable_data_checksums(boolean)=20FROM=20public;=0A+=0A=20= --=0A=20--=20We=20also=20set=20up=20some=20things=20as=20accessible=20to=20= standard=20roles.=0A=20--=0Adiff=20--git=20= a/src/backend/catalog/system_views.sql=20= b/src/backend/catalog/system_views.sql=0Aindex=20= 1b3c5a55882..22f67c7ee4a=20100644=0A---=20= a/src/backend/catalog/system_views.sql=0A+++=20= b/src/backend/catalog/system_views.sql=0A@@=20-1354,6=20+1354,26=20@@=20= CREATE=20VIEW=20pg_stat_progress_copy=20AS=0A=20=20=20=20=20FROM=20= pg_stat_get_progress_info('COPY')=20AS=20S=0A=20=20=20=20=20=20=20=20=20= LEFT=20JOIN=20pg_database=20D=20ON=20S.datid=20=3D=20D.oid;=0A=20=0A= +CREATE=20VIEW=20pg_stat_progress_data_checksums=20AS=0A+=20=20=20=20= SELECT=0A+=20=20=20=20=20=20=20=20S.pid=20AS=20pid,=20S.datid,=20= D.datname=20AS=20datname,=0A+=20=20=20=20=20=20=20=20CASE=20S.param1=20= WHEN=200=20THEN=20'enabling'=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20WHEN=201=20THEN=20'disabling'=0A+=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20WHEN=202=20THEN=20= 'waiting'=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20WHEN=203=20THEN=20'waiting=20on=20temporary=20tables'=0A+=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20WHEN=204=20THEN=20= 'waiting=20on=20checkpoint'=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20WHEN=205=20THEN=20'done'=0A+=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20END=20AS=20phase,=0A+=20=20=20= =20=20=20=20=20CASE=20S.param2=20WHEN=20-1=20THEN=20NULL=20ELSE=20= S.param2=20END=20AS=20databases_total,=0A+=20=20=20=20=20=20=20=20= S.param3=20AS=20databases_done,=0A+=20=20=20=20=20=20=20=20CASE=20= S.param4=20WHEN=20-1=20THEN=20NULL=20ELSE=20S.param4=20END=20AS=20= relations_total,=0A+=20=20=20=20=20=20=20=20CASE=20S.param5=20WHEN=20-1=20= THEN=20NULL=20ELSE=20S.param5=20END=20AS=20relations_done,=0A+=20=20=20=20= =20=20=20=20CASE=20S.param6=20WHEN=20-1=20THEN=20NULL=20ELSE=20S.param6=20= END=20AS=20blocks_total,=0A+=20=20=20=20=20=20=20=20CASE=20S.param7=20= WHEN=20-1=20THEN=20NULL=20ELSE=20S.param7=20END=20AS=20blocks_done=0A+=20= =20=20=20FROM=20pg_stat_get_progress_info('DATACHECKSUMS')=20AS=20S=0A+=20= =20=20=20=20=20=20=20LEFT=20JOIN=20pg_database=20D=20ON=20S.datid=20=3D=20= D.oid=0A+=20=20=20=20ORDER=20BY=20S.datid;=20--=20return=20the=20= launcher=20process=20first=0A+=0A=20CREATE=20VIEW=20pg_user_mappings=20= AS=0A=20=20=20=20=20SELECT=0A=20=20=20=20=20=20=20=20=20U.oid=20=20=20=20= =20=20=20AS=20umid,=0Adiff=20--git=20a/src/backend/postmaster/Makefile=20= b/src/backend/postmaster/Makefile=0Aindex=200f4435d2d97..0c36765acfe=20= 100644=0A---=20a/src/backend/postmaster/Makefile=0A+++=20= b/src/backend/postmaster/Makefile=0A@@=20-18,6=20+18,7=20@@=20OBJS=20=3D=20= \=0A=20=09bgworker.o=20\=0A=20=09bgwriter.o=20\=0A=20=09checkpointer.o=20= \=0A+=09datachecksumsworker.o=20\=0A=20=09fork_process.o=20\=0A=20=09= interrupt.o=20\=0A=20=09launch_backend.o=20\=0Adiff=20--git=20= a/src/backend/postmaster/auxprocess.c=20= b/src/backend/postmaster/auxprocess.c=0Aindex=20a6d3630398f..5742a1dd724=20= 100644=0A---=20a/src/backend/postmaster/auxprocess.c=0A+++=20= b/src/backend/postmaster/auxprocess.c=0A@@=20-15,6=20+15,7=20@@=0A=20= #include=20=0A=20#include=20=0A=20=0A+#include=20= "access/xlog.h"=0A=20#include=20"miscadmin.h"=0A=20#include=20"pgstat.h"=0A= =20#include=20"postmaster/auxprocess.h"=0A@@=20-68,6=20+69,24=20@@=20= AuxiliaryProcessMainCommon(void)=0A=20=0A=20=09ProcSignalInit(NULL,=20= 0);=0A=20=0A+=09/*=0A+=09=20*=20Initialize=20a=20local=20cache=20of=20= the=20data_checksum_version,=20to=20be=20updated=20by=0A+=09=20*=20the=20= procsignal-based=20barriers.=0A+=09=20*=0A+=09=20*=20This=20= intentionally=20happens=20after=20initializing=20the=20procsignal,=20= otherwise=0A+=09=20*=20we=20might=20miss=20a=20state=20change.=20This=20= means=20we=20can=20get=20a=20barrier=20for=20the=0A+=09=20*=20state=20= we've=20just=20initialized=20-=20but=20it=20can=20happen=20only=20once.=0A= +=09=20*=0A+=09=20*=20The=20postmaster=20(which=20is=20what=20gets=20= forked=20into=20the=20new=20child=20process)=0A+=09=20*=20does=20not=20= handle=20barriers,=20therefore=20it=20may=20not=20have=20the=20current=20= value=0A+=09=20*=20of=20LocalDataChecksumVersion=20value=20(it'll=20have=20= the=20value=20read=20from=20the=0A+=09=20*=20control=20file,=20which=20= may=20be=20arbitrarily=20old).=0A+=09=20*=0A+=09=20*=20NB:=20Even=20if=20= the=20postmaster=20handled=20barriers,=20the=20value=20might=20still=20= be=0A+=09=20*=20stale,=20as=20it=20might=20have=20changed=20after=20this=20= process=20forked.=0A+=09=20*/=0A+=09InitLocalDataChecksumVersion();=0A+=0A= =20=09/*=0A=20=09=20*=20Auxiliary=20processes=20don't=20run=20= transactions,=20but=20they=20may=20need=20a=0A=20=09=20*=20resource=20= owner=20anyway=20to=20manage=20buffer=20pins=20acquired=20outside=0Adiff=20= --git=20a/src/backend/postmaster/bgworker.c=20= b/src/backend/postmaster/bgworker.c=0Aindex=201ad65c237c3..0d2ade1f905=20= 100644=0A---=20a/src/backend/postmaster/bgworker.c=0A+++=20= b/src/backend/postmaster/bgworker.c=0A@@=20-18,6=20+18,7=20@@=0A=20= #include=20"pgstat.h"=0A=20#include=20"port/atomics.h"=0A=20#include=20= "postmaster/bgworker_internals.h"=0A+#include=20= "postmaster/datachecksumsworker.h"=0A=20#include=20= "postmaster/postmaster.h"=0A=20#include=20= "replication/logicallauncher.h"=0A=20#include=20= "replication/logicalworker.h"=0A@@=20-132,6=20+133,12=20@@=20static=20= const=20struct=0A=20=09},=0A=20=09{=0A=20=09=09"TablesyncWorkerMain",=20= TablesyncWorkerMain=0A+=09},=0A+=09{=0A+=09=09= "DataChecksumsWorkerLauncherMain",=20DataChecksumsWorkerLauncherMain=0A+=09= },=0A+=09{=0A+=09=09"DataChecksumsWorkerMain",=20DataChecksumsWorkerMain=0A= =20=09}=0A=20};=0A=20=0Adiff=20--git=20= a/src/backend/postmaster/datachecksumsworker.c=20= b/src/backend/postmaster/datachecksumsworker.c=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..ff451d502ba=0A---=20/dev/null=0A+++=20= b/src/backend/postmaster/datachecksumsworker.c=0A@@=20-0,0=20+1,1463=20= @@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20datachecksumsworker.c=0A+=20*=09=20=20Background=20= worker=20for=20enabling=20or=20disabling=20data=20checksums=20online=0A+=20= *=0A+=20*=20When=20enabling=20data=20checksums=20on=20a=20database=20at=20= initdb=20time=20or=20when=20shut=20down=0A+=20*=20with=20pg_checksums,=20= no=20extra=20process=20is=20required=20as=20each=20page=20is=20= checksummed,=0A+=20*=20and=20verified,=20when=20accessed.=20=20When=20= enabling=20checksums=20on=20an=20already=20running=0A+=20*=20cluster,=20= this=20worker=20will=20ensure=20that=20all=20pages=20are=20checksummed=20= before=0A+=20*=20verification=20of=20the=20checksums=20is=20turned=20on.=20= In=20the=20case=20of=20disabling=0A+=20*=20checksums,=20the=20state=20= transition=20is=20performed=20only=20in=20the=20control=20file,=20no=0A+=20= *=20changes=20are=20performed=20on=20the=20data=20pages.=0A+=20*=0A+=20*=20= Checksums=20can=20be=20either=20enabled=20or=20disabled=20cluster-wide,=20= with=20on/off=20being=0A+=20*=20the=20end=20state=20for=20= data_checksums.=0A+=20*=0A+=20*=20Enabling=20checksums=0A+=20*=20= ------------------=0A+=20*=20When=20enabling=20checksums=20in=20an=20= online=20cluster,=20data_checksums=20will=20be=20set=20to=0A+=20*=20= "inprogress-on"=20which=20signals=20that=20write=20operations=20MUST=20= compute=20and=20write=0A+=20*=20the=20checksum=20on=20the=20data=20page,=20= but=20during=20reading=20the=20checksum=20SHALL=20NOT=20be=0A+=20*=20= verified.=20This=20ensures=20that=20all=20objects=20created=20during=20= checksumming=20will=0A+=20*=20have=20checksums=20set,=20but=20no=20reads=20= will=20fail=20due=20to=20incorrect=20checksum.=20The=0A+=20*=20= DataChecksumsWorker=20will=20compile=20a=20list=20of=20databases=20which=20= exist=20at=20the=0A+=20*=20start=20of=20checksumming,=20and=20all=20of=20= these=20which=20haven't=20been=20dropped=20during=0A+=20*=20the=20= processing=20MUST=20have=20been=20processed=20successfully=20in=20order=20= for=20checksums=0A+=20*=20to=20be=20enabled.=20Any=20new=20relation=20= created=20during=20processing=20will=20see=20the=0A+=20*=20in-progress=20= state=20and=20will=20automatically=20be=20checksummed.=0A+=20*=0A+=20*=20= For=20each=20database,=20all=20relations=20which=20have=20storage=20are=20= read=20and=20every=20data=0A+=20*=20page=20is=20marked=20dirty=20to=20= force=20a=20write=20with=20the=20checksum.=20This=20will=20generate=0A+=20= *=20a=20lot=20of=20WAL=20as=20the=20entire=20database=20is=20read=20and=20= written.=0A+=20*=0A+=20*=20If=20the=20processing=20is=20interrupted=20by=20= a=20cluster=20restart,=20it=20will=20be=20restarted=0A+=20*=20from=20the=20= beginning=20again=20as=20state=20isn't=20persisted.=0A+=20*=0A+=20*=20= Disabling=20checksums=0A+=20*=20-------------------=0A+=20*=20When=20= disabling=20checksums,=20data_checksums=20will=20be=20set=20to=20= "inprogress-off"=0A+=20*=20which=20signals=20that=20checksums=20are=20= written=20but=20no=20longer=20verified.=20This=20ensure=0A+=20*=20that=20= backends=20which=20have=20yet=20to=20move=20from=20the=20"on"=20state=20= will=20still=20be=20able=0A+=20*=20to=20process=20data=20checksum=20= validation.=0A+=20*=0A+=20*=20Synchronization=20and=20Correctness=0A+=20= *=20-------------------------------=0A+=20*=20The=20processes=20involved=20= in=20enabling,=20or=20disabling,=20data=20checksums=20in=20an=0A+=20*=20= online=20cluster=20must=20be=20properly=20synchronized=20with=20the=20= normal=20backends=0A+=20*=20serving=20concurrent=20queries=20to=20ensure=20= correctness.=20Correctness=20is=20defined=0A+=20*=20as=20the=20= following:=0A+=20*=0A+=20*=20=20=20=20-=20Backends=20SHALL=20NOT=20= violate=20local=20data_checksums=20state=0A+=20*=20=20=20=20-=20Data=20= checksums=20SHALL=20NOT=20be=20considered=20enabled=20cluster-wide=20= until=20all=0A+=20*=20=20=20=20=20=20currently=20connected=20backends=20= have=20the=20local=20state=20"enabled"=0A+=20*=0A+=20*=20There=20are=20= two=20levels=20of=20synchronization=20required=20for=20enabling=20data=20= checksums=0A+=20*=20in=20an=20online=20cluster:=20(i)=20changing=20state=20= in=20the=20active=20backends=20("on",=0A+=20*=20"off",=20"inprogress-on"=20= and=20"inprogress-off"),=20and=20(ii)=20ensuring=20no=0A+=20*=20= incompatible=20objects=20and=20processes=20are=20left=20in=20a=20= database=20when=20workers=20end.=0A+=20*=20The=20former=20deals=20with=20= cluster-wide=20agreement=20on=20data=20checksum=20state=20and=20the=0A+=20= *=20latter=20with=20ensuring=20that=20any=20concurrent=20activity=20= cannot=20break=20the=20data=0A+=20*=20checksum=20contract=20during=20= processing.=0A+=20*=0A+=20*=20Synchronizing=20the=20state=20change=20is=20= done=20with=20procsignal=20barriers,=20where=20the=0A+=20*=20WAL=20= logging=20backend=20updating=20the=20global=20state=20in=20the=20= controlfile=20will=20wait=0A+=20*=20for=20all=20other=20backends=20to=20= absorb=20the=20barrier.=20Barrier=20absorption=20will=20happen=0A+=20*=20= during=20interrupt=20processing,=20which=20means=20that=20connected=20= backends=20will=20change=0A+=20*=20state=20at=20different=20times.=20To=20= prevent=20data=20checksum=20state=20changes=20when=0A+=20*=20writing=20= and=20verifying=20checksums,=20interrupts=20shall=20be=20held=20off=20= before=0A+=20*=20interrogating=20state=20and=20resumed=20when=20the=20IO=20= operation=20has=20been=20performed.=0A+=20*=0A+=20*=20=20=20When=20= Enabling=20Data=20Checksums=0A+=20*=20=20=20----------------------------=0A= +=20*=20=20=20A=20process=20which=20fails=20to=20observe=20data=20= checksums=20being=20enabled=20can=20induce=0A+=20*=20=20=20two=20types=20= of=20errors:=20failing=20to=20write=20the=20checksum=20when=20modifying=20= the=20page=0A+=20*=20=20=20and=20failing=20to=20validate=20the=20data=20= checksum=20on=20the=20page=20when=20reading=20it.=0A+=20*=0A+=20*=20=20=20= When=20processing=20starts=20all=20backends=20belong=20to=20one=20of=20= the=20below=20sets,=20with=0A+=20*=20=20=20one=20set=20being=20empty:=0A= +=20*=0A+=20*=20=20=20Bd:=20Backends=20in=20"off"=20state=0A+=20*=20=20=20= Bi:=20Backends=20in=20"inprogress-on"=20state=0A+=20*=0A+=20*=20=20=20If=20= processing=20is=20started=20in=20an=20online=20cluster=20then=20all=20= backends=20are=20in=20Bd.=0A+=20*=20=20=20If=20processing=20was=20halted=20= by=20the=20cluster=20shutting=20down,=20the=20controlfile=0A+=20*=20=20=20= state=20"inprogress-on"=20will=20be=20observed=20on=20system=20startup=20= and=20all=20backends=0A+=20*=20=20=20will=20be=20in=20Bd.=20Backends=20= transition=20Bd=20->=20Bi=20via=20a=20procsignalbarrier.=20=20When=0A+=20= *=20=20=20the=20DataChecksumsWorker=20has=20finished=20writing=20= checksums=20on=20all=20pages=20and=0A+=20*=20=20=20enables=20data=20= checksums=20cluster-wide,=20there=20are=20four=20sets=20of=20backends=20= where=0A+=20*=20=20=20Bd=20shall=20be=20an=20empty=20set:=0A+=20*=0A+=20= *=20=20=20Bg:=20Backend=20updating=20the=20global=20state=20and=20= emitting=20the=20procsignalbarrier=0A+=20*=20=20=20Bd:=20Backends=20in=20= "off"=20state=0A+=20*=20=20=20Be:=20Backends=20in=20"on"=20state=0A+=20*=20= =20=20Bi:=20Backends=20in=20"inprogress-on"=20state=0A+=20*=0A+=20*=20=20= =20Backends=20in=20Bi=20and=20Be=20will=20write=20checksums=20when=20= modifying=20a=20page,=20but=20only=0A+=20*=20=20=20backends=20in=20Be=20= will=20verify=20the=20checksum=20during=20reading.=20The=20Bg=20backend=20= is=0A+=20*=20=20=20blocked=20waiting=20for=20all=20backends=20in=20Bi=20= to=20process=20interrupts=20and=20move=20to=0A+=20*=20=20=20Be.=20Any=20= backend=20starting=20while=20Bg=20is=20waiting=20on=20the=20= procsignalbarrier=20will=0A+=20*=20=20=20observe=20the=20global=20state=20= being=20"on"=20and=20will=20thus=20automatically=20belong=20to=0A+=20*=20= =20=20Be.=20=20Checksums=20are=20enabled=20cluster-wide=20when=20Bi=20is=20= an=20empty=20set.=20Bi=20and=20Be=0A+=20*=20=20=20are=20compatible=20= sets=20while=20still=20operating=20based=20on=20their=20local=20state=20= as=0A+=20*=20=20=20both=20write=20data=20checksums.=0A+=20*=0A+=20*=20=20= =20When=20Disabling=20Data=20Checksums=0A+=20*=20=20=20= -----------------------------=0A+=20*=20=20=20A=20process=20which=20= fails=20to=20observe=20that=20data=20checksums=20have=20been=20disabled=0A= +=20*=20=20=20can=20induce=20two=20types=20of=20errors:=20writing=20the=20= checksum=20when=20modifying=20the=0A+=20*=20=20=20page=20and=20= validating=20a=20data=20checksum=20which=20is=20no=20longer=20correct=20= due=20to=0A+=20*=20=20=20modifications=20to=20the=20page.=0A+=20*=0A+=20= *=20=20=20Bg:=20Backend=20updating=20the=20global=20state=20and=20= emitting=20the=20procsignalbarrier=0A+=20*=20=20=20Bd:=20Backends=20in=20= "off"=20state=0A+=20*=20=20=20Be:=20Backends=20in=20"on"=20state=0A+=20*=20= =20=20Bo:=20Backends=20in=20"inprogress-off"=20state=0A+=20*=0A+=20*=20=20= =20Backends=20transition=20from=20the=20Be=20state=20to=20Bd=20like=20= so:=20Be=20->=20Bo=20->=20Bd=0A+=20*=0A+=20*=20=20=20The=20goal=20is=20= to=20transition=20all=20backends=20to=20Bd=20making=20the=20others=20= empty=20sets.=0A+=20*=20=20=20Backends=20in=20Bo=20write=20data=20= checksums,=20but=20don't=20validate=20them,=20such=20that=0A+=20*=20=20=20= backends=20still=20in=20Be=20can=20continue=20to=20validate=20pages=20= until=20the=20barrier=20has=0A+=20*=20=20=20been=20absorbed=20such=20= that=20they=20are=20in=20Bo.=20Once=20all=20backends=20are=20in=20Bo,=20= the=0A+=20*=20=20=20barrier=20to=20transition=20to=20"off"=20can=20be=20= raised=20and=20all=20backends=20can=20safely=0A+=20*=20=20=20stop=20= writing=20data=20checksums=20as=20no=20backend=20is=20enforcing=20data=20= checksum=0A+=20*=20=20=20validation=20any=20longer.=0A+=20*=0A+=20*=0A+=20= *=20Potential=20optimizations=0A+=20*=20-----------------------=0A+=20*=20= Below=20are=20some=20potential=20optimizations=20and=20improvements=20= which=20were=20brought=0A+=20*=20up=20during=20reviews=20of=20this=20= feature,=20but=20which=20weren't=20implemented=20in=20the=0A+=20*=20= initial=20version.=20These=20are=20ideas=20listed=20without=20any=20= validation=20on=20their=0A+=20*=20feasibility=20or=20potential=20payoff.=20= More=20discussion=20on=20these=20can=20be=20found=20on=0A+=20*=20the=20= -hackers=20threads=20linked=20to=20in=20the=20commit=20message=20of=20= this=20feature.=0A+=20*=0A+=20*=20=20=20*=20Launching=20= datachecksumsworker=20for=20resuming=20operation=20from=20the=20startup=0A= +=20*=20=20=20=20=20process:=20Currently=20users=20have=20to=20restart=20= processing=20manually=20after=20a=0A+=20*=20=20=20=20=20restart=20since=20= dynamic=20background=20worker=20cannot=20be=20started=20from=20the=0A+=20= *=20=20=20=20=20postmaster.=20Changing=20the=20startup=20process=20could=20= make=20restarting=20the=0A+=20*=20=20=20=20=20processing=20automatic=20= on=20cluster=20restart.=0A+=20*=20=20=20*=20Avoid=20dirtying=20the=20= page=20when=20checksums=20already=20match:=20Iff=20the=20checksum=0A+=20= *=20=20=20=20=20on=20the=20page=20happens=20to=20already=20match=20we=20= still=20dirty=20the=20page.=20It=20should=0A+=20*=20=20=20=20=20be=20= enough=20to=20only=20do=20the=20log_newpage_buffer()=20call=20in=20that=20= case.=0A+=20*=20=20=20*=20Invent=20a=20lightweight=20WAL=20record=20that=20= doesn't=20contain=20the=20full-page=0A+=20*=20=20=20=20=20image=20but=20= just=20the=20block=20number:=20On=20replay,=20the=20redo=20routine=20= would=20read=0A+=20*=20=20=20=20=20the=20page=20from=20disk.=0A+=20*=20=20= =20*=20Teach=20pg_checksums=20to=20avoid=20checksummed=20pages=20when=20= pg_checksums=20is=20used=0A+=20*=20=20=20=20=20to=20enable=20checksums=20= on=20a=20cluster=20which=20is=20in=20inprogress-on=20state=20and=0A+=20*=20= =20=20=20=20may=20have=20checksummed=20pages=20(make=20pg_checksums=20be=20= able=20to=20resume=20an=0A+=20*=20=20=20=20=20online=20operation).=0A+=20= *=0A+=20*=0A+=20*=20Portions=20Copyright=20(c)=201996-2025,=20PostgreSQL=20= Global=20Development=20Group=0A+=20*=20Portions=20Copyright=20(c)=20= 1994,=20Regents=20of=20the=20University=20of=20California=0A+=20*=0A+=20= *=0A+=20*=20IDENTIFICATION=0A+=20*=09=20=20= src/backend/postmaster/datachecksumsworker.c=0A+=20*=0A+=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres.h"=0A+=0A+#include=20"access/genam.h"=0A= +#include=20"access/heapam.h"=0A+#include=20"access/htup_details.h"=0A= +#include=20"access/xact.h"=0A+#include=20"access/xloginsert.h"=0A= +#include=20"catalog/indexing.h"=0A+#include=20"catalog/pg_class.h"=0A= +#include=20"catalog/pg_database.h"=0A+#include=20"commands/progress.h"=0A= +#include=20"commands/vacuum.h"=0A+#include=20"common/relpath.h"=0A= +#include=20"miscadmin.h"=0A+#include=20"pgstat.h"=0A+#include=20= "postmaster/bgworker.h"=0A+#include=20"postmaster/bgwriter.h"=0A= +#include=20"postmaster/datachecksumsworker.h"=0A+#include=20= "storage/bufmgr.h"=0A+#include=20"storage/checksum.h"=0A+#include=20= "storage/ipc.h"=0A+#include=20"storage/lmgr.h"=0A+#include=20= "storage/lwlock.h"=0A+#include=20"storage/procarray.h"=0A+#include=20= "storage/smgr.h"=0A+#include=20"tcop/tcopprot.h"=0A+#include=20= "utils/fmgroids.h"=0A+#include=20"utils/injection_point.h"=0A+#include=20= "utils/lsyscache.h"=0A+#include=20"utils/ps_status.h"=0A+#include=20= "utils/syscache.h"=0A+=0A+/*=0A+=20*=20Number=20of=20times=20we=20retry=20= to=20open=20a=20database=20before=20giving=20up=20and=20consider=0A+=20*=20= it=20to=20have=20failed=20processing.=0A+=20*/=0A+#define=20= DATACHECKSUMSWORKER_MAX_DB_RETRIES=205=0A+=0A+/*=0A+=20*=20Signaling=20= between=20backends=20calling=20pg_enable/disable_data_checksums,=20the=0A= +=20*=20checksums=20launcher=20process,=20and=20the=20checksums=20worker=20= process.=0A+=20*=0A+=20*=20This=20struct=20is=20protected=20by=20= DataChecksumsWorkerLock=0A+=20*/=0A+typedef=20struct=20= DataChecksumsWorkerShmemStruct=0A+{=0A+=09/*=0A+=09=20*=20These=20are=20= set=20by=20pg_{enable|disable|verify}_data_checksums,=20to=20tell=20the=0A= +=09=20*=20launcher=20what=20the=20target=20state=20is.=0A+=09=20*/=0A+=09= DataChecksumsWorkerOperation=20launch_operation;=0A+=09int=09=09=09= launch_cost_delay;=0A+=09int=09=09=09launch_cost_limit;=0A+=09bool=09=09= launch_fast;=0A+=0A+=09/*=0A+=09=20*=20Is=20a=20launcher=20process=20is=20= currently=20running?=0A+=09=20*=0A+=09=20*=20This=20is=20set=20by=20the=20= launcher=20process,=20after=20it=20has=20read=20the=20above=0A+=09=20*=20= launch_*=20parameters.=0A+=09=20*/=0A+=09bool=09=09launcher_running;=0A+=0A= +=09/*=0A+=09=20*=20These=20fields=20indicate=20the=20target=20state=20= that=20the=20launcher=20is=20currently=0A+=09=20*=20working=20towards.=20= They=20can=20be=20different=20from=20the=20corresponding=20launch_*=0A+=09= =20*=20fields,=20if=20a=20new=20pg_enable/disable_data_checksums()=20= call=20was=20made=20while=0A+=09=20*=20the=20launcher/worker=20was=20= already=20running.=0A+=09=20*=0A+=09=20*=20The=20below=20members=20are=20= set=20when=20the=20launcher=20starts,=20and=20are=20only=0A+=09=20*=20= accessed=20read-only=20by=20the=20single=20worker.=20Thus,=20we=20can=20= access=20these=0A+=09=20*=20without=20a=20lock.=20If=20multiple=20= workers,=20or=20dynamic=20cost=20parameters,=20are=0A+=09=20*=20= supported=20at=20some=20point=20then=20this=20would=20need=20to=20be=20= revisited.=0A+=09=20*/=0A+=09DataChecksumsWorkerOperation=20operation;=0A= +=09int=09=09=09cost_delay;=0A+=09int=09=09=09cost_limit;=0A+=09bool=09=09= immediate_checkpoint;=0A+=0A+=09/*=0A+=09=20*=20Signaling=20between=20= the=20launcher=20and=20the=20worker=20process.=0A+=09=20*=0A+=09=20*=20= As=20there=20is=20only=20a=20single=20worker,=20and=20the=20launcher=20= won't=20read=20these=0A+=09=20*=20until=20the=20worker=20exits,=20they=20= can=20be=20accessed=20without=20the=20need=20for=20a=0A+=09=20*=20lock.=20= If=20multiple=20workers=20are=20supported=20then=20this=20will=20have=20= to=20be=0A+=09=20*=20revisited.=0A+=09=20*/=0A+=0A+=09/*=20result,=20set=20= by=20worker=20before=20exiting=20*/=0A+=09DataChecksumsWorkerResult=20= success;=0A+=0A+=09/*=0A+=09=20*=20tells=20the=20worker=20process=20= whether=20it=20should=20also=20process=20the=20shared=0A+=09=20*=20= catalogs=0A+=09=20*/=0A+=09bool=09=09process_shared_catalogs;=0A+}=20= DataChecksumsWorkerShmemStruct;=0A+=0A+/*=20Shared=20memory=20segment=20= for=20datachecksumsworker=20*/=0A+static=20= DataChecksumsWorkerShmemStruct=20*DataChecksumsWorkerShmem;=0A+=0A= +typedef=20struct=20DataChecksumsWorkerDatabase=0A+{=0A+=09Oid=09=09=09= dboid;=0A+=09char=09=20=20=20*dbname;=0A+}=20= DataChecksumsWorkerDatabase;=0A+=0A+typedef=20struct=20= DataChecksumsWorkerResultEntry=0A+{=0A+=09Oid=09=09=09dboid;=0A+=09= DataChecksumsWorkerResult=20result;=0A+=09int=09=09=09retries;=0A+}=20= DataChecksumsWorkerResultEntry;=0A+=0A+=0A+/*=0A+=20*=20Flag=20set=20by=20= the=20interrupt=20handler=0A+=20*/=0A+static=20volatile=20sig_atomic_t=20= abort_requested=20=3D=20false;=0A+=0A+/*=0A+=20*=20Have=20we=20set=20the=20= DataChecksumsWorkerShmemStruct->launcher_running=20flag?=0A+=20*=20If=20= we=20have,=20we=20need=20to=20clear=20it=20before=20exiting!=0A+=20*/=0A= +static=20volatile=20sig_atomic_t=20launcher_running=20=3D=20false;=0A+=0A= +/*=0A+=20*=20Are=20we=20enabling=20data=20checksums,=20or=20disabling=20= them?=0A+=20*/=0A+static=20DataChecksumsWorkerOperation=20operation;=0A+=0A= +/*=20Prototypes=20*/=0A+static=20List=20*BuildDatabaseList(void);=0A= +static=20List=20*BuildRelationList(bool=20temp_relations,=20bool=20= include_shared);=0A+static=20void=20FreeDatabaseList(List=20*dblist);=0A= +static=20DataChecksumsWorkerResult=20= ProcessDatabase(DataChecksumsWorkerDatabase=20*db);=0A+static=20bool=20= ProcessAllDatabases(bool=20immediate_checkpoint);=0A+static=20bool=20= ProcessSingleRelationFork(Relation=20reln,=20ForkNumber=20forkNum,=20= BufferAccessStrategy=20strategy);=0A+static=20void=20= launcher_cancel_handler(SIGNAL_ARGS);=0A+static=20void=20= WaitForAllTransactionsToFinish(void);=0A+=0A+/*=0A+=20*=20= StartDataChecksumsWorkerLauncher=0A+=20*=09=09Main=20entry=20point=20for=20= datachecksumsworker=20launcher=20process=0A+=20*=0A+=20*=20The=20main=20= entrypoint=20for=20starting=20data=20checksums=20processing=20for=20= enabling=20as=0A+=20*=20well=20as=20disabling.=0A+=20*/=0A+void=0A= +StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A+=09= =09=09=09=09=09=09=09=20int=20cost_delay,=0A+=09=09=09=09=09=09=09=09=20= int=20cost_limit,=0A+=09=09=09=09=09=09=09=09=20bool=20fast)=0A+{=0A+=09= BackgroundWorker=20bgw;=0A+=09BackgroundWorkerHandle=20*bgw_handle;=0A+=09= bool=09=09launcher_running;=0A+=0A+#ifdef=20USE_ASSERT_CHECKING=0A+=09/*=20= The=20cost=20delay=20settings=20have=20no=20effect=20when=20disabling=20= */=0A+=09if=20(op=20=3D=3D=20DISABLE_DATACHECKSUMS)=0A+=09=09= Assert(cost_delay=20=3D=3D=200=20&&=20cost_limit=20=3D=3D=200);=0A= +#endif=0A+=0A+=09/*=20Store=20the=20desired=20state=20in=20shared=20= memory=20*/=0A+=09LWLockAcquire(DataChecksumsWorkerLock,=20= LW_EXCLUSIVE);=0A+=0A+=09DataChecksumsWorkerShmem->launch_operation=20=3D=20= op;=0A+=09DataChecksumsWorkerShmem->launch_cost_delay=20=3D=20= cost_delay;=0A+=09DataChecksumsWorkerShmem->launch_cost_limit=20=3D=20= cost_limit;=0A+=09DataChecksumsWorkerShmem->launch_fast=20=3D=20fast;=0A= +=0A+=09/*=20is=20the=20launcher=20already=20running?=20*/=0A+=09= launcher_running=20=3D=20DataChecksumsWorkerShmem->launcher_running;=0A+=0A= +=09LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09/*=0A+=09=20*=20= Launch=20a=20new=20launcher=20process,=20if=20it's=20not=20running=20= already.=0A+=09=20*=0A+=09=20*=20If=20the=20launcher=20is=20currently=20= busy=20enabling=20the=20checksums,=20and=20we=20want=0A+=09=20*=20them=20= disabled=20(or=20vice=20versa),=20the=20launcher=20will=20notice=20that=20= at=20latest=0A+=09=20*=20when=20it's=20about=20to=20exit,=20and=20will=20= loop=20back=20process=20the=20new=20request.=20So=0A+=09=20*=20if=20the=20= launcher=20is=20already=20running,=20we=20don't=20need=20to=20do=20= anything=20more=0A+=09=20*=20here=20to=20abort=20it.=0A+=09=20*=0A+=09=20= *=20If=20you=20call=20pg_enable/disable_data_checksums()=20twice=20in=20= a=20row,=20before=0A+=09=20*=20the=20launcher=20has=20had=20a=20chance=20= to=20start=20up,=20we=20still=20end=20up=20launching=20it=0A+=09=20*=20= twice.=20=20That's=20OK,=20the=20second=20invocation=20will=20see=20that=20= a=20launcher=20is=0A+=09=20*=20already=20running=20and=20exit=20quickly.=0A= +=09=20*=0A+=09=20*=20TODO:=20We=20could=20optimize=20here=20and=20skip=20= launching=20the=20launcher,=20if=20we=20are=0A+=09=20*=20already=20in=20= the=20desired=20state,=20i.e.=20if=20the=20checksums=20are=20already=20= enabled=0A+=09=20*=20and=20you=20call=20pg_enable_data_checksums().=0A+=09= =20*/=0A+=09if=20(!launcher_running)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20= Prepare=20the=20BackgroundWorker=20and=20launch=20it.=0A+=09=09=20*/=0A+=09= =09memset(&bgw,=200,=20sizeof(bgw));=0A+=09=09bgw.bgw_flags=20=3D=20= BGWORKER_SHMEM_ACCESS=20|=20BGWORKER_BACKEND_DATABASE_CONNECTION;=0A+=09=09= bgw.bgw_start_time=20=3D=20BgWorkerStart_RecoveryFinished;=0A+=09=09= snprintf(bgw.bgw_library_name,=20BGW_MAXLEN,=20"postgres");=0A+=09=09= snprintf(bgw.bgw_function_name,=20BGW_MAXLEN,=20= "DataChecksumsWorkerLauncherMain");=0A+=09=09snprintf(bgw.bgw_name,=20= BGW_MAXLEN,=20"datachecksumsworker=20launcher");=0A+=09=09= snprintf(bgw.bgw_type,=20BGW_MAXLEN,=20"datachecksumsworker=20= launcher");=0A+=09=09bgw.bgw_restart_time=20=3D=20BGW_NEVER_RESTART;=0A+=09= =09bgw.bgw_notify_pid=20=3D=20MyProcPid;=0A+=09=09bgw.bgw_main_arg=20=3D=20= (Datum)=200;=0A+=0A+=09=09if=20(!RegisterDynamicBackgroundWorker(&bgw,=20= &bgw_handle))=0A+=09=09=09ereport(ERROR,=0A+=09=09=09=09=09= errmsg("failed=20to=20start=20background=20worker=20to=20process=20data=20= checksums"));=0A+=09}=0A+}=0A+=0A+/*=0A+=20*=20ProcessSingleRelationFork=0A= +=20*=09=09Enable=20data=20checksums=20in=20a=20single=20relation/fork.=0A= +=20*=0A+=20*=20Returns=20true=20if=20successful,=20and=20false=20if=20= *aborted*.=20On=20error,=20an=20actual=0A+=20*=20error=20is=20raised=20= in=20the=20lower=20levels.=0A+=20*/=0A+static=20bool=0A= +ProcessSingleRelationFork(Relation=20reln,=20ForkNumber=20forkNum,=20= BufferAccessStrategy=20strategy)=0A+{=0A+=09BlockNumber=20numblocks=20=3D=20= RelationGetNumberOfBlocksInFork(reln,=20forkNum);=0A+=09char=09=09= activity[NAMEDATALEN=20*=202=20+=20128];=0A+=09char=09=20=20=20*relns;=0A= +=0A+=09relns=20=3D=20get_namespace_name(RelationGetNamespace(reln));=0A= +=0A+=09if=20(!relns)=0A+=09=09return=20false;=0A+=0A+=09/*=20Report=20= the=20current=20relation=20to=20pgstat_activity=20*/=0A+=09= snprintf(activity,=20sizeof(activity)=20-=201,=20"processing:=20%s.%s=20= (%s,=20%dblocks)",=0A+=09=09=09=20relns,=20= RelationGetRelationName(reln),=20forkNames[forkNum],=20numblocks);=0A+=09= pgstat_report_activity(STATE_RUNNING,=20activity);=0A+=0A+=09/*=0A+=09=20= *=20As=20of=20now=20we=20only=20update=20the=20block=20counter=20for=20= main=20forks=20in=20order=20to=0A+=09=20*=20not=20cause=20too=20frequent=20= calls.=20TODO:=20investigate=20whether=20we=20should=20do=20it=0A+=09=20= *=20more=20frequent?=0A+=09=20*/=0A+=09if=20(forkNum=20=3D=3D=20= MAIN_FORKNUM)=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_BLOCKS_TOTAL,=0A+=09=09= =09=09=09=09=09=09=09=20numblocks);=0A+=0A+=09/*=0A+=09=20*=20We=20are=20= looping=20over=20the=20blocks=20which=20existed=20at=20the=20time=20of=20= process=0A+=09=20*=20start,=20which=20is=20safe=20since=20new=20blocks=20= are=20created=20with=20checksums=20set=0A+=09=20*=20already=20due=20to=20= the=20state=20being=20"inprogress-on".=0A+=09=20*/=0A+=09for=20= (BlockNumber=20blknum=20=3D=200;=20blknum=20<=20numblocks;=20blknum++)=0A= +=09{=0A+=09=09Buffer=09=09buf=20=3D=20ReadBufferExtended(reln,=20= forkNum,=20blknum,=20RBM_NORMAL,=20strategy);=0A+=0A+=09=09/*=20Need=20= to=20get=20an=20exclusive=20lock=20before=20we=20can=20flag=20as=20dirty=20= */=0A+=09=09LockBuffer(buf,=20BUFFER_LOCK_EXCLUSIVE);=0A+=0A+=09=09/*=0A= +=09=09=20*=20Mark=20the=20buffer=20as=20dirty=20and=20force=20a=20full=20= page=20write.=20=20We=20have=20to=0A+=09=09=20*=20re-write=20the=20page=20= to=20WAL=20even=20if=20the=20checksum=20hasn't=20changed,=0A+=09=09=20*=20= because=20if=20there=20is=20a=20replica=20it=20might=20have=20a=20= slightly=20different=0A+=09=09=20*=20version=20of=20the=20page=20with=20= an=20invalid=20checksum,=20caused=20by=20unlogged=0A+=09=09=20*=20= changes=20(e.g.=20hintbits)=20on=20the=20master=20happening=20while=20= checksums=0A+=09=09=20*=20were=20off.=20This=20can=20happen=20if=20there=20= was=20a=20valid=20checksum=20on=20the=20page=0A+=09=09=20*=20at=20one=20= point=20in=20the=20past,=20so=20only=20when=20checksums=20are=20first=20= on,=20then=0A+=09=09=20*=20off,=20and=20then=20turned=20on=20again.=20=20= TODO:=20investigate=20if=20this=20could=20be=0A+=09=09=20*=20avoided=20= if=20the=20checksum=20is=20calculated=20to=20be=20correct=20and=20= wal_level=0A+=09=09=20*=20is=20set=20to=20"minimal",=0A+=09=09=20*/=0A+=09= =09START_CRIT_SECTION();=0A+=09=09MarkBufferDirty(buf);=0A+=09=09= log_newpage_buffer(buf,=20false);=0A+=09=09END_CRIT_SECTION();=0A+=0A+=09= =09UnlockReleaseBuffer(buf);=0A+=0A+=09=09/*=0A+=09=09=20*=20This=20is=20= the=20only=20place=20where=20we=20check=20if=20we=20are=20asked=20to=20= abort,=20the=0A+=09=09=20*=20abortion=20will=20bubble=20up=20from=20= here.=20It's=20safe=20to=20check=20this=20without=0A+=09=09=20*=20a=20= lock,=20because=20if=20we=20miss=20it=20being=20set,=20we=20will=20try=20= again=20soon.=0A+=09=09=20*/=0A+=09=09Assert(operation=20=3D=3D=20= ENABLE_DATACHECKSUMS);=0A+=09=09if=20= (DataChecksumsWorkerShmem->launch_operation=20=3D=3D=20= DISABLE_DATACHECKSUMS)=0A+=09=09=09abort_requested=20=3D=20true;=0A+=0A+=09= =09if=20(abort_requested)=0A+=09=09=09return=20false;=0A+=0A+=09=09/*=0A= +=09=09=20*=20As=20of=20now=20we=20only=20update=20the=20block=20counter=20= for=20main=20forks=20in=20order=0A+=09=09=20*=20to=20not=20cause=20too=20= frequent=20calls.=20TODO:=20investigate=20whether=20we=0A+=09=09=20*=20= should=20do=20it=20more=20frequent?=0A+=09=09=20*/=0A+=09=09if=20= (forkNum=20=3D=3D=20MAIN_FORKNUM)=0A+=09=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_BLOCKS_DONE,=0A+=09=09= =09=09=09=09=09=09=09=09=20(blknum=20+=201));=0A+=0A+=09=09= vacuum_delay_point(false);=0A+=09}=0A+=0A+=09pfree(relns);=0A+=09return=20= true;=0A+}=0A+=0A+/*=0A+=20*=20ProcessSingleRelationByOid=0A+=20*=09=09= Process=20a=20single=20relation=20based=20on=20oid.=0A+=20*=0A+=20*=20= Returns=20true=20if=20successful,=20and=20false=20if=20*aborted*.=20On=20= error,=20an=20actual=0A+=20*=20error=20is=20raised=20in=20the=20lower=20= levels.=0A+=20*/=0A+static=20bool=0A+ProcessSingleRelationByOid(Oid=20= relationId,=20BufferAccessStrategy=20strategy)=0A+{=0A+=09Relation=09= rel;=0A+=09bool=09=09aborted=20=3D=20false;=0A+=0A+=09= StartTransactionCommand();=0A+=0A+=09rel=20=3D=20= try_relation_open(relationId,=20AccessShareLock);=0A+=09if=20(rel=20=3D=3D= =20NULL)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20Relation=20no=20longer=20= exists.=20We=20don't=20consider=20this=20an=20error=20since=0A+=09=09=20= *=20there=20are=20no=20pages=20in=20it=20that=20need=20data=20checksums,=20= and=20thus=20return=0A+=09=09=20*=20true.=20The=20worker=20operates=20= off=20a=20list=20of=20relations=20generated=20at=20the=0A+=09=09=20*=20= start=20of=20processing,=20so=20relations=20being=20dropped=20in=20the=20= meantime=20is=0A+=09=09=20*=20to=20be=20expected.=0A+=09=09=20*/=0A+=09=09= CommitTransactionCommand();=0A+=09=09pgstat_report_activity(STATE_IDLE,=20= NULL);=0A+=09=09return=20true;=0A+=09}=0A+=09RelationGetSmgr(rel);=0A+=0A= +=09for=20(ForkNumber=20fnum=20=3D=200;=20fnum=20<=3D=20MAX_FORKNUM;=20= fnum++)=0A+=09{=0A+=09=09if=20(smgrexists(rel->rd_smgr,=20fnum))=0A+=09=09= {=0A+=09=09=09if=20(!ProcessSingleRelationFork(rel,=20fnum,=20strategy))=0A= +=09=09=09{=0A+=09=09=09=09aborted=20=3D=20true;=0A+=09=09=09=09break;=0A= +=09=09=09}=0A+=09=09}=0A+=09}=0A+=09relation_close(rel,=20= AccessShareLock);=0A+=09elog(DEBUG2,=0A+=09=09=20"data=20checksum=20= processing=20done=20for=20relation=20with=20OID=20%u:=20%s",=0A+=09=09=20= relationId,=20(aborted=20?=20"aborted"=20:=20"finished"));=0A+=0A+=09= CommitTransactionCommand();=0A+=0A+=09pgstat_report_activity(STATE_IDLE,=20= NULL);=0A+=0A+=09return=20!aborted;=0A+}=0A+=0A+/*=0A+=20*=20= ProcessDatabase=0A+=20*=09=09Enable=20data=20checksums=20in=20a=20single=20= database.=0A+=20*=0A+=20*=20We=20do=20this=20by=20launching=20a=20= dynamic=20background=20worker=20into=20this=20database,=20and=0A+=20*=20= waiting=20for=20it=20to=20finish.=20=20We=20have=20to=20do=20this=20in=20= a=20separate=20worker,=20since=0A+=20*=20each=20process=20can=20only=20= be=20connected=20to=20one=20database=20during=20its=20lifetime.=0A+=20*/=0A= +static=20DataChecksumsWorkerResult=0A= +ProcessDatabase(DataChecksumsWorkerDatabase=20*db)=0A+{=0A+=09= BackgroundWorker=20bgw;=0A+=09BackgroundWorkerHandle=20*bgw_handle;=0A+=09= BgwHandleStatus=20status;=0A+=09pid_t=09=09pid;=0A+=09char=09=09= activity[NAMEDATALEN=20+=2064];=0A+=0A+=09= DataChecksumsWorkerShmem->success=20=3D=20DATACHECKSUMSWORKER_FAILED;=0A= +=0A+=09memset(&bgw,=200,=20sizeof(bgw));=0A+=09bgw.bgw_flags=20=3D=20= BGWORKER_SHMEM_ACCESS=20|=20BGWORKER_BACKEND_DATABASE_CONNECTION;=0A+=09= bgw.bgw_start_time=20=3D=20BgWorkerStart_RecoveryFinished;=0A+=09= snprintf(bgw.bgw_library_name,=20BGW_MAXLEN,=20"postgres");=0A+=09= snprintf(bgw.bgw_function_name,=20BGW_MAXLEN,=20"%s",=20= "DataChecksumsWorkerMain");=0A+=09snprintf(bgw.bgw_name,=20BGW_MAXLEN,=20= "datachecksumsworker=20worker");=0A+=09snprintf(bgw.bgw_type,=20= BGW_MAXLEN,=20"datachecksumsworker=20worker");=0A+=09= bgw.bgw_restart_time=20=3D=20BGW_NEVER_RESTART;=0A+=09bgw.bgw_notify_pid=20= =3D=20MyProcPid;=0A+=09bgw.bgw_main_arg=20=3D=20= ObjectIdGetDatum(db->dboid);=0A+=0A+=09/*=0A+=09=20*=20If=20there=20are=20= no=20worker=20slots=20available,=20make=20sure=20we=20retry=20processing=0A= +=09=20*=20this=20database.=20This=20will=20make=20the=20= datachecksumsworker=20move=20on=20to=20the=0A+=09=20*=20next=20database=20= and=20quite=20likely=20fail=20with=20the=20same=20problem.=20TODO:=20= Maybe=0A+=09=20*=20we=20need=20a=20backoff=20to=20avoid=20running=20= through=20all=20the=20databases=20here=20in=0A+=09=20*=20short=20order.=0A= +=09=20*/=0A+=09if=20(!RegisterDynamicBackgroundWorker(&bgw,=20= &bgw_handle))=0A+=09{=0A+=09=09ereport(WARNING,=0A+=09=09=09=09= errmsg("failed=20to=20start=20worker=20for=20enabling=20data=20checksums=20= in=20database=20\"%s\",=20retrying",=0A+=09=09=09=09=09=20=20=20= db->dbname),=0A+=09=09=09=09errhint("The=20max_worker_processes=20= setting=20might=20be=20too=20low."));=0A+=09=09return=20= DATACHECKSUMSWORKER_RETRYDB;=0A+=09}=0A+=0A+=09status=20=3D=20= WaitForBackgroundWorkerStartup(bgw_handle,=20&pid);=0A+=09if=20(status=20= =3D=3D=20BGWH_STOPPED)=0A+=09{=0A+=09=09ereport(WARNING,=0A+=09=09=09=09= errmsg("could=20not=20start=20background=20worker=20for=20enabling=20= data=20checksums=20in=20database=20\"%s\"",=0A+=09=09=09=09=09=20=20=20= db->dbname),=0A+=09=09=09=09errhint("More=20details=20on=20the=20error=20= might=20be=20found=20in=20the=20server=20log."));=0A+=09=09return=20= DATACHECKSUMSWORKER_FAILED;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20If=20the=20= postmaster=20crashed=20we=20cannot=20end=20up=20with=20a=20processed=20= database=20so=0A+=09=20*=20we=20have=20no=20alternative=20other=20than=20= exiting.=20When=20enabling=20checksums=20we=0A+=09=20*=20won't=20at=20= this=20time=20have=20changed=20the=20pg_control=20version=20to=20enabled=20= so=0A+=09=20*=20when=20the=20cluster=20comes=20back=20up=20processing=20= will=20have=20to=20be=20restarted.=0A+=09=20*=20When=20disabling,=20the=20= pg_control=20version=20will=20be=20set=20to=20off=20before=20this=0A+=09=20= *=20so=20when=20the=20cluster=20comes=20up=20checksums=20will=20be=20off=20= as=20expected.=0A+=09=20*/=0A+=09if=20(status=20=3D=3D=20= BGWH_POSTMASTER_DIED)=0A+=09=09ereport(FATAL,=0A+=09=09=09=09= errmsg("cannot=20enable=20data=20checksums=20without=20the=20postmaster=20= process"),=0A+=09=09=09=09errhint("Restart=20the=20database=20and=20= restart=20data=20checksum=20processing=20by=20calling=20= pg_enable_data_checksums()."));=0A+=0A+=09Assert(status=20=3D=3D=20= BGWH_STARTED);=0A+=09ereport(DEBUG1,=0A+=09=09=09errmsg("initiating=20= data=20checksum=20processing=20in=20database=20\"%s\"",=0A+=09=09=09=09=20= =20=20db->dbname));=0A+=0A+=09snprintf(activity,=20sizeof(activity)=20-=20= 1,=0A+=09=09=09=20"Waiting=20for=20worker=20in=20database=20%s=20(pid=20= %ld)",=20db->dbname,=20(long)=20pid);=0A+=09= pgstat_report_activity(STATE_RUNNING,=20activity);=0A+=0A+=09status=20=3D=20= WaitForBackgroundWorkerShutdown(bgw_handle);=0A+=09if=20(status=20=3D=3D=20= BGWH_POSTMASTER_DIED)=0A+=09=09ereport(FATAL,=0A+=09=09=09=09= errmsg("postmaster=20exited=20during=20data=20checksum=20processing=20in=20= \"%s\"",=0A+=09=09=09=09=09=20=20=20db->dbname),=0A+=09=09=09=09= errhint("Restart=20the=20database=20and=20restart=20data=20checksum=20= processing=20by=20calling=20pg_enable_data_checksums()."));=0A+=0A+=09if=20= (DataChecksumsWorkerShmem->success=20=3D=3D=20= DATACHECKSUMSWORKER_ABORTED)=0A+=09=09ereport(LOG,=0A+=09=09=09=09= errmsg("data=20checksums=20processing=20was=20aborted=20in=20database=20= \"%s\"",=0A+=09=09=09=09=09=20=20=20db->dbname));=0A+=0A+=09= pgstat_report_activity(STATE_IDLE,=20NULL);=0A+=0A+=09return=20= DataChecksumsWorkerShmem->success;=0A+}=0A+=0A+/*=0A+=20*=20= launcher_exit=0A+=20*=0A+=20*=20Internal=20routine=20for=20cleaning=20up=20= state=20when=20the=20launcher=20process=20exits.=20We=0A+=20*=20need=20= to=20clean=20up=20the=20abort=20flag=20to=20ensure=20that=20processing=20= can=20be=20restarted=0A+=20*=20again=20after=20it=20was=20previously=20= aborted.=0A+=20*/=0A+static=20void=0A+launcher_exit(int=20code,=20Datum=20= arg)=0A+{=0A+=09if=20(launcher_running)=0A+=09{=0A+=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09= launcher_running=20=3D=20false;=0A+=09=09= DataChecksumsWorkerShmem->launcher_running=20=3D=20false;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09}=0A+}=0A+=0A+/*=0A+=20*=20= launcher_cancel_handler=0A+=20*=0A+=20*=20Internal=20routine=20for=20= reacting=20to=20SIGINT=20and=20flagging=20the=20worker=20to=20abort.=0A+=20= *=20The=20worker=20won't=20be=20interrupted=20immediately=20but=20will=20= check=20for=20abort=20flag=0A+=20*=20between=20each=20block=20in=20a=20= relation.=0A+=20*/=0A+static=20void=0A= +launcher_cancel_handler(SIGNAL_ARGS)=0A+{=0A+=09int=09=09=09save_errno=20= =3D=20errno;=0A+=0A+=09abort_requested=20=3D=20true;=0A+=0A+=09/*=0A+=09=20= *=20There=20is=20no=20sleeping=20in=20the=20main=20loop,=20the=20flag=20= will=20be=20checked=0A+=09=20*=20periodically=20in=20= ProcessSingleRelationFork.=20The=20worker=20does=20however=0A+=09=20*=20= sleep=20when=20waiting=20for=20concurrent=20transactions=20to=20end=20so=20= we=20still=20need=0A+=09=20*=20to=20set=20the=20latch.=0A+=09=20*/=0A+=09= SetLatch(MyLatch);=0A+=0A+=09errno=20=3D=20save_errno;=0A+}=0A+=0A+/*=0A= +=20*=20WaitForAllTransactionsToFinish=0A+=20*=09=09Blocks=20awaiting=20= all=20current=20transactions=20to=20finish=0A+=20*=0A+=20*=20Returns=20= when=20all=20transactions=20which=20are=20active=20at=20the=20call=20of=20= the=20function=0A+=20*=20have=20ended,=20or=20if=20the=20postmaster=20= dies=20while=20waiting.=20If=20the=20postmaster=20dies=0A+=20*=20the=20= abort=20flag=20will=20be=20set=20to=20indicate=20that=20the=20caller=20= of=20this=20shouldn't=0A+=20*=20proceed.=0A+=20*=0A+=20*=20NB:=20this=20= will=20return=20early,=20if=20aborted=20by=20SIGINT=20or=20if=20the=20= target=20state=0A+=20*=20is=20changed=20while=20we're=20running.=0A+=20= */=0A+static=20void=0A+WaitForAllTransactionsToFinish(void)=0A+{=0A+=09= TransactionId=20waitforxid;=0A+=0A+=09LWLockAcquire(XidGenLock,=20= LW_SHARED);=0A+=09waitforxid=20=3D=20= XidFromFullTransactionId(TransamVariables->nextXid);=0A+=09= LWLockRelease(XidGenLock);=0A+=0A+=09while=20= (TransactionIdPrecedes(GetOldestActiveTransactionId(false,=20true),=20= waitforxid))=0A+=09{=0A+=09=09char=09=09activity[64];=0A+=09=09int=09=09=09= rc;=0A+=0A+=09=09/*=20Oldest=20running=20xid=20is=20older=20than=20us,=20= so=20wait=20*/=0A+=09=09snprintf(activity,=0A+=09=09=09=09=20= sizeof(activity),=0A+=09=09=09=09=20"Waiting=20for=20current=20= transactions=20to=20finish=20(waiting=20for=20%u)",=0A+=09=09=09=09=20= waitforxid);=0A+=09=09pgstat_report_activity(STATE_RUNNING,=20activity);=0A= +=0A+=09=09/*=20Retry=20every=203=20seconds=20*/=0A+=09=09= ResetLatch(MyLatch);=0A+=09=09rc=20=3D=20WaitLatch(MyLatch,=0A+=09=09=09=09= =09=20=20=20WL_LATCH_SET=20|=20WL_TIMEOUT=20|=20WL_POSTMASTER_DEATH,=0A+=09= =09=09=09=09=20=20=203000,=0A+=09=09=09=09=09=20=20=20= WAIT_EVENT_CHECKSUM_ENABLE_STARTCONDITION);=0A+=0A+=09=09/*=0A+=09=09=20= *=20If=20the=20postmaster=20died=20we=20won't=20be=20able=20to=20enable=20= checksums=0A+=09=09=20*=20cluster-wide=20so=20abort=20and=20hope=20to=20= continue=20when=20restarted.=0A+=09=09=20*/=0A+=09=09if=20(rc=20&=20= WL_POSTMASTER_DEATH)=0A+=09=09=09ereport(FATAL,=0A+=09=09=09=09=09= errmsg("postmaster=20exited=20during=20data=20checksum=20processing"),=0A= +=09=09=09=09=09errhint("Restart=20the=20database=20and=20restart=20data=20= checksum=20processing=20by=20calling=20pg_enable_data_checksums()."));=0A= +=0A+=09=09LWLockAcquire(DataChecksumsWorkerLock,=20LW_SHARED);=0A+=09=09= if=20(DataChecksumsWorkerShmem->launch_operation=20!=3D=20operation)=0A+=09= =09=09abort_requested=20=3D=20true;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09if=20(abort_requested)=0A= +=09=09=09break;=0A+=09}=0A+=0A+=09pgstat_report_activity(STATE_IDLE,=20= NULL);=0A+=09return;=0A+}=0A+=0A+/*=0A+=20*=20= DataChecksumsWorkerLauncherMain=0A+=20*=0A+=20*=20Main=20function=20for=20= launching=20dynamic=20background=20workers=20for=20processing=20data=0A+=20= *=20checksums=20in=20databases.=20This=20function=20has=20the=20bgworker=20= management,=20with=0A+=20*=20ProcessAllDatabases=20being=20responsible=20= for=20looping=20over=20the=20databases=20and=0A+=20*=20initiating=20= processing.=0A+=20*/=0A+void=0A+DataChecksumsWorkerLauncherMain(Datum=20= arg)=0A+{=0A+=09on_shmem_exit(launcher_exit,=200);=0A+=0A+=09= ereport(DEBUG1,=0A+=09=09=09errmsg("background=20worker=20= \"datachecksumsworker\"=20launcher=20started"));=0A+=0A+=09= pqsignal(SIGTERM,=20die);=0A+=09pqsignal(SIGINT,=20= launcher_cancel_handler);=0A+=0A+=09BackgroundWorkerUnblockSignals();=0A= +=0A+=09MyBackendType=20=3D=20B_DATACHECKSUMSWORKER_LAUNCHER;=0A+=09= init_ps_display(NULL);=0A+=0A+=09LWLockAcquire(DataChecksumsWorkerLock,=20= LW_EXCLUSIVE);=0A+=0A+=09if=20= (DataChecksumsWorkerShmem->launcher_running)=0A+=09{=0A+=09=09/*=20= Launcher=20was=20already=20running,=20let=20it=20finish=20*/=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09return;=0A+=09}=0A+=0A+=09= launcher_running=20=3D=20true;=0A+=0A+=09/*=0A+=09=20*=20Initialize=20a=20= connection=20to=20shared=20catalogs=20only.=0A+=09=20*/=0A+=09= BackgroundWorkerInitializeConnectionByOid(InvalidOid,=20InvalidOid,=20= 0);=0A+=0A+=09operation=20=3D=20= DataChecksumsWorkerShmem->launch_operation;=0A+=09= DataChecksumsWorkerShmem->launcher_running=20=3D=20true;=0A+=09= DataChecksumsWorkerShmem->operation=20=3D=20operation;=0A+=09= DataChecksumsWorkerShmem->cost_delay=20=3D=20= DataChecksumsWorkerShmem->launch_cost_delay;=0A+=09= DataChecksumsWorkerShmem->cost_limit=20=3D=20= DataChecksumsWorkerShmem->launch_cost_limit;=0A+=09= DataChecksumsWorkerShmem->immediate_checkpoint=20=3D=20= DataChecksumsWorkerShmem->launch_fast;=0A+=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09/*=0A+=09=20*=20The=20= target=20state=20can=20change=20while=20we=20are=20busy=20= enabling/disabling=0A+=09=20*=20checksums,=20if=20the=20user=20calls=20= pg_disable/enable_data_checksums()=20before=0A+=09=20*=20we=20are=20= finished=20with=20the=20previous=20request.=20In=20that=20case,=20we=20= will=20loop=0A+=09=20*=20back=20here,=20to=20process=20the=20new=20= request.=0A+=09=20*/=0A+again:=0A+=0A+=09= pgstat_progress_start_command(PROGRESS_COMMAND_DATACHECKSUMS,=0A+=09=09=09= =09=09=09=09=09=20=20InvalidOid);=0A+=0A+=09if=20(operation=20=3D=3D=20= ENABLE_DATACHECKSUMS)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20If=20we=20are=20= asked=20to=20enable=20checksums=20in=20a=20cluster=20which=20already=20= has=0A+=09=09=20*=20checksums=20enabled,=20exit=20immediately=20as=20= there=20is=20nothing=20more=20to=20do.=0A+=09=09=20*=20Hold=20interrupts=20= to=20make=20sure=20state=20doesn't=20change=20during=20checking.=0A+=09=09= =20*/=0A+=09=09HOLD_INTERRUPTS();=0A+=09=09if=20= (DataChecksumsNeedVerify())=0A+=09=09{=0A+=09=09=09RESUME_INTERRUPTS();=0A= +=09=09=09goto=20done;=0A+=09=09}=0A+=09=09RESUME_INTERRUPTS();=0A+=0A+=09= =09/*=0A+=09=09=20*=20Set=20the=20state=20to=20inprogress-on=20and=20= wait=20on=20the=20procsignal=20barrier.=0A+=09=09=20*/=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_ENABLING);=0A+=09=09= SetDataChecksumsOnInProgress();=0A+=0A+=09=09/*=0A+=09=09=20*=20All=20= backends=20are=20now=20in=20inprogress-on=20state=20and=20are=20writing=20= data=0A+=09=09=20*=20checksums.=20=20Start=20processing=20all=20data=20= at=20rest.=0A+=09=09=20*/=0A+=09=09if=20= (!ProcessAllDatabases(DataChecksumsWorkerShmem->immediate_checkpoint))=0A= +=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20If=20the=20target=20state=20= changed=20during=20processing=20then=20it's=20not=20a=0A+=09=09=09=20*=20= failure,=20so=20restart=20processing=20instead.=0A+=09=09=09=20*/=0A+=09=09= =09LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09=09= if=20(DataChecksumsWorkerShmem->launch_operation=20!=3D=20operation)=0A+=09= =09=09{=0A+=09=09=09=09LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09=09= =09goto=20done;=0A+=09=09=09}=0A+=09=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09=09ereport(ERROR,=0A+=09= =09=09=09=09errmsg("unable=20to=20enable=20data=20checksums=20in=20= cluster"));=0A+=09=09}=0A+=0A+=09=09/*=0A+=09=09=20*=20Data=20checksums=20= have=20been=20set=20on=20all=20pages,=20set=20the=20state=20to=20on=20in=0A= +=09=09=20*=20order=20to=20instruct=20backends=20to=20validate=20= checksums=20on=20reading.=0A+=09=09=20*/=0A+=09=09SetDataChecksumsOn();=0A= +=09}=0A+=09else=0A+=09{=0A+=09=09int=09=09=09flags;=0A+=0A+=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_DISABLING);=0A+=09=09= SetDataChecksumsOff();=0A+=0A+=09=09flags=20=3D=20CHECKPOINT_FORCE=20|=20= CHECKPOINT_WAIT;=0A+=09=09if=20= (DataChecksumsWorkerShmem->immediate_checkpoint)=0A+=09=09=09flags=20=3D=20= flags=20|=20CHECKPOINT_FAST;=0A+=09=09RequestCheckpoint(flags);=0A+=09}=0A= +=0A+done:=0A+=0A+=09/*=0A+=09=20*=20All=20done.=20But=20before=20we=20= exit,=20check=20if=20the=20target=20state=20was=20changed=0A+=09=20*=20= while=20we=20were=20running.=20In=20that=20case=20we=20will=20have=20to=20= start=20all=20over=0A+=09=20*=20again.=0A+=09=20*/=0A+=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09if=20= (DataChecksumsWorkerShmem->launch_operation=20!=3D=20operation)=0A+=09{=0A= +=09=09DataChecksumsWorkerShmem->operation=20=3D=20= DataChecksumsWorkerShmem->launch_operation;=0A+=09=09operation=20=3D=20= DataChecksumsWorkerShmem->launch_operation;=0A+=09=09= DataChecksumsWorkerShmem->cost_delay=20=3D=20= DataChecksumsWorkerShmem->launch_cost_delay;=0A+=09=09= DataChecksumsWorkerShmem->cost_limit=20=3D=20= DataChecksumsWorkerShmem->launch_cost_limit;=0A+=09=09= LWLockRelease(DataChecksumsWorkerLock);=0A+=09=09goto=20again;=0A+=09}=0A= +=0A+=09/*=20Shut=20down=20progress=20reporting=20as=20we=20are=20done=20= */=0A+=09pgstat_progress_end_command();=0A+=0A+=09launcher_running=20=3D=20= false;=0A+=09DataChecksumsWorkerShmem->launcher_running=20=3D=20false;=0A= +=09LWLockRelease(DataChecksumsWorkerLock);=0A+}=0A+=0A+/*=0A+=20*=20= ProcessAllDatabases=0A+=20*=09=09Compute=20the=20list=20of=20all=20= databases=20and=20process=20checksums=20in=20each=0A+=20*=0A+=20*=20This=20= will=20repeatedly=20generate=20a=20list=20of=20databases=20to=20process=20= for=20enabling=0A+=20*=20checksums.=20Until=20no=20new=20databases=20are=20= found,=20this=20will=20loop=20around=20computing=0A+=20*=20a=20new=20= list=20and=20comparing=20it=20to=20the=20already=20seen=20ones.=0A+=20*=0A= +=20*=20If=20immediate_checkpoint=20is=20set=20to=20true=20then=20a=20= CHECKPOINT_FAST=20will=20be=0A+=20*=20issued.=20This=20is=20useful=20for=20= testing=20but=20should=20be=20avoided=20in=20production=20use=0A+=20*=20= as=20it=20may=20affect=20cluster=20performance=20drastically.=0A+=20*/=0A= +static=20bool=0A+ProcessAllDatabases(bool=20immediate_checkpoint)=0A+{=0A= +=09List=09=20=20=20*DatabaseList;=0A+=09HTAB=09=20=20=20= *ProcessedDatabases=20=3D=20NULL;=0A+=09HASHCTL=09=09hash_ctl;=0A+=09= bool=09=09found_failed=20=3D=20false;=0A+=09int=09=09=09flags;=0A+=0A+=09= /*=20Initialize=20a=20hash=20tracking=20all=20processed=20databases=20*/=0A= +=09memset(&hash_ctl,=200,=20sizeof(hash_ctl));=0A+=09hash_ctl.keysize=20= =3D=20sizeof(Oid);=0A+=09hash_ctl.entrysize=20=3D=20= sizeof(DataChecksumsWorkerResultEntry);=0A+=09ProcessedDatabases=20=3D=20= hash_create("Processed=20databases",=0A+=09=09=09=09=09=09=09=09=09=20= 64,=0A+=09=09=09=09=09=09=09=09=09=20&hash_ctl,=0A+=09=09=09=09=09=09=09=09= =09=20HASH_ELEM=20|=20HASH_BLOBS);=0A+=0A+=09/*=0A+=09=20*=20Set=20up=20= so=20first=20run=20processes=20shared=20catalogs,=20but=20not=20once=20= in=20every=0A+=09=20*=20db.=0A+=09=20*/=0A+=09= DataChecksumsWorkerShmem->process_shared_catalogs=20=3D=20true;=0A+=0A+=09= /*=0A+=09=20*=20Get=20a=20list=20of=20all=20databases=20to=20process.=20= This=20may=20include=20databases=20that=0A+=09=20*=20were=20created=20= during=20our=20runtime.=20=20Since=20a=20database=20can=20be=20created=20= as=20a=0A+=09=20*=20copy=20of=20any=20other=20database=20(which=20may=20= not=20have=20existed=20in=20our=20last=0A+=09=20*=20run),=20we=20have=20= to=20repeat=20this=20loop=20until=20no=20new=20databases=20show=20up=20= in=20the=0A+=09=20*=20list.=0A+=09=20*/=0A+=09DatabaseList=20=3D=20= BuildDatabaseList();=0A+=0A+=09/*=20Allow=20a=20test=20case=20to=20= modify=20the=20initial=20list=20of=20databases=20*/=0A+=09= INJECTION_POINT("datachecksumsworker-initial-dblist",=20DatabaseList);=0A= +=0A+=09/*=0A+=09=20*=20Update=20progress=20reporting=20with=20the=20= total=20number=20of=20databases=20we=20need=20to=0A+=09=20*=20process.=20= =20This=20number=20should=20not=20be=20changed=20during=20processing,=20= the=0A+=09=20*=20columns=20for=20processed=20databases=20is=20instead=20= increased=20such=20that=20it=20can=0A+=09=20*=20be=20compared=20against=20= the=20total.=0A+=09=20*/=0A+=09{=0A+=09=09const=20int=09index[]=20=3D=20= {=0A+=09=09=09PROGRESS_DATACHECKSUMS_DBS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_DBS_DONE,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_DONE,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_BLOCKS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_BLOCKS_DONE,=0A+=09=09};=0A+=0A+=09=09int64=09=09= vals[6];=0A+=0A+=09=09vals[0]=20=3D=20list_length(DatabaseList);=0A+=09=09= vals[1]=20=3D=200;=0A+=0A+=09=09/*=20translated=20to=20NULL=20*/=0A+=09=09= vals[2]=20=3D=20-1;=0A+=09=09vals[3]=20=3D=20-1;=0A+=09=09vals[4]=20=3D=20= -1;=0A+=09=09vals[5]=20=3D=20-1;=0A+=0A+=09=09= pgstat_progress_update_multi_param(6,=20index,=20vals);=0A+=09}=0A+=0A+=09= while=20(true)=0A+=09{=0A+=09=09int=09=09=09processed_databases=20=3D=20= 0;=0A+=0A+=09=09foreach_ptr(DataChecksumsWorkerDatabase,=20db,=20= DatabaseList)=0A+=09=09{=0A+=09=09=09DataChecksumsWorkerResult=20result;=0A= +=09=09=09DataChecksumsWorkerResultEntry=20*entry;=0A+=09=09=09bool=09=09= found;=0A+=0A+=09=09=09/*=0A+=09=09=09=20*=20Check=20if=20this=20= database=20has=20been=20processed=20already,=20and=20if=20so=0A+=09=09=09= =20*=20whether=20it=20should=20be=20retried=20or=20skipped.=0A+=09=09=09=20= */=0A+=09=09=09entry=20=3D=20(DataChecksumsWorkerResultEntry=20*)=20= hash_search(ProcessedDatabases,=20&db->dboid,=0A+=09=09=09=09=09=09=09=09= =09=09=09=09=09=09=09=09=20=20=20HASH_FIND,=20NULL);=0A+=0A+=09=09=09if=20= (entry)=0A+=09=09=09{=0A+=09=09=09=09if=20(entry->result=20=3D=3D=20= DATACHECKSUMSWORKER_RETRYDB)=0A+=09=09=09=09{=0A+=09=09=09=09=09/*=0A+=09= =09=09=09=09=20*=20Limit=20the=20number=20of=20retries=20to=20avoid=20= infinite=20looping=0A+=09=09=09=09=09=20*=20in=20case=20there=20simply=20= won't=20be=20enough=20workers=20in=20the=0A+=09=09=09=09=09=20*=20= cluster=20to=20finish=20this=20operation.=0A+=09=09=09=09=09=20*/=0A+=09=09= =09=09=09if=20(entry->retries=20>=20DATACHECKSUMSWORKER_MAX_DB_RETRIES)=0A= +=09=09=09=09=09=09entry->result=20=3D=20DATACHECKSUMSWORKER_FAILED;=0A+=09= =09=09=09}=0A+=0A+=09=09=09=09/*=20Skip=20if=20this=20database=20has=20= been=20processed=20already=20*/=0A+=09=09=09=09if=20(entry->result=20!=3D=20= DATACHECKSUMSWORKER_RETRYDB)=0A+=09=09=09=09=09continue;=0A+=09=09=09}=0A= +=0A+=09=09=09result=20=3D=20ProcessDatabase(db);=0A+=09=09=09= processed_databases++;=0A+=0A+=09=09=09/*=0A+=09=09=09=20*=20Update=20= the=20number=20of=20processed=20databases=20in=20the=20progress=0A+=09=09= =09=20*=20report.=0A+=09=09=09=20*/=0A+=09=09=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_DBS_DONE,=0A+=09=09=09= =09=09=09=09=09=09=09=20processed_databases);=0A+=0A+=09=09=09/*=20Allow=20= a=20test=20process=20to=20alter=20the=20result=20of=20the=20operation=20= */=0A+=09=09=09INJECTION_POINT("datachecksumsworker-fail-db",=20= &result);=0A+=0A+=09=09=09if=20(result=20=3D=3D=20= DATACHECKSUMSWORKER_SUCCESSFUL)=0A+=09=09=09{=0A+=09=09=09=09/*=0A+=09=09= =09=09=20*=20If=20one=20database=20has=20completed=20shared=20catalogs,=20= we=20don't=0A+=09=09=09=09=20*=20have=20to=20process=20them=20again.=0A+=09= =09=09=09=20*/=0A+=09=09=09=09if=20= (DataChecksumsWorkerShmem->process_shared_catalogs)=0A+=09=09=09=09=09= DataChecksumsWorkerShmem->process_shared_catalogs=20=3D=20false;=0A+=09=09= =09}=0A+=09=09=09else=20if=20(result=20=3D=3D=20= DATACHECKSUMSWORKER_ABORTED)=0A+=09=09=09{=0A+=09=09=09=09/*=20Abort=20= flag=20set,=20so=20exit=20the=20whole=20process=20*/=0A+=09=09=09=09= return=20false;=0A+=09=09=09}=0A+=0A+=09=09=09entry=20=3D=20= hash_search(ProcessedDatabases,=20&db->dboid,=20HASH_ENTER,=20&found);=0A= +=09=09=09entry->dboid=20=3D=20db->dboid;=0A+=09=09=09entry->result=20=3D=20= result;=0A+=09=09=09if=20(!found)=0A+=09=09=09=09entry->retries=20=3D=20= 0;=0A+=09=09=09else=0A+=09=09=09=09entry->retries++;=0A+=09=09}=0A+=0A+=09= =09elog(DEBUG1,=0A+=09=09=09=20"%i=20databases=20processed=20for=20data=20= checksum=20enabling,=20%s",=0A+=09=09=09=20processed_databases,=0A+=09=09= =09=20(processed_databases=20?=20"process=20with=20restart"=20:=20= "process=20completed"));=0A+=0A+=09=09FreeDatabaseList(DatabaseList);=0A= +=0A+=09=09/*=0A+=09=09=20*=20If=20no=20databases=20were=20processed=20= in=20this=20run=20of=20the=20loop,=20we=20have=20now=0A+=09=09=20*=20= finished=20all=20databases=20and=20no=20concurrently=20created=20ones=20= can=20exist.=0A+=09=09=20*/=0A+=09=09if=20(processed_databases=20=3D=3D=20= 0)=0A+=09=09=09break;=0A+=0A+=09=09/*=0A+=09=09=20*=20Re-generate=20the=20= list=20of=20databases=20for=20another=20pass.=20Since=20we=20wait=0A+=09=09= =20*=20for=20all=20pre-existing=20transactions=20finish,=20this=20way=20= we=20can=20be=0A+=09=09=20*=20certain=20that=20there=20are=20no=20= databases=20left=20without=20checksums.=0A+=09=09=20*/=0A+=09=09= WaitForAllTransactionsToFinish();=0A+=09=09DatabaseList=20=3D=20= BuildDatabaseList();=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20= ProcessedDatabases=20now=20has=20all=20databases=20and=20the=20results=20= of=20their=0A+=09=20*=20processing.=20Failure=20to=20enable=20checksums=20= for=20a=20database=20can=20be=20because=0A+=09=20*=20they=20actually=20= failed=20for=20some=20reason,=20or=20because=20the=20database=20was=0A+=09= =20*=20dropped=20between=20us=20getting=20the=20database=20list=20and=20= trying=20to=20process=20it.=0A+=09=20*=20Get=20a=20fresh=20list=20of=20= databases=20to=20detect=20the=20second=20case=20where=20the=0A+=09=20*=20= database=20was=20dropped=20before=20we=20had=20started=20processing=20= it.=20If=20a=20database=0A+=09=20*=20still=20exists,=20but=20enabling=20= checksums=20failed=20then=20we=20fail=20the=20entire=0A+=09=20*=20= checksumming=20process=20and=20exit=20with=20an=20error.=0A+=09=20*/=0A+=09= WaitForAllTransactionsToFinish();=0A+=09DatabaseList=20=3D=20= BuildDatabaseList();=0A+=0A+=09foreach_ptr(DataChecksumsWorkerDatabase,=20= db,=20DatabaseList)=0A+=09{=0A+=09=09DataChecksumsWorkerResultEntry=20= *entry;=0A+=09=09bool=09=09found;=0A+=0A+=09=09entry=20=3D=20= hash_search(ProcessedDatabases,=20(void=20*)=20&db->dboid,=0A+=09=09=09=09= =09=09=09HASH_FIND,=20&found);=0A+=0A+=09=09/*=0A+=09=09=20*=20We=20are=20= only=20interested=20in=20the=20processed=20databases=20which=20failed,=20= and=0A+=09=09=20*=20where=20the=20failed=20database=20still=20exists.=20=20= This=20indicates=20that=0A+=09=09=20*=20enabling=20checksums=20actually=20= failed,=20and=20not=20that=20the=20failure=20was=0A+=09=09=20*=20due=20= to=20the=20db=20being=20concurrently=20dropped.=0A+=09=09=20*/=0A+=09=09= if=20(found=20&&=20entry->result=20=3D=3D=20DATACHECKSUMSWORKER_FAILED)=0A= +=09=09{=0A+=09=09=09ereport(WARNING,=0A+=09=09=09=09=09errmsg("failed=20= to=20enable=20data=20checksums=20in=20\"%s\"",=20db->dbname));=0A+=09=09=09= found_failed=20=3D=20found;=0A+=09=09=09continue;=0A+=09=09}=0A+=09}=0A+=0A= +=09FreeDatabaseList(DatabaseList);=0A+=0A+=09if=20(found_failed)=0A+=09= {=0A+=09=09/*=20Disable=20checksums=20on=20cluster,=20because=20we=20= failed=20*/=0A+=09=09SetDataChecksumsOff();=0A+=09=09/*=20Force=20a=20= checkpoint=20to=20make=20everything=20consistent=20*/=0A+=09=09flags=20=3D= =20CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT;=0A+=09=09if=20= (immediate_checkpoint)=0A+=09=09=09flags=20=3D=20flags=20|=20= CHECKPOINT_FAST;=0A+=09=09RequestCheckpoint(flags);=0A+=09=09= ereport(ERROR,=0A+=09=09=09=09errmsg("data=20checksums=20failed=20to=20= get=20enabled=20in=20all=20databases,=20aborting"),=0A+=09=09=09=09= errhint("The=20server=20log=20might=20have=20more=20information=20on=20= the=20cause=20of=20the=20error."));=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20= When=20enabling=20checksums,=20we=20have=20to=20wait=20for=20a=20= checkpoint=20for=20the=0A+=09=20*=20checksums=20to=20change=20from=20= in-progress=20to=20on.=0A+=09=20*/=0A+=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_WAITING_CHECKPOINT);=0A+=0A+=09= /*=0A+=09=20*=20Force=20a=20checkpoint=20to=20get=20everything=20out=20= to=20disk.=20The=20use=20of=20immediate=0A+=09=20*=20checkpoints=20is=20= for=20running=20tests,=20as=20they=20would=20otherwise=20not=20execute=0A= +=09=20*=20in=20such=20a=20way=20that=20they=20can=20reliably=20be=20= placed=20under=20timeout=20control.=0A+=09=20*/=0A+=09flags=20=3D=20= CHECKPOINT_FORCE=20|=20CHECKPOINT_WAIT;=0A+=09if=20= (immediate_checkpoint)=0A+=09=09flags=20=3D=20flags=20|=20= CHECKPOINT_FAST;=0A+=09RequestCheckpoint(flags);=0A+=0A+=09return=20= true;=0A+}=0A+=0A+/*=0A+=20*=20DataChecksumsWorkerShmemSize=0A+=20*=09=09= Compute=20required=20space=20for=20datachecksumsworker-related=20shared=20= memory=0A+=20*/=0A+Size=0A+DataChecksumsWorkerShmemSize(void)=0A+{=0A+=09= Size=09=09size;=0A+=0A+=09size=20=3D=20= sizeof(DataChecksumsWorkerShmemStruct);=0A+=09size=20=3D=20= MAXALIGN(size);=0A+=0A+=09return=20size;=0A+}=0A+=0A+/*=0A+=20*=20= DataChecksumsWorkerShmemInit=0A+=20*=09=09Allocate=20and=20initialize=20= datachecksumsworker-related=20shared=20memory=0A+=20*/=0A+void=0A= +DataChecksumsWorkerShmemInit(void)=0A+{=0A+=09bool=09=09found;=0A+=0A+=09= DataChecksumsWorkerShmem=20=3D=20(DataChecksumsWorkerShmemStruct=20*)=0A= +=09=09ShmemInitStruct("DataChecksumsWorker=20Data",=0A+=09=09=09=09=09=09= DataChecksumsWorkerShmemSize(),=0A+=09=09=09=09=09=09&found);=0A+=0A+=09= if=20(!found)=0A+=09{=0A+=09=09MemSet(DataChecksumsWorkerShmem,=200,=20= DataChecksumsWorkerShmemSize());=0A+=0A+=09=09/*=0A+=09=09=20*=20Even=20= if=20this=20is=20a=20redundant=20assignment,=20we=20want=20to=20be=20= explicit=0A+=09=09=20*=20about=20our=20intent=20for=20readability,=20= since=20we=20want=20to=20be=20able=20to=20query=0A+=09=09=20*=20this=20= state=20in=20case=20of=20restartability.=0A+=09=09=20*/=0A+=09=09= DataChecksumsWorkerShmem->launch_operation=20=3D=20false;=0A+=09=09= DataChecksumsWorkerShmem->launcher_running=20=3D=20false;=0A+=09=09= DataChecksumsWorkerShmem->launch_fast=20=3D=20false;=0A+=09}=0A+}=0A+=0A= +/*=0A+=20*=20BuildDatabaseList=0A+=20*=09=09Compile=20a=20list=20of=20= all=20currently=20available=20databases=20in=20the=20cluster=0A+=20*=0A+=20= *=20This=20creates=20the=20list=20of=20databases=20for=20the=20= datachecksumsworker=20workers=20to=0A+=20*=20add=20checksums=20to.=20If=20= the=20caller=20wants=20to=20ensure=20that=20no=20concurrently=0A+=20*=20= running=20CREATE=20DATABASE=20calls=20exist,=20this=20needs=20to=20be=20= preceded=20by=20a=20call=0A+=20*=20to=20= WaitForAllTransactionsToFinish().=0A+=20*/=0A+static=20List=20*=0A= +BuildDatabaseList(void)=0A+{=0A+=09List=09=20=20=20*DatabaseList=20=3D=20= NIL;=0A+=09Relation=09rel;=0A+=09TableScanDesc=20scan;=0A+=09HeapTuple=09= tup;=0A+=09MemoryContext=20ctx=20=3D=20CurrentMemoryContext;=0A+=09= MemoryContext=20oldctx;=0A+=0A+=09StartTransactionCommand();=0A+=0A+=09= rel=20=3D=20table_open(DatabaseRelationId,=20AccessShareLock);=0A+=09= scan=20=3D=20table_beginscan_catalog(rel,=200,=20NULL);=0A+=0A+=09while=20= (HeapTupleIsValid(tup=20=3D=20heap_getnext(scan,=20= ForwardScanDirection)))=0A+=09{=0A+=09=09Form_pg_database=20pgdb=20=3D=20= (Form_pg_database)=20GETSTRUCT(tup);=0A+=09=09= DataChecksumsWorkerDatabase=20*db;=0A+=0A+=09=09oldctx=20=3D=20= MemoryContextSwitchTo(ctx);=0A+=0A+=09=09db=20=3D=20= (DataChecksumsWorkerDatabase=20*)=20= palloc0(sizeof(DataChecksumsWorkerDatabase));=0A+=0A+=09=09db->dboid=20=3D= =20pgdb->oid;=0A+=09=09db->dbname=20=3D=20= pstrdup(NameStr(pgdb->datname));=0A+=0A+=09=09DatabaseList=20=3D=20= lappend(DatabaseList,=20db);=0A+=0A+=09=09MemoryContextSwitchTo(oldctx);=0A= +=09}=0A+=0A+=09table_endscan(scan);=0A+=09table_close(rel,=20= AccessShareLock);=0A+=0A+=09CommitTransactionCommand();=0A+=0A+=09return=20= DatabaseList;=0A+}=0A+=0A+static=20void=0A+FreeDatabaseList(List=20= *dblist)=0A+{=0A+=09if=20(!dblist)=0A+=09=09return;=0A+=0A+=09= foreach_ptr(DataChecksumsWorkerDatabase,=20db,=20dblist)=0A+=09{=0A+=09=09= if=20(db->dbname=20!=3D=20NULL)=0A+=09=09=09pfree(db->dbname);=0A+=09}=0A= +=0A+=09list_free_deep(dblist);=0A+}=0A+=0A+/*=0A+=20*=20= BuildRelationList=0A+=20*=09=09Compile=20a=20list=20of=20relations=20in=20= the=20database=0A+=20*=0A+=20*=20Returns=20a=20list=20of=20OIDs=20for=20= the=20request=20relation=20types.=20If=20temp_relations=0A+=20*=20is=20= True=20then=20only=20temporary=20relations=20are=20returned.=20If=20= temp_relations=20is=0A+=20*=20False=20then=20non-temporary=20relations=20= which=20have=20data=20checksums=20are=20returned.=0A+=20*=20If=20= include_shared=20is=20True=20then=20shared=20relations=20are=20included=20= as=20well=20in=20a=0A+=20*=20non-temporary=20list.=20include_shared=20= has=20no=20relevance=20when=20building=20a=20list=20of=0A+=20*=20= temporary=20relations.=0A+=20*/=0A+static=20List=20*=0A= +BuildRelationList(bool=20temp_relations,=20bool=20include_shared)=0A+{=0A= +=09List=09=20=20=20*RelationList=20=3D=20NIL;=0A+=09Relation=09rel;=0A+=09= TableScanDesc=20scan;=0A+=09HeapTuple=09tup;=0A+=09MemoryContext=20ctx=20= =3D=20CurrentMemoryContext;=0A+=09MemoryContext=20oldctx;=0A+=0A+=09= StartTransactionCommand();=0A+=0A+=09rel=20=3D=20= table_open(RelationRelationId,=20AccessShareLock);=0A+=09scan=20=3D=20= table_beginscan_catalog(rel,=200,=20NULL);=0A+=0A+=09while=20= (HeapTupleIsValid(tup=20=3D=20heap_getnext(scan,=20= ForwardScanDirection)))=0A+=09{=0A+=09=09Form_pg_class=20pgc=20=3D=20= (Form_pg_class)=20GETSTRUCT(tup);=0A+=0A+=09=09/*=0A+=09=09=20*=20Only=20= include=20temporary=20relations=20when=20asked=20for=20a=20temp=20= relation=0A+=09=09=20*=20list.=0A+=09=09=20*/=0A+=09=09if=20= (pgc->relpersistence=20=3D=3D=20RELPERSISTENCE_TEMP)=0A+=09=09{=0A+=09=09= =09if=20(!temp_relations)=0A+=09=09=09=09continue;=0A+=09=09}=0A+=09=09= else=0A+=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20If=20we=20are=20only=20= interested=20in=20temp=20relations=20then=20continue=0A+=09=09=09=20*=20= immediately=20as=20the=20current=20relation=20isn't=20a=20temp=20= relation.=0A+=09=09=09=20*/=0A+=09=09=09if=20(temp_relations)=0A+=09=09=09= =09continue;=0A+=0A+=09=09=09if=20(!RELKIND_HAS_STORAGE(pgc->relkind))=0A= +=09=09=09=09continue;=0A+=0A+=09=09=09if=20(pgc->relisshared=20&&=20= !include_shared)=0A+=09=09=09=09continue;=0A+=09=09}=0A+=0A+=09=09oldctx=20= =3D=20MemoryContextSwitchTo(ctx);=0A+=09=09RelationList=20=3D=20= lappend_oid(RelationList,=20pgc->oid);=0A+=09=09= MemoryContextSwitchTo(oldctx);=0A+=09}=0A+=0A+=09table_endscan(scan);=0A= +=09table_close(rel,=20AccessShareLock);=0A+=0A+=09= CommitTransactionCommand();=0A+=0A+=09return=20RelationList;=0A+}=0A+=0A= +/*=0A+=20*=20DataChecksumsWorkerMain=0A+=20*=0A+=20*=20Main=20function=20= for=20enabling=20checksums=20in=20a=20single=20database,=20This=20is=20= the=0A+=20*=20function=20set=20as=20the=20bgw_function_name=20in=20the=20= dynamic=20background=20worker=0A+=20*=20process=20initiated=20for=20each=20= database=20by=20the=20worker=20launcher.=20After=20enabling=0A+=20*=20= data=20checksums=20in=20each=20applicable=20relation=20in=20the=20= database,=20it=20will=20wait=20for=0A+=20*=20all=20temporary=20relations=20= that=20were=20present=20when=20the=20function=20started=20to=0A+=20*=20= disappear=20before=20returning.=20This=20is=20required=20since=20we=20= cannot=20rewrite=0A+=20*=20existing=20temporary=20relations=20with=20= data=20checksums.=0A+=20*/=0A+void=0A+DataChecksumsWorkerMain(Datum=20= arg)=0A+{=0A+=09Oid=09=09=09dboid=20=3D=20DatumGetObjectId(arg);=0A+=09= List=09=20=20=20*RelationList=20=3D=20NIL;=0A+=09List=09=20=20=20= *InitialTempTableList=20=3D=20NIL;=0A+=09BufferAccessStrategy=20= strategy;=0A+=09bool=09=09aborted=20=3D=20false;=0A+=09int64=09=09= rels_done;=0A+=0A+=09operation=20=3D=20ENABLE_DATACHECKSUMS;=0A+=0A+=09= pqsignal(SIGTERM,=20die);=0A+=0A+=09BackgroundWorkerUnblockSignals();=0A= +=0A+=09MyBackendType=20=3D=20B_DATACHECKSUMSWORKER_WORKER;=0A+=09= init_ps_display(NULL);=0A+=0A+=09= BackgroundWorkerInitializeConnectionByOid(dboid,=20InvalidOid,=0A+=09=09=09= =09=09=09=09=09=09=09=09=20=20BGWORKER_BYPASS_ALLOWCONN);=0A+=0A+=09/*=20= worker=20will=20have=20a=20separate=20entry=20in=20= pg_stat_progress_data_checksums=20*/=0A+=09= pgstat_progress_start_command(PROGRESS_COMMAND_DATACHECKSUMS,=0A+=09=09=09= =09=09=09=09=09=20=20InvalidOid);=0A+=0A+=09/*=0A+=09=20*=20Get=20a=20= list=20of=20all=20temp=20tables=20present=20as=20we=20start=20in=20this=20= database.=20We=0A+=09=20*=20need=20to=20wait=20until=20they=20are=20all=20= gone=20until=20we=20are=20done,=20since=20we=20cannot=0A+=09=20*=20= access=20these=20relations=20and=20modify=20them.=0A+=09=20*/=0A+=09= InitialTempTableList=20=3D=20BuildRelationList(true,=20false);=0A+=0A+=09= /*=0A+=09=20*=20Enable=20vacuum=20cost=20delay,=20if=20any.=0A+=09=20*/=0A= +=09Assert(DataChecksumsWorkerShmem->operation=20=3D=3D=20= ENABLE_DATACHECKSUMS);=0A+=09VacuumCostDelay=20=3D=20= DataChecksumsWorkerShmem->cost_delay;=0A+=09VacuumCostLimit=20=3D=20= DataChecksumsWorkerShmem->cost_limit;=0A+=09VacuumCostActive=20=3D=20= (VacuumCostDelay=20>=200);=0A+=09VacuumCostBalance=20=3D=200;=0A+=09= VacuumCostPageHit=20=3D=200;=0A+=09VacuumCostPageMiss=20=3D=200;=0A+=09= VacuumCostPageDirty=20=3D=200;=0A+=0A+=09/*=0A+=09=20*=20Create=20and=20= set=20the=20vacuum=20strategy=20as=20our=20buffer=20strategy.=0A+=09=20= */=0A+=09strategy=20=3D=20GetAccessStrategy(BAS_VACUUM);=0A+=0A+=09= RelationList=20=3D=20BuildRelationList(false,=0A+=09=09=09=09=09=09=09=09= =09=20DataChecksumsWorkerShmem->process_shared_catalogs);=0A+=0A+=09/*=20= Update=20the=20total=20number=20of=20relations=20to=20be=20processed=20= in=20this=20DB.=20*/=0A+=09{=0A+=09=09const=20int=09index[]=20=3D=20{=0A= +=09=09=09PROGRESS_DATACHECKSUMS_RELS_TOTAL,=0A+=09=09=09= PROGRESS_DATACHECKSUMS_RELS_DONE=0A+=09=09};=0A+=0A+=09=09int64=09=09= vals[2];=0A+=0A+=09=09vals[0]=20=3D=20list_length(RelationList);=0A+=09=09= vals[1]=20=3D=200;=0A+=0A+=09=09pgstat_progress_update_multi_param(2,=20= index,=20vals);=0A+=09}=0A+=0A+=09/*=20Process=20the=20relations=20*/=0A= +=09rels_done=20=3D=200;=0A+=09foreach_oid(reloid,=20RelationList)=0A+=09= {=0A+=09=09if=20(!ProcessSingleRelationByOid(reloid,=20strategy))=0A+=09=09= {=0A+=09=09=09aborted=20=3D=20true;=0A+=09=09=09break;=0A+=09=09}=0A+=0A= +=09=09pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_RELS_DONE,=0A= +=09=09=09=09=09=09=09=09=09=20++rels_done);=0A+=09}=0A+=09= list_free(RelationList);=0A+=0A+=09if=20(aborted)=0A+=09{=0A+=09=09= DataChecksumsWorkerShmem->success=20=3D=20DATACHECKSUMSWORKER_ABORTED;=0A= +=09=09ereport(DEBUG1,=0A+=09=09=09=09errmsg("data=20checksum=20= processing=20aborted=20in=20database=20OID=20%u",=0A+=09=09=09=09=09=20=20= =20dboid));=0A+=09=09return;=0A+=09}=0A+=0A+=09/*=20The=20worker=20is=20= about=20to=20wait=20for=20temporary=20tables=20to=20go=20away.=20*/=0A+=09= pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE,=0A+=09=09=09=09= =09=09=09=09=20PROGRESS_DATACHECKSUMS_PHASE_WAITING_TEMPREL);=0A+=0A+=09= /*=0A+=09=20*=20Wait=20for=20all=20temp=20tables=20that=20existed=20when=20= we=20started=20to=20go=20away.=20This=0A+=09=20*=20is=20necessary=20= since=20we=20cannot=20"reach"=20them=20to=20enable=20checksums.=20Any=20= temp=0A+=09=20*=20tables=20created=20after=20we=20started=20will=20= already=20have=20checksums=20in=20them=0A+=09=20*=20(due=20to=20the=20= "inprogress-on"=20state),=20so=20no=20need=20to=20wait=20for=20those.=0A= +=09=20*/=0A+=09for=20(;;)=0A+=09{=0A+=09=09List=09=20=20=20= *CurrentTempTables;=0A+=09=09int=09=09=09numleft;=0A+=09=09char=09=09= activity[64];=0A+=0A+=09=09CurrentTempTables=20=3D=20= BuildRelationList(true,=20false);=0A+=09=09numleft=20=3D=200;=0A+=09=09= foreach_oid(tmptbloid,=20InitialTempTableList)=0A+=09=09{=0A+=09=09=09if=20= (list_member_oid(CurrentTempTables,=20tmptbloid))=0A+=09=09=09=09= numleft++;=0A+=09=09}=0A+=09=09list_free(CurrentTempTables);=0A+=0A+=09=09= INJECTION_POINT("datachecksumsworker-fake-temptable-wait",=20&numleft);=0A= +=0A+=09=09if=20(numleft=20=3D=3D=200)=0A+=09=09=09break;=0A+=0A+=09=09= /*=0A+=09=09=20*=20At=20least=20one=20temp=20table=20is=20left=20to=20= wait=20for,=20indicate=20in=20pgstat=0A+=09=09=20*=20activity=20and=20= progress=20reporting.=0A+=09=09=20*/=0A+=09=09snprintf(activity,=0A+=09=09= =09=09=20sizeof(activity),=0A+=09=09=09=09=20"Waiting=20for=20%d=20temp=20= tables=20to=20be=20removed",=20numleft);=0A+=09=09= pgstat_report_activity(STATE_RUNNING,=20activity);=0A+=0A+=09=09/*=20= Retry=20every=203=20seconds=20*/=0A+=09=09ResetLatch(MyLatch);=0A+=09=09= (void)=20WaitLatch(MyLatch,=0A+=09=09=09=09=09=09=20WL_LATCH_SET=20|=20= WL_TIMEOUT=20|=20WL_EXIT_ON_PM_DEATH,=0A+=09=09=09=09=09=09=203000,=0A+=09= =09=09=09=09=09=20WAIT_EVENT_CHECKSUM_ENABLE_TEMPTABLE_WAIT);=0A+=0A+=09=09= LWLockAcquire(DataChecksumsWorkerLock,=20LW_EXCLUSIVE);=0A+=09=09aborted=20= =3D=20DataChecksumsWorkerShmem->launch_operation=20!=3D=20operation;=0A+=09= =09LWLockRelease(DataChecksumsWorkerLock);=0A+=0A+=09=09if=20(aborted=20= ||=20abort_requested)=0A+=09=09{=0A+=09=09=09= DataChecksumsWorkerShmem->success=20=3D=20DATACHECKSUMSWORKER_ABORTED;=0A= +=09=09=09ereport(DEBUG1,=0A+=09=09=09=09=09errmsg("data=20checksum=20= processing=20aborted=20in=20database=20OID=20%u",=0A+=09=09=09=09=09=09=20= =20=20dboid));=0A+=09=09=09return;=0A+=09=09}=0A+=09}=0A+=0A+=09= list_free(InitialTempTableList);=0A+=0A+=09/*=20worker=20done=20*/=0A+=09= pgstat_progress_end_command();=0A+=0A+=09= DataChecksumsWorkerShmem->success=20=3D=20= DATACHECKSUMSWORKER_SUCCESSFUL;=0A+}=0Adiff=20--git=20= a/src/backend/postmaster/launch_backend.c=20= b/src/backend/postmaster/launch_backend.c=0Aindex=20= bf6b55ee830..955df32be5d=20100644=0A---=20= a/src/backend/postmaster/launch_backend.c=0A+++=20= b/src/backend/postmaster/launch_backend.c=0A@@=20-204,6=20+204,9=20@@=20= static=20child_process_kind=20child_process_kinds[]=20=3D=20{=0A=20=09= [B_WAL_SUMMARIZER]=20=3D=20{"wal_summarizer",=20WalSummarizerMain,=20= true},=0A=20=09[B_WAL_WRITER]=20=3D=20{"wal_writer",=20WalWriterMain,=20= true},=0A=20=0A+=09[B_DATACHECKSUMSWORKER_LAUNCHER]=20=3D=20= {"datachecksum=20launcher",=20NULL,=20false},=0A+=09= [B_DATACHECKSUMSWORKER_WORKER]=20=3D=20{"datachecksum=20worker",=20NULL,=20= false},=0A+=0A=20=09[B_LOGGER]=20=3D=20{"syslogger",=20SysLoggerMain,=20= false},=0A=20};=0A=20=0Adiff=20--git=20= a/src/backend/postmaster/meson.build=20= b/src/backend/postmaster/meson.build=0Aindex=200008603cfee..ce10ef1059a=20= 100644=0A---=20a/src/backend/postmaster/meson.build=0A+++=20= b/src/backend/postmaster/meson.build=0A@@=20-6,6=20+6,7=20@@=20= backend_sources=20+=3D=20files(=0A=20=20=20'bgworker.c',=0A=20=20=20= 'bgwriter.c',=0A=20=20=20'checkpointer.c',=0A+=20=20= 'datachecksumsworker.c',=0A=20=20=20'fork_process.c',=0A=20=20=20= 'interrupt.c',=0A=20=20=20'launch_backend.c',=0Adiff=20--git=20= a/src/backend/postmaster/postmaster.c=20= b/src/backend/postmaster/postmaster.c=0Aindex=20e01d9f0cfe8..f8303992ca5=20= 100644=0A---=20a/src/backend/postmaster/postmaster.c=0A+++=20= b/src/backend/postmaster/postmaster.c=0A@@=20-2980,6=20+2980,11=20@@=20= PostmasterStateMachine(void)=0A=20=09=09=09=09=09=09=09=09=09B_INVALID,=0A= =20=09=09=09=09=09=09=09=09=09B_STANDALONE_BACKEND);=0A=20=0A+=09=09=09= /*=20also=20add=20checksumming=20processes=20*/=0A+=09=09=09remainMask=20= =3D=20btmask_add(remainMask,=0A+=09=09=09=09=09=09=09=09=09= B_DATACHECKSUMSWORKER_LAUNCHER,=0A+=09=09=09=09=09=09=09=09=09= B_DATACHECKSUMSWORKER_WORKER);=0A+=0A=20=09=09=09/*=20All=20types=20= should=20be=20included=20in=20targetMask=20or=20remainMask=20*/=0A=20=09=09= =09Assert((remainMask.mask=20|=20targetMask.mask)=20=3D=3D=20= BTYPE_MASK_ALL.mask);=0A=20=09=09}=0Adiff=20--git=20= a/src/backend/replication/logical/decode.c=20= b/src/backend/replication/logical/decode.c=0Aindex=20= cc03f0706e9..f9f06821a8f=20100644=0A---=20= a/src/backend/replication/logical/decode.c=0A+++=20= b/src/backend/replication/logical/decode.c=0A@@=20-186,6=20+186,7=20@@=20= xlog_decode(LogicalDecodingContext=20*ctx,=20XLogRecordBuffer=20*buf)=0A=20= =09=09case=20XLOG_FPW_CHANGE:=0A=20=09=09case=20XLOG_FPI_FOR_HINT:=0A=20=09= =09case=20XLOG_FPI:=0A+=09=09case=20XLOG_CHECKSUMS:=0A=20=09=09case=20= XLOG_OVERWRITE_CONTRECORD:=0A=20=09=09case=20XLOG_CHECKPOINT_REDO:=0A=20=09= =09=09break;=0Adiff=20--git=20a/src/backend/storage/ipc/ipci.c=20= b/src/backend/storage/ipc/ipci.c=0Aindex=202fa045e6b0f..44213d140ae=20= 100644=0A---=20a/src/backend/storage/ipc/ipci.c=0A+++=20= b/src/backend/storage/ipc/ipci.c=0A@@=20-30,6=20+30,8=20@@=0A=20#include=20= "postmaster/autovacuum.h"=0A=20#include=20= "postmaster/bgworker_internals.h"=0A=20#include=20= "postmaster/bgwriter.h"=0A+#include=20"postmaster/datachecksumsworker.h"=0A= +#include=20"postmaster/postmaster.h"=0A=20#include=20= "postmaster/walsummarizer.h"=0A=20#include=20= "replication/logicallauncher.h"=0A=20#include=20"replication/origin.h"=0A= @@=20-150,6=20+152,7=20@@=20CalculateShmemSize(int=20*num_semaphores)=0A=20= =09size=20=3D=20add_size(size,=20InjectionPointShmemSize());=0A=20=09= size=20=3D=20add_size(size,=20SlotSyncShmemSize());=0A=20=09size=20=3D=20= add_size(size,=20AioShmemSize());=0A+=09size=20=3D=20add_size(size,=20= DataChecksumsWorkerShmemSize());=0A=20=0A=20=09/*=20include=20additional=20= requested=20shmem=20from=20preload=20libraries=20*/=0A=20=09size=20=3D=20= add_size(size,=20total_addin_request);=0A@@=20-332,6=20+335,7=20@@=20= CreateOrAttachShmemStructs(void)=0A=20=09PgArchShmemInit();=0A=20=09= ApplyLauncherShmemInit();=0A=20=09SlotSyncShmemInit();=0A+=09= DataChecksumsWorkerShmemInit();=0A=20=0A=20=09/*=0A=20=09=20*=20Set=20up=20= other=20modules=20that=20need=20some=20shared=20memory=20space=0Adiff=20= --git=20a/src/backend/storage/ipc/procsignal.c=20= b/src/backend/storage/ipc/procsignal.c=0Aindex=20= 087821311cc..6881c6f4069=20100644=0A---=20= a/src/backend/storage/ipc/procsignal.c=0A+++=20= b/src/backend/storage/ipc/procsignal.c=0A@@=20-18,6=20+18,7=20@@=0A=20= #include=20=0A=20=0A=20#include=20"access/parallel.h"=0A= +#include=20"access/xlog.h"=0A=20#include=20"commands/async.h"=0A=20= #include=20"miscadmin.h"=0A=20#include=20"pgstat.h"=0A@@=20-576,6=20= +577,18=20@@=20ProcessProcSignalBarrier(void)=0A=20=09=09=09=09=09case=20= PROCSIGNAL_BARRIER_SMGRRELEASE:=0A=20=09=09=09=09=09=09processed=20=3D=20= ProcessBarrierSmgrRelease();=0A=20=09=09=09=09=09=09break;=0A+=09=09=09=09= =09case=20PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON:=0A+=09=09=09=09=09=09= processed=20=3D=20AbsorbChecksumsOnInProgressBarrier();=0A+=09=09=09=09=09= =09break;=0A+=09=09=09=09=09case=20PROCSIGNAL_BARRIER_CHECKSUM_ON:=0A+=09= =09=09=09=09=09processed=20=3D=20AbsorbChecksumsOnBarrier();=0A+=09=09=09= =09=09=09break;=0A+=09=09=09=09=09case=20= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF:=0A+=09=09=09=09=09=09= processed=20=3D=20AbsorbChecksumsOffInProgressBarrier();=0A+=09=09=09=09=09= =09break;=0A+=09=09=09=09=09case=20PROCSIGNAL_BARRIER_CHECKSUM_OFF:=0A+=09= =09=09=09=09=09processed=20=3D=20AbsorbChecksumsOffBarrier();=0A+=09=09=09= =09=09=09break;=0A=20=09=09=09=09}=0A=20=0A=20=09=09=09=09/*=0Adiff=20= --git=20a/src/backend/storage/page/README=20= b/src/backend/storage/page/README=0Aindex=20e30d7ac59ad..73c36a63908=20= 100644=0A---=20a/src/backend/storage/page/README=0A+++=20= b/src/backend/storage/page/README=0A@@=20-10,7=20+10,9=20@@=20= http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf,=20discussed=0A= =202010/12/22=20on=20-hackers=20list.=0A=20=0A=20Current=20= implementation=20requires=20this=20be=20enabled=20system-wide=20at=20= initdb=20time,=20or=0A-by=20using=20the=20pg_checksums=20tool=20on=20an=20= offline=20cluster.=0A+by=20using=20the=20pg_checksums=20tool=20on=20an=20= offline=20cluster.=20=20Checksums=20can=20also=20be=0A+enabled=20at=20= runtime=20using=20pg_enable_data_checksums(),=20and=20disabled=20by=20= using=0A+pg_disable_data_checksums().=0A=20=0A=20The=20checksum=20is=20= not=20valid=20at=20all=20times=20on=20a=20data=20page!!=0A=20The=20= checksum=20is=20valid=20when=20the=20page=20leaves=20the=20shared=20pool=20= and=20is=20checked=0Adiff=20--git=20a/src/backend/storage/page/bufpage.c=20= b/src/backend/storage/page/bufpage.c=0Aindex=20dbb49ed9197..19cf6512e52=20= 100644=0A---=20a/src/backend/storage/page/bufpage.c=0A+++=20= b/src/backend/storage/page/bufpage.c=0A@@=20-107,7=20+107,7=20@@=20= PageIsVerified(PageData=20*page,=20BlockNumber=20blkno,=20int=20flags,=20= bool=20*checksum_fail=0A=20=09=20*/=0A=20=09if=20(!PageIsNew(page))=0A=20= =09{=0A-=09=09if=20(DataChecksumsEnabled())=0A+=09=09if=20= (DataChecksumsNeedVerify())=0A=20=09=09{=0A=20=09=09=09checksum=20=3D=20= pg_checksum_page(page,=20blkno);=0A=20=0A@@=20-1511,7=20+1511,7=20@@=20= PageSetChecksumCopy(Page=20page,=20BlockNumber=20blkno)=0A=20=09static=20= char=20*pageCopy=20=3D=20NULL;=0A=20=0A=20=09/*=20If=20we=20don't=20need=20= a=20checksum,=20just=20return=20the=20passed-in=20data=20*/=0A-=09if=20= (PageIsNew(page)=20||=20!DataChecksumsEnabled())=0A+=09if=20= (PageIsNew(page)=20||=20!DataChecksumsNeedWrite())=0A=20=09=09return=20= page;=0A=20=0A=20=09/*=0A@@=20-1541,7=20+1541,7=20@@=20void=0A=20= PageSetChecksumInplace(Page=20page,=20BlockNumber=20blkno)=0A=20{=0A=20=09= /*=20If=20we=20don't=20need=20a=20checksum,=20just=20return=20*/=0A-=09= if=20(PageIsNew(page)=20||=20!DataChecksumsEnabled())=0A+=09if=20= (PageIsNew(page)=20||=20!DataChecksumsNeedWrite())=0A=20=09=09return;=0A=20= =0A=20=09((PageHeader)=20page)->pd_checksum=20=3D=20= pg_checksum_page(page,=20blkno);=0Adiff=20--git=20= a/src/backend/utils/activity/pgstat_backend.c=20= b/src/backend/utils/activity/pgstat_backend.c=0Aindex=20= 8714a85e2d9..edc2512d79f=20100644=0A---=20= a/src/backend/utils/activity/pgstat_backend.c=0A+++=20= b/src/backend/utils/activity/pgstat_backend.c=0A@@=20-378,6=20+378,8=20= @@=20pgstat_tracks_backend_bktype(BackendType=20bktype)=0A=20=09=09case=20= B_CHECKPOINTER:=0A=20=09=09case=20B_IO_WORKER:=0A=20=09=09case=20= B_STARTUP:=0A+=09=09case=20B_DATACHECKSUMSWORKER_LAUNCHER:=0A+=09=09case=20= B_DATACHECKSUMSWORKER_WORKER:=0A=20=09=09=09return=20false;=0A=20=0A=20=09= =09case=20B_AUTOVAC_WORKER:=0Adiff=20--git=20= a/src/backend/utils/activity/pgstat_io.c=20= b/src/backend/utils/activity/pgstat_io.c=0Aindex=20= 13ae57ed649..a290d56f409=20100644=0A---=20= a/src/backend/utils/activity/pgstat_io.c=0A+++=20= b/src/backend/utils/activity/pgstat_io.c=0A@@=20-362,6=20+362,8=20@@=20= pgstat_tracks_io_bktype(BackendType=20bktype)=0A=20=09=09case=20= B_LOGGER:=0A=20=09=09=09return=20false;=0A=20=0A+=09=09case=20= B_DATACHECKSUMSWORKER_LAUNCHER:=0A+=09=09case=20= B_DATACHECKSUMSWORKER_WORKER:=0A=20=09=09case=20B_AUTOVAC_LAUNCHER:=0A=20= =09=09case=20B_AUTOVAC_WORKER:=0A=20=09=09case=20B_BACKEND:=0Adiff=20= --git=20a/src/backend/utils/activity/wait_event_names.txt=20= b/src/backend/utils/activity/wait_event_names.txt=0Aindex=20= 0be307d2ca0..bf709b59543=20100644=0A---=20= a/src/backend/utils/activity/wait_event_names.txt=0A+++=20= b/src/backend/utils/activity/wait_event_names.txt=0A@@=20-116,6=20+116,9=20= @@=20CHECKPOINT_DELAY_COMPLETE=09"Waiting=20for=20a=20backend=20that=20= blocks=20a=20checkpoint=20from=20c=0A=20CHECKPOINT_DELAY_START=09= "Waiting=20for=20a=20backend=20that=20blocks=20a=20checkpoint=20from=20= starting."=0A=20CHECKPOINT_DONE=09"Waiting=20for=20a=20checkpoint=20to=20= complete."=0A=20CHECKPOINT_START=09"Waiting=20for=20a=20checkpoint=20to=20= start."=0A+CHECKSUM_ENABLE_STARTCONDITION=09"Waiting=20for=20data=20= checksums=20enabling=20to=20start."=0A+CHECKSUM_ENABLE_FINISHCONDITION=09= "Waiting=20for=20data=20checksums=20to=20be=20enabled."=0A= +CHECKSUM_ENABLE_TEMPTABLE_WAIT=09"Waiting=20for=20temporary=20tables=20= to=20be=20dropped=20for=20data=20checksums=20to=20be=20enabled."=0A=20= EXECUTE_GATHER=09"Waiting=20for=20activity=20from=20a=20child=20process=20= while=20executing=20a=20Gather=20plan=20node."=0A=20= HASH_BATCH_ALLOCATE=09"Waiting=20for=20an=20elected=20Parallel=20Hash=20= participant=20to=20allocate=20a=20hash=20table."=0A=20HASH_BATCH_ELECT=09= "Waiting=20to=20elect=20a=20Parallel=20Hash=20participant=20to=20= allocate=20a=20hash=20table."=0A@@=20-352,6=20+355,7=20@@=20DSMRegistry=09= "Waiting=20to=20read=20or=20update=20the=20dynamic=20shared=20memory=20= registry."=0A=20InjectionPoint=09"Waiting=20to=20read=20or=20update=20= information=20related=20to=20injection=20points."=0A=20SerialControl=09= "Waiting=20to=20read=20or=20update=20shared=20= pg_serial=20state."=0A=20AioWorkerSubmissionQueue=09= "Waiting=20to=20access=20AIO=20worker=20submission=20queue."=0A= +DataChecksumsWorker=09"Waiting=20for=20data=20checksumsworker."=0A=20=0A= =20#=0A=20#=20END=20OF=20PREDEFINED=20LWLOCKS=20(DO=20NOT=20CHANGE=20= THIS=20LINE)=0Adiff=20--git=20a/src/backend/utils/adt/pgstatfuncs.c=20= b/src/backend/utils/adt/pgstatfuncs.c=0Aindex=20c756c2bebaa..f4e264ebf33=20= 100644=0A---=20a/src/backend/utils/adt/pgstatfuncs.c=0A+++=20= b/src/backend/utils/adt/pgstatfuncs.c=0A@@=20-274,6=20+274,8=20@@=20= pg_stat_get_progress_info(PG_FUNCTION_ARGS)=0A=20=09=09cmdtype=20=3D=20= PROGRESS_COMMAND_BASEBACKUP;=0A=20=09else=20if=20(pg_strcasecmp(cmd,=20= "COPY")=20=3D=3D=200)=0A=20=09=09cmdtype=20=3D=20PROGRESS_COMMAND_COPY;=0A= +=09else=20if=20(pg_strcasecmp(cmd,=20"DATACHECKSUMS")=20=3D=3D=200)=0A+=09= =09cmdtype=20=3D=20PROGRESS_COMMAND_DATACHECKSUMS;=0A=20=09else=0A=20=09=09= ereport(ERROR,=0A=20=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A@@=20-1146,9=20+1148,6=20@@=20= pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)=0A=20=09int64=09=09= result;=0A=20=09PgStat_StatDBEntry=20*dbentry;=0A=20=0A-=09if=20= (!DataChecksumsEnabled())=0A-=09=09PG_RETURN_NULL();=0A-=0A=20=09if=20= ((dbentry=20=3D=20pgstat_fetch_stat_dbentry(dbid))=20=3D=3D=20NULL)=0A=20= =09=09result=20=3D=200;=0A=20=09else=0A@@=20-1164,9=20+1163,6=20@@=20= pg_stat_get_db_checksum_last_failure(PG_FUNCTION_ARGS)=0A=20=09= TimestampTz=20result;=0A=20=09PgStat_StatDBEntry=20*dbentry;=0A=20=0A-=09= if=20(!DataChecksumsEnabled())=0A-=09=09PG_RETURN_NULL();=0A-=0A=20=09if=20= ((dbentry=20=3D=20pgstat_fetch_stat_dbentry(dbid))=20=3D=3D=20NULL)=0A=20= =09=09result=20=3D=200;=0A=20=09else=0Adiff=20--git=20= a/src/backend/utils/init/miscinit.c=20= b/src/backend/utils/init/miscinit.c=0Aindex=20545d1e90fbd..34cce2ce0be=20= 100644=0A---=20a/src/backend/utils/init/miscinit.c=0A+++=20= b/src/backend/utils/init/miscinit.c=0A@@=20-293,9=20+293,18=20@@=20= GetBackendTypeDesc(BackendType=20backendType)=0A=20=09=09case=20= B_CHECKPOINTER:=0A=20=09=09=09backendDesc=20=3D=20= gettext_noop("checkpointer");=0A=20=09=09=09break;=0A+=0A=20=09=09case=20= B_IO_WORKER:=0A=20=09=09=09backendDesc=20=3D=20gettext_noop("io=20= worker");=0A=20=09=09=09break;=0A+=0A+=09=09case=20= B_DATACHECKSUMSWORKER_LAUNCHER:=0A+=09=09=09backendDesc=20=3D=20= "datachecksumsworker=20launcher";=0A+=09=09=09break;=0A+=09=09case=20= B_DATACHECKSUMSWORKER_WORKER:=0A+=09=09=09backendDesc=20=3D=20= "datachecksumsworker=20worker";=0A+=09=09=09break;=0A+=0A=20=09=09case=20= B_LOGGER:=0A=20=09=09=09backendDesc=20=3D=20gettext_noop("logger");=0A=20= =09=09=09break;=0A@@=20-895,7=20+904,8=20@@=20= InitializeSessionUserIdStandalone(void)=0A=20=09=20*=20workers,=20in=20= slot=20sync=20worker=20and=20in=20background=20workers.=0A=20=09=20*/=0A=20= =09Assert(!IsUnderPostmaster=20||=20AmAutoVacuumWorkerProcess()=20||=0A-=09= =09=20=20=20AmLogicalSlotSyncWorkerProcess()=20||=20= AmBackgroundWorkerProcess());=0A+=09=09=20=20=20= AmLogicalSlotSyncWorkerProcess()=20||=20AmBackgroundWorkerProcess()=20||=0A= +=09=09=20=20=20AmDataChecksumsWorkerProcess());=0A=20=0A=20=09/*=20call=20= only=20once=20*/=0A=20=09Assert(!OidIsValid(AuthenticatedUserId));=0A= diff=20--git=20a/src/backend/utils/init/postinit.c=20= b/src/backend/utils/init/postinit.c=0Aindex=20641e535a73c..589e7eab9e8=20= 100644=0A---=20a/src/backend/utils/init/postinit.c=0A+++=20= b/src/backend/utils/init/postinit.c=0A@@=20-750,6=20+750,24=20@@=20= InitPostgres(const=20char=20*in_dbname,=20Oid=20dboid,=0A=20=0A=20=09= ProcSignalInit(MyCancelKey,=20MyCancelKeyLength);=0A=20=0A+=09/*=0A+=09=20= *=20Initialize=20a=20local=20cache=20of=20the=20data_checksum_version,=20= to=20be=20updated=20by=0A+=09=20*=20the=20procsignal-based=20barriers.=0A= +=09=20*=0A+=09=20*=20This=20intentionally=20happens=20after=20= initializing=20the=20procsignal,=20otherwise=0A+=09=20*=20we=20might=20= miss=20a=20state=20change.=20This=20means=20we=20can=20get=20a=20barrier=20= for=20the=0A+=09=20*=20state=20we've=20just=20initialized=20-=20but=20it=20= can=20happen=20only=20once.=0A+=09=20*=0A+=09=20*=20The=20postmaster=20= (which=20is=20what=20gets=20forked=20into=20the=20new=20child=20process)=0A= +=09=20*=20does=20not=20handle=20barriers,=20therefore=20it=20may=20not=20= have=20the=20current=20value=0A+=09=20*=20of=20LocalDataChecksumVersion=20= value=20(it'll=20have=20the=20value=20read=20from=20the=0A+=09=20*=20= control=20file,=20which=20may=20be=20arbitrarily=20old).=0A+=09=20*=0A+=09= =20*=20NB:=20Even=20if=20the=20postmaster=20handled=20barriers,=20the=20= value=20might=20still=20be=0A+=09=20*=20stale,=20as=20it=20might=20have=20= changed=20after=20this=20process=20forked.=0A+=09=20*/=0A+=09= InitLocalDataChecksumVersion();=0A+=0A=20=09/*=0A=20=09=20*=20Also=20set=20= up=20timeout=20handlers=20needed=20for=20backend=20operation.=20=20We=20= need=0A=20=09=20*=20these=20in=20every=20case=20except=20bootstrap.=0A@@=20= -878,7=20+896,7=20@@=20InitPostgres(const=20char=20*in_dbname,=20Oid=20= dboid,=0A=20=09=09=09=09=09=20errhint("You=20should=20immediately=20run=20= CREATE=20USER=20\"%s\"=20SUPERUSER;.",=0A=20=09=09=09=09=09=09=09=20= username=20!=3D=20NULL=20?=20username=20:=20"postgres")));=0A=20=09}=0A-=09= else=20if=20(AmBackgroundWorkerProcess())=0A+=09else=20if=20= (AmBackgroundWorkerProcess()=20||=20AmDataChecksumsWorkerProcess())=0A=20= =09{=0A=20=09=09if=20(username=20=3D=3D=20NULL=20&&=20= !OidIsValid(useroid))=0A=20=09=09{=0Adiff=20--git=20= a/src/backend/utils/misc/guc_tables.c=20= b/src/backend/utils/misc/guc_tables.c=0Aindex=20d14b1678e7f..de4d21de504=20= 100644=0A---=20a/src/backend/utils/misc/guc_tables.c=0A+++=20= b/src/backend/utils/misc/guc_tables.c=0A@@=20-491,6=20+491,14=20@@=20= static=20const=20struct=20config_enum_entry=20file_copy_method_options[]=20= =3D=20{=0A=20=09{NULL,=200,=20false}=0A=20};=0A=20=0A+static=20const=20= struct=20config_enum_entry=20data_checksums_options[]=20=3D=20{=0A+=09= {"on",=20PG_DATA_CHECKSUM_VERSION,=20true},=0A+=09{"off",=20= PG_DATA_CHECKSUM_OFF,=20true},=0A+=09{"inprogress-on",=20= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION,=20true},=0A+=09= {"inprogress-off",=20PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION,=20true},=0A= +=09{NULL,=200,=20false}=0A+};=0A+=0A=20/*=0A=20=20*=20Options=20for=20= enum=20values=20stored=20in=20other=20modules=0A=20=20*/=0A@@=20-616,7=20= +624,6=20@@=20static=20int=09shared_memory_size_mb;=0A=20static=20int=09= shared_memory_size_in_huge_pages;=0A=20static=20int=09wal_block_size;=0A=20= static=20int=09num_os_semaphores;=0A-static=20bool=20data_checksums;=0A=20= static=20bool=20integer_datetimes;=0A=20=0A=20#ifdef=20= USE_ASSERT_CHECKING=0A@@=20-1968,17=20+1975,6=20@@=20struct=20= config_bool=20ConfigureNamesBool[]=20=3D=0A=20=09=09NULL,=20NULL,=20NULL=0A= =20=09},=0A=20=0A-=09{=0A-=09=09{"data_checksums",=20PGC_INTERNAL,=20= PRESET_OPTIONS,=0A-=09=09=09gettext_noop("Shows=20whether=20data=20= checksums=20are=20turned=20on=20for=20this=20cluster."),=0A-=09=09=09= NULL,=0A-=09=09=09GUC_NOT_IN_SAMPLE=20|=20GUC_DISALLOW_IN_FILE=20|=20= GUC_RUNTIME_COMPUTED=0A-=09=09},=0A-=09=09&data_checksums,=0A-=09=09= false,=0A-=09=09NULL,=20NULL,=20NULL=0A-=09},=0A-=0A=20=09{=0A=20=09=09= {"syslog_sequence_numbers",=20PGC_SIGHUP,=20LOGGING_WHERE,=0A=20=09=09=09= gettext_noop("Add=20sequence=20number=20to=20syslog=20messages=20to=20= avoid=20duplicate=20suppression."),=0A@@=20-5417,6=20+5413,16=20@@=20= struct=20config_enum=20ConfigureNamesEnum[]=20=3D=0A=20=09=09= DEFAULT_IO_METHOD,=20io_method_options,=0A=20=09=09NULL,=20= assign_io_method,=20NULL=0A=20=09},=0A+=09{=0A+=09=09{"data_checksums",=20= PGC_INTERNAL,=20PRESET_OPTIONS,=0A+=09=09=09gettext_noop("Shows=20= whether=20data=20checksums=20are=20turned=20on=20for=20this=20= cluster."),=0A+=09=09=09NULL,=0A+=09=09=09GUC_NOT_IN_SAMPLE=20|=20= GUC_DISALLOW_IN_FILE=20|=20GUC_RUNTIME_COMPUTED=0A+=09=09},=0A+=09=09= &data_checksums,=0A+=09=09PG_DATA_CHECKSUM_OFF,=20= data_checksums_options,=0A+=09=09NULL,=20NULL,=20show_data_checksums=0A+=09= },=0A=20=0A=20=09/*=20End-of-list=20marker=20*/=0A=20=09{=0Adiff=20--git=20= a/src/bin/pg_checksums/pg_checksums.c=20= b/src/bin/pg_checksums/pg_checksums.c=0Aindex=20f20be82862a..4e9e85fdc54=20= 100644=0A---=20a/src/bin/pg_checksums/pg_checksums.c=0A+++=20= b/src/bin/pg_checksums/pg_checksums.c=0A@@=20-576,7=20+576,7=20@@=20= main(int=20argc,=20char=20*argv[])=0A=20=09=09mode=20=3D=3D=20= PG_MODE_DISABLE)=0A=20=09=09pg_fatal("data=20checksums=20are=20already=20= disabled=20in=20cluster");=0A=20=0A-=09if=20= (ControlFile->data_checksum_version=20>=200=20&&=0A+=09if=20= (ControlFile->data_checksum_version=20=3D=3D=20PG_DATA_CHECKSUM_VERSION=20= &&=0A=20=09=09mode=20=3D=3D=20PG_MODE_ENABLE)=0A=20=09=09pg_fatal("data=20= checksums=20are=20already=20enabled=20in=20cluster");=0A=20=0Adiff=20= --git=20a/src/bin/pg_controldata/pg_controldata.c=20= b/src/bin/pg_controldata/pg_controldata.c=0Aindex=20= 10de058ce91..acf5c7b026e=20100644=0A---=20= a/src/bin/pg_controldata/pg_controldata.c=0A+++=20= b/src/bin/pg_controldata/pg_controldata.c=0A@@=20-280,6=20+280,8=20@@=20= main(int=20argc,=20char=20*argv[])=0A=20=09=09=20=20=20= ControlFile->checkPointCopy.oldestCommitTsXid);=0A=20=09printf(_("Latest=20= checkpoint's=20newestCommitTsXid:%u\n"),=0A=20=09=09=20=20=20= ControlFile->checkPointCopy.newestCommitTsXid);=0A+=09printf(_("Latest=20= checkpoint's=20data_checksum_version:%u\n"),=0A+=09=09=20=20=20= ControlFile->checkPointCopy.data_checksum_version);=0A=20=09= printf(_("Time=20of=20latest=20checkpoint:=20=20=20=20=20=20=20=20=20=20=20= =20%s\n"),=0A=20=09=09=20=20=20ckpttime_str);=0A=20=09printf(_("Fake=20= LSN=20counter=20for=20unlogged=20rels:=20=20=20%X/%08X\n"),=0Adiff=20= --git=20a/src/bin/pg_upgrade/controldata.c=20= b/src/bin/pg_upgrade/controldata.c=0Aindex=2090cef0864de..29684e82440=20= 100644=0A---=20a/src/bin/pg_upgrade/controldata.c=0A+++=20= b/src/bin/pg_upgrade/controldata.c=0A@@=20-15,6=20+15,7=20@@=0A=20= #include=20"access/xlog_internal.h"=0A=20#include=20"common/string.h"=0A=20= #include=20"pg_upgrade.h"=0A+#include=20"storage/bufpage.h"=0A=20=0A=20=0A= =20/*=0A@@=20-736,6=20+737,14=20@@=20check_control_data(ControlData=20= *oldctrl,=0A=20=09=20*=20check_for_isn_and_int8_passing_mismatch().=0A=20= =09=20*/=0A=20=0A+=09/*=0A+=09=20*=20If=20data=20checksums=20are=20in=20= any=20in-progress=20state=20then=20disallow=20the=0A+=09=20*=20upgrade.=20= The=20user=20should=20either=20let=20the=20process=20finish,=20or=20turn=20= off=0A+=09=20*=20data=20checksums,=20before=20retrying.=0A+=09=20*/=0A+=09= if=20(oldctrl->data_checksum_version=20>=20PG_DATA_CHECKSUM_VERSION)=0A+=09= =09pg_fatal("checksums=20are=20being=20enabled=20in=20the=20old=20= cluster");=0A+=0A=20=09/*=0A=20=09=20*=20We=20might=20eventually=20allow=20= upgrades=20from=20checksum=20to=20no-checksum=0A=20=09=20*=20clusters.=0A= diff=20--git=20a/src/include/access/xlog.h=20b/src/include/access/xlog.h=0A= index=20d12798be3d8..8bcc5aa8a63=20100644=0A---=20= a/src/include/access/xlog.h=0A+++=20b/src/include/access/xlog.h=0A@@=20= -56,6=20+56,7=20@@=20extern=20PGDLLIMPORT=20int=20CommitDelay;=0A=20= extern=20PGDLLIMPORT=20int=20CommitSiblings;=0A=20extern=20PGDLLIMPORT=20= bool=20track_wal_io_timing;=0A=20extern=20PGDLLIMPORT=20int=20= wal_decode_buffer_size;=0A+extern=20PGDLLIMPORT=20int=20data_checksums;=0A= =20=0A=20extern=20PGDLLIMPORT=20int=20CheckPointSegments;=0A=20=0A@@=20= -117,7=20+118,7=20@@=20extern=20PGDLLIMPORT=20int=20wal_level;=0A=20=20*=20= of=20the=20bits=20make=20it=20to=20disk,=20but=20the=20checksum=20= wouldn't=20match.=20=20Also=20WAL-log=0A=20=20*=20them=20if=20forced=20= by=20wal_log_hints=3Don.=0A=20=20*/=0A-#define=20XLogHintBitIsNeeded()=20= (DataChecksumsEnabled()=20||=20wal_log_hints)=0A+#define=20= XLogHintBitIsNeeded()=20(wal_log_hints=20||=20DataChecksumsNeedWrite())=0A= =20=0A=20/*=20Do=20we=20need=20to=20WAL-log=20information=20required=20= only=20for=20Hot=20Standby=20and=20logical=20replication?=20*/=0A=20= #define=20XLogStandbyInfoActive()=20(wal_level=20>=3D=20= WAL_LEVEL_REPLICA)=0A@@=20-229,7=20+230,19=20@@=20extern=20XLogRecPtr=20= GetXLogWriteRecPtr(void);=0A=20=0A=20extern=20uint64=20= GetSystemIdentifier(void);=0A=20extern=20char=20= *GetMockAuthenticationNonce(void);=0A-extern=20bool=20= DataChecksumsEnabled(void);=0A+extern=20bool=20= DataChecksumsNeedWrite(void);=0A+extern=20bool=20= DataChecksumsNeedVerify(void);=0A+extern=20bool=20= DataChecksumsOnInProgress(void);=0A+extern=20bool=20= DataChecksumsOffInProgress(void);=0A+extern=20void=20= SetDataChecksumsOnInProgress(void);=0A+extern=20void=20= SetDataChecksumsOn(void);=0A+extern=20void=20SetDataChecksumsOff(void);=0A= +extern=20bool=20AbsorbChecksumsOnInProgressBarrier(void);=0A+extern=20= bool=20AbsorbChecksumsOffInProgressBarrier(void);=0A+extern=20bool=20= AbsorbChecksumsOnBarrier(void);=0A+extern=20bool=20= AbsorbChecksumsOffBarrier(void);=0A+extern=20const=20char=20= *show_data_checksums(void);=0A+extern=20void=20= InitLocalDataChecksumVersion(void);=0A=20extern=20bool=20= GetDefaultCharSignedness(void);=0A=20extern=20XLogRecPtr=20= GetFakeLSNForUnloggedRel(void);=0A=20extern=20Size=20= XLOGShmemSize(void);=0Adiff=20--git=20= a/src/include/access/xlog_internal.h=20= b/src/include/access/xlog_internal.h=0Aindex=20cc06fc29ab2..cc78b00fe4c=20= 100644=0A---=20a/src/include/access/xlog_internal.h=0A+++=20= b/src/include/access/xlog_internal.h=0A@@=20-25,6=20+25,7=20@@=0A=20= #include=20"lib/stringinfo.h"=0A=20#include=20"pgtime.h"=0A=20#include=20= "storage/block.h"=0A+#include=20"storage/checksum.h"=0A=20#include=20= "storage/relfilelocator.h"=0A=20=0A=20=0A@@=20-289,6=20+290,12=20@@=20= typedef=20struct=20xl_restore_point=0A=20=09char=09=09= rp_name[MAXFNAMELEN];=0A=20}=20xl_restore_point;=0A=20=0A+/*=20= Information=20logged=20when=20data=20checksum=20level=20is=20changed=20= */=0A+typedef=20struct=20xl_checksum_state=0A+{=0A+=09uint32=09=09= new_checksumtype;=0A+}=20xl_checksum_state;=0A+=0A=20/*=20Overwrite=20of=20= prior=20contrecord=20*/=0A=20typedef=20struct=20xl_overwrite_contrecord=0A= =20{=0Adiff=20--git=20a/src/include/catalog/pg_control.h=20= b/src/include/catalog/pg_control.h=0Aindex=2063e834a6ce4..a8877fb87d1=20= 100644=0A---=20a/src/include/catalog/pg_control.h=0A+++=20= b/src/include/catalog/pg_control.h=0A@@=20-62,6=20+62,9=20@@=20typedef=20= struct=20CheckPoint=0A=20=09=20*=20set=20to=20InvalidTransactionId.=0A=20= =09=20*/=0A=20=09TransactionId=20oldestActiveXid;=0A+=0A+=09/*=20data=20= checksums=20at=20the=20time=20of=20the=20checkpoint=20=20*/=0A+=09uint32=09= =09data_checksum_version;=0A=20}=20CheckPoint;=0A=20=0A=20/*=20XLOG=20= info=20values=20for=20XLOG=20rmgr=20*/=0A@@=20-80,6=20+83,7=20@@=20= typedef=20struct=20CheckPoint=0A=20/*=200xC0=20is=20used=20in=20Postgres=20= 9.5-11=20*/=0A=20#define=20XLOG_OVERWRITE_CONTRECORD=09=090xD0=0A=20= #define=20XLOG_CHECKPOINT_REDO=09=09=090xE0=0A+#define=20XLOG_CHECKSUMS=09= =09=09=09=090xF0=0A=20=0A=20=0A=20/*=0A@@=20-219,7=20+223,7=20@@=20= typedef=20struct=20ControlFileData=0A=20=09bool=09=09float8ByVal;=09/*=20= float8,=20int8,=20etc=20pass-by-value?=20*/=0A=20=0A=20=09/*=20Are=20= data=20pages=20protected=20by=20checksums?=20Zero=20if=20no=20checksum=20= version=20*/=0A-=09uint32=09=09data_checksum_version;=0A+=09uint32=09=09= data_checksum_version;=09/*=20persistent=20*/=0A=20=0A=20=09/*=0A=20=09=20= *=20True=20if=20the=20default=20signedness=20of=20char=20is=20"signed"=20= on=20a=20platform=20where=0Adiff=20--git=20= a/src/include/catalog/pg_proc.dat=20b/src/include/catalog/pg_proc.dat=0A= index=20118d6da1ace..c6f4e31a12f=20100644=0A---=20= a/src/include/catalog/pg_proc.dat=0A+++=20= b/src/include/catalog/pg_proc.dat=0A@@=20-12356,6=20+12356,25=20@@=0A=20=20= =20proname=20=3D>=20'jsonb_subscript_handler',=20prorettype=20=3D>=20= 'internal',=0A=20=20=20proargtypes=20=3D>=20'internal',=20prosrc=20=3D>=20= 'jsonb_subscript_handler'=20},=0A=20=0A+#=20data=20checksum=20management=20= functions=0A+{=20oid=20=3D>=20'9258',=0A+=20=20descr=20=3D>=20'disable=20= data=20checksums',=0A+=20=20proname=20=3D>=20= 'pg_disable_data_checksums',=20provolatile=20=3D>=20'v',=20prorettype=20= =3D>=20'void',=0A+=20=20proparallel=20=3D>=20'r',=0A+=20=20proargtypes=20= =3D>=20'bool',=20proallargtypes=20=3D>=20'{bool}',=0A+=20=20proargmodes=20= =3D>=20'{i}',=0A+=20=20proargnames=20=3D>=20'{fast}',=0A+=20=20prosrc=20= =3D>=20'disable_data_checksums'=20},=0A+=0A+{=20oid=20=3D>=20'9257',=0A+=20= =20descr=20=3D>=20'enable=20data=20checksums',=0A+=20=20proname=20=3D>=20= 'pg_enable_data_checksums',=20provolatile=20=3D>=20'v',=20prorettype=20= =3D>=20'void',=0A+=20=20proparallel=20=3D>=20'r',=0A+=20=20proargtypes=20= =3D>=20'int4=20int4=20bool',=20proallargtypes=20=3D>=20= '{int4,int4,bool}',=0A+=20=20proargmodes=20=3D>=20'{i,i,i}',=0A+=20=20= proargnames=20=3D>=20'{cost_delay,cost_limit,fast}',=0A+=20=20prosrc=20= =3D>=20'enable_data_checksums'=20},=0A+=0A=20#=20collation=20management=20= functions=0A=20{=20oid=20=3D>=20'3445',=20descr=20=3D>=20'import=20= collations=20from=20operating=20system',=0A=20=20=20proname=20=3D>=20= 'pg_import_system_collations',=20procost=20=3D>=20'100',=0Adiff=20--git=20= a/src/include/commands/progress.h=20b/src/include/commands/progress.h=0A= index=201cde4bd9bcf..cf6de4ef12d=20100644=0A---=20= a/src/include/commands/progress.h=0A+++=20= b/src/include/commands/progress.h=0A@@=20-162,4=20+162,20=20@@=0A=20= #define=20PROGRESS_COPY_TYPE_PIPE=203=0A=20#define=20= PROGRESS_COPY_TYPE_CALLBACK=204=0A=20=0A+/*=20Progress=20parameters=20= for=20PROGRESS_DATACHECKSUMS=20*/=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE=09=090=0A+#define=20= PROGRESS_DATACHECKSUMS_DBS_TOTAL=091=0A+#define=20= PROGRESS_DATACHECKSUMS_DBS_DONE=09=092=0A+#define=20= PROGRESS_DATACHECKSUMS_RELS_TOTAL=093=0A+#define=20= PROGRESS_DATACHECKSUMS_RELS_DONE=094=0A+#define=20= PROGRESS_DATACHECKSUMS_BLOCKS_TOTAL=095=0A+#define=20= PROGRESS_DATACHECKSUMS_BLOCKS_DONE=096=0A+=0A+/*=20Phases=20of=20= datachecksumsworker=20operation=20*/=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_ENABLING=09=09=090=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_DISABLING=09=09=091=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_WAITING=09=09=092=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_WAITING_TEMPREL=093=0A+#define=20= PROGRESS_DATACHECKSUMS_PHASE_WAITING_CHECKPOINT=204=0A+=0A=20#endif=0A= diff=20--git=20a/src/include/miscadmin.h=20b/src/include/miscadmin.h=0A= index=201bef98471c3..2a0d7b6de42=20100644=0A---=20= a/src/include/miscadmin.h=0A+++=20b/src/include/miscadmin.h=0A@@=20= -366,6=20+366,9=20@@=20typedef=20enum=20BackendType=0A=20=09= B_WAL_SUMMARIZER,=0A=20=09B_WAL_WRITER,=0A=20=0A+=09= B_DATACHECKSUMSWORKER_LAUNCHER,=0A+=09B_DATACHECKSUMSWORKER_WORKER,=0A+=0A= =20=09/*=0A=20=09=20*=20Logger=20is=20not=20connected=20to=20shared=20= memory=20and=20does=20not=20have=20a=20PGPROC=0A=20=09=20*=20entry.=0A@@=20= -391,6=20+394,9=20@@=20extern=20PGDLLIMPORT=20BackendType=20= MyBackendType;=0A=20#define=20AmWalSummarizerProcess()=09(MyBackendType=20= =3D=3D=20B_WAL_SUMMARIZER)=0A=20#define=20AmWalWriterProcess()=09=09= (MyBackendType=20=3D=3D=20B_WAL_WRITER)=0A=20#define=20= AmIoWorkerProcess()=09=09=09(MyBackendType=20=3D=3D=20B_IO_WORKER)=0A= +#define=20AmDataChecksumsWorkerProcess()=20\=0A+=09(MyBackendType=20=3D=3D= =20B_DATACHECKSUMSWORKER_LAUNCHER=20||=20\=0A+=09=20MyBackendType=20=3D=3D= =20B_DATACHECKSUMSWORKER_WORKER)=0A=20=0A=20#define=20= AmSpecialWorkerProcess()=20\=0A=20=09(AmAutoVacuumLauncherProcess()=20||=20= \=0Adiff=20--git=20a/src/include/postmaster/datachecksumsworker.h=20= b/src/include/postmaster/datachecksumsworker.h=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..2cd066fd0fe=0A---=20/dev/null=0A+++=20= b/src/include/postmaster/datachecksumsworker.h=0A@@=20-0,0=20+1,51=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20datachecksumsworker.h=0A+=20*=09=20=20header=20file=20= for=20data=20checksum=20helper=20background=20worker=0A+=20*=0A+=20*=0A+=20= *=20Portions=20Copyright=20(c)=201996-2025,=20PostgreSQL=20Global=20= Development=20Group=0A+=20*=20Portions=20Copyright=20(c)=201994,=20= Regents=20of=20the=20University=20of=20California=0A+=20*=0A+=20*=20= src/include/postmaster/datachecksumsworker.h=0A+=20*=0A+=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#ifndef=20DATACHECKSUMSWORKER_H=0A+#define=20= DATACHECKSUMSWORKER_H=0A+=0A+/*=20Shared=20memory=20*/=0A+extern=20Size=20= DataChecksumsWorkerShmemSize(void);=0A+extern=20void=20= DataChecksumsWorkerShmemInit(void);=0A+=0A+/*=20Possible=20operations=20= the=20Datachecksumsworker=20can=20perform=20*/=0A+typedef=20enum=20= DataChecksumsWorkerOperation=0A+{=0A+=09ENABLE_DATACHECKSUMS,=0A+=09= DISABLE_DATACHECKSUMS,=0A+=09/*=20TODO:=20VERIFY_DATACHECKSUMS,=20*/=0A= +}=09=09=09DataChecksumsWorkerOperation;=0A+=0A+/*=0A+=20*=20Possible=20= states=20for=20a=20database=20entry=20which=20has=20been=20processed.=20= Exported=0A+=20*=20here=20since=20we=20want=20to=20be=20able=20to=20= reference=20this=20from=20injection=20point=20tests.=0A+=20*/=0A+typedef=20= enum=0A+{=0A+=09DATACHECKSUMSWORKER_SUCCESSFUL=20=3D=200,=0A+=09= DATACHECKSUMSWORKER_ABORTED,=0A+=09DATACHECKSUMSWORKER_FAILED,=0A+=09= DATACHECKSUMSWORKER_RETRYDB,=0A+}=20DataChecksumsWorkerResult;=0A+=0A+/*=20= Start=20the=20background=20processes=20for=20enabling=20or=20disabling=20= checksums=20*/=0A+void=09=09= StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation=20op,=0A+=09= =09=09=09=09=09=09=09=09=09=09=20int=20cost_delay,=0A+=09=09=09=09=09=09=09= =09=09=09=09=20int=20cost_limit,=0A+=09=09=09=09=09=09=09=09=09=09=09=20= bool=20fast);=0A+=0A+/*=20Background=20worker=20entrypoints=20*/=0A+void=09= =09DataChecksumsWorkerLauncherMain(Datum=20arg);=0A+void=09=09= DataChecksumsWorkerMain(Datum=20arg);=0A+=0A+#endif=09=09=09=09=09=09=09= /*=20DATACHECKSUMSWORKER_H=20*/=0Adiff=20--git=20= a/src/include/storage/bufpage.h=20b/src/include/storage/bufpage.h=0A= index=20aeb67c498c5..30fb0f62d4c=20100644=0A---=20= a/src/include/storage/bufpage.h=0A+++=20b/src/include/storage/bufpage.h=0A= @@=20-16,6=20+16,7=20@@=0A=20=0A=20#include=20"access/xlogdefs.h"=0A=20= #include=20"storage/block.h"=0A+#include=20"storage/checksum.h"=0A=20= #include=20"storage/item.h"=0A=20#include=20"storage/off.h"=0A=20=0A@@=20= -205,7=20+206,6=20@@=20typedef=20PageHeaderData=20*PageHeader;=0A=20=20*=20= handling=20pages.=0A=20=20*/=0A=20#define=20PG_PAGE_LAYOUT_VERSION=09=09= 4=0A-#define=20PG_DATA_CHECKSUM_VERSION=091=0A=20=0A=20/*=20= ----------------------------------------------------------------=0A=20=20= *=09=09=09=09=09=09page=20support=20functions=0Adiff=20--git=20= a/src/include/storage/checksum.h=20b/src/include/storage/checksum.h=0A= index=2025d13a798d1..b3f368a15b5=20100644=0A---=20= a/src/include/storage/checksum.h=0A+++=20= b/src/include/storage/checksum.h=0A@@=20-15,6=20+15,20=20@@=0A=20=0A=20= #include=20"storage/block.h"=0A=20=0A+/*=0A+=20*=20Checksum=20version=20= 0=20is=20used=20for=20when=20data=20checksums=20are=20disabled=20(OFF).=0A= +=20*=20PG_DATA_CHECKSUM_VERSION=20defines=20that=20data=20checksums=20= are=20enabled=20in=20the=0A+=20*=20cluster=20and=20= PG_DATA_CHECKSUM_INPROGRESS_{ON|OFF}_VERSION=20defines=20that=20data=0A+=20= *=20checksums=20are=20either=20currently=20being=20enabled=20or=20= disabled.=0A+=20*/=0A+typedef=20enum=20ChecksumType=0A+{=0A+=09= PG_DATA_CHECKSUM_OFF=20=3D=200,=0A+=09PG_DATA_CHECKSUM_VERSION,=0A+=09= PG_DATA_CHECKSUM_INPROGRESS_ON_VERSION,=0A+=09= PG_DATA_CHECKSUM_INPROGRESS_OFF_VERSION=0A+}=20ChecksumType;=0A+=0A=20/*=0A= =20=20*=20Compute=20the=20checksum=20for=20a=20Postgres=20page.=20=20The=20= page=20must=20be=20aligned=20on=20a=0A=20=20*=204-byte=20boundary.=0A= diff=20--git=20a/src/include/storage/lwlocklist.h=20= b/src/include/storage/lwlocklist.h=0Aindex=20208d2e3a8ed..4a90b7fa372=20= 100644=0A---=20a/src/include/storage/lwlocklist.h=0A+++=20= b/src/include/storage/lwlocklist.h=0A@@=20-85,6=20+85,7=20@@=20= PG_LWLOCK(50,=20DSMRegistry)=0A=20PG_LWLOCK(51,=20InjectionPoint)=0A=20= PG_LWLOCK(52,=20SerialControl)=0A=20PG_LWLOCK(53,=20= AioWorkerSubmissionQueue)=0A+PG_LWLOCK(54,=20DataChecksumsWorker)=0A=20=0A= =20/*=0A=20=20*=20There=20also=20exist=20several=20built-in=20LWLock=20= tranches.=20=20As=20with=20the=20predefined=0Adiff=20--git=20= a/src/include/storage/proc.h=20b/src/include/storage/proc.h=0Aindex=20= c6f5ebceefd..d90d35b1d6f=20100644=0A---=20a/src/include/storage/proc.h=0A= +++=20b/src/include/storage/proc.h=0A@@=20-463,11=20+463,11=20@@=20= extern=20PGDLLIMPORT=20PGPROC=20*PreparedXactProcs;=0A=20=20*=20= Background=20writer,=20checkpointer,=20WAL=20writer,=20WAL=20summarizer,=20= and=20archiver=0A=20=20*=20run=20during=20normal=20operation.=20=20= Startup=20process=20and=20WAL=20receiver=20also=20consume=0A=20=20*=202=20= slots,=20but=20WAL=20writer=20is=20launched=20only=20after=20startup=20= has=20exited,=20so=20we=0A-=20*=20only=20need=206=20slots.=0A+=20*=20= only=20need=206=20slots=20to=20cover=20these.=20The=20DataChecksums=20= worker=20and=20launcher=0A+=20*=20can=20consume=202=20slots=20when=20= data=20checksums=20are=20enabled=20or=20disabled.=0A=20=20*/=0A=20= #define=20MAX_IO_WORKERS=20=20=20=20=20=20=20=20=20=2032=0A-#define=20= NUM_AUXILIARY_PROCS=09=09(6=20+=20MAX_IO_WORKERS)=0A-=0A+#define=20= NUM_AUXILIARY_PROCS=09=09(8=20+=20MAX_IO_WORKERS)=0A=20=0A=20/*=20= configurable=20options=20*/=0A=20extern=20PGDLLIMPORT=20int=20= DeadlockTimeout;=0Adiff=20--git=20a/src/include/storage/procsignal.h=20= b/src/include/storage/procsignal.h=0Aindex=20afeeb1ca019..c54c61e2cd8=20= 100644=0A---=20a/src/include/storage/procsignal.h=0A+++=20= b/src/include/storage/procsignal.h=0A@@=20-54,6=20+54,11=20@@=20typedef=20= enum=0A=20typedef=20enum=0A=20{=0A=20=09PROCSIGNAL_BARRIER_SMGRRELEASE,=20= /*=20ask=20smgr=20to=20close=20files=20*/=0A+=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_OFF,=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_ON,=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_INPROGRESS_OFF,=0A+=09= PROCSIGNAL_BARRIER_CHECKSUM_ON,=0A=20}=20ProcSignalBarrierType;=0A=20=0A=20= /*=0Adiff=20--git=20a/src/include/utils/backend_progress.h=20= b/src/include/utils/backend_progress.h=0Aindex=20= dda813ab407..c664e92dbfe=20100644=0A---=20= a/src/include/utils/backend_progress.h=0A+++=20= b/src/include/utils/backend_progress.h=0A@@=20-28,6=20+28,7=20@@=20= typedef=20enum=20ProgressCommandType=0A=20=09= PROGRESS_COMMAND_CREATE_INDEX,=0A=20=09PROGRESS_COMMAND_BASEBACKUP,=0A=20= =09PROGRESS_COMMAND_COPY,=0A+=09PROGRESS_COMMAND_DATACHECKSUMS,=0A=20}=20= ProgressCommandType;=0A=20=0A=20#define=20PGSTAT_NUM_PROGRESS_PARAM=0920=0A= diff=20--git=20a/src/test/Makefile=20b/src/test/Makefile=0Aindex=20= 511a72e6238..278ce3e8a86=20100644=0A---=20a/src/test/Makefile=0A+++=20= b/src/test/Makefile=0A@@=20-12,7=20+12,16=20@@=20subdir=20=3D=20src/test=0A= =20top_builddir=20=3D=20../..=0A=20include=20= $(top_builddir)/src/Makefile.global=0A=20=0A-SUBDIRS=20=3D=20perl=20= postmaster=20regress=20isolation=20modules=20authentication=20recovery=20= subscription=0A+SUBDIRS=20=3D=20\=0A+=09=09perl=20\=0A+=09=09postmaster=20= \=0A+=09=09regress=20\=0A+=09=09isolation=20\=0A+=09=09modules=20\=0A+=09= =09authentication=20\=0A+=09=09recovery=20\=0A+=09=09subscription=20\=0A= +=09=09checksum=0A=20=0A=20ifeq=20($(with_icu),yes)=0A=20SUBDIRS=20+=3D=20= icu=0Adiff=20--git=20a/src/test/modules/meson.build=20= b/src/test/modules/meson.build=0Aindex=2093be0f57289..6b4450eb473=20= 100644=0A---=20a/src/test/modules/meson.build=0A+++=20= b/src/test/modules/meson.build=0A@@=20-16,6=20+16,7=20@@=20= subdir('ssl_passphrase_callback')=0A=20subdir('test_aio')=0A=20= subdir('test_binaryheap')=0A=20subdir('test_bloomfilter')=0A= +subdir('test_checksums')=0A=20subdir('test_copy_callbacks')=0A=20= subdir('test_custom_rmgrs')=0A=20subdir('test_ddl_deparse')=0Adiff=20= --git=20a/src/test/modules/test_checksums/.gitignore=20= b/src/test/modules/test_checksums/.gitignore=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..871e943d50e=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/.gitignore=0A@@=20-0,0=20+1,2=20@@=0A= +#=20Generated=20by=20test=20suite=0A+/tmp_check/=0Adiff=20--git=20= a/src/test/modules/test_checksums/Makefile=20= b/src/test/modules/test_checksums/Makefile=0Anew=20file=20mode=20100644=0A= index=2000000000000..b9136bb513f=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/Makefile=0A@@=20-0,0=20+1,36=20@@=0A= +#------------------------------------------------------------------------= -=0A+#=0A+#=20Makefile=20for=20src/test/checksum=0A+#=0A+#=20Portions=20= Copyright=20(c)=201996-2025,=20PostgreSQL=20Global=20Development=20Group=0A= +#=20Portions=20Copyright=20(c)=201994,=20Regents=20of=20the=20= University=20of=20California=0A+#=0A+#=20src/test/checksum/Makefile=0A+#=0A= +#------------------------------------------------------------------------= -=0A+=0A+subdir=20=3D=20src/test/checksum=0A+top_builddir=20=3D=20= ../../..=0A+include=20$(top_builddir)/src/Makefile.global=0A+=0A= +EXTRA_INSTALL=20=3D=20src/test/modules/injection_points=0A+=0A+export=20= enable_injection_points=0A+=0A+MODULE_big=20=3D=20test_checksums=0A+OBJS=20= =3D=20\=0A+=09$(WIN32RES)=0A+=09test_checksums.o=0A+PGFILEDESC=20=3D=20= "test_checksums=20-=20test=20code=20for=20data=20checksums"=0A+=0A= +EXTENSION=20=3D=20test_checksums=0A+DATA=20=3D=20= test_checksums--1.0.sql=0A+=0A+check:=0A+=09$(prove_check)=0A+=0A= +installcheck:=0A+=09$(prove_installcheck)=0A+=0A+clean=20distclean=20= maintainer-clean:=0A+=09rm=20-rf=20tmp_check=0Adiff=20--git=20= a/src/test/modules/test_checksums/README=20= b/src/test/modules/test_checksums/README=0Anew=20file=20mode=20100644=0A= index=2000000000000..0f0317060b3=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/README=0A@@=20-0,0=20+1,22=20@@=0A= +src/test/checksum/README=0A+=0A+Regression=20tests=20for=20data=20= checksums=0A+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A+=0A+This=20directory=20= contains=20a=20test=20suite=20for=20enabling=20data=20checksums=0A+in=20= a=20running=20cluster.=0A+=0A+Running=20the=20tests=0A+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A+=0A+=20=20=20=20make=20check=0A+=0A+or=0A= +=0A+=20=20=20=20make=20installcheck=0A+=0A+NOTE:=20This=20creates=20a=20= temporary=20installation=20(in=20the=20case=20of=20"check"),=0A+with=20= multiple=20nodes,=20be=20they=20master=20or=20standby(s)=20for=20the=20= purpose=20of=0A+the=20tests.=0A+=0A+NOTE:=20This=20requires=20the=20= --enable-tap-tests=20argument=20to=20configure.=0Adiff=20--git=20= a/src/test/modules/test_checksums/meson.build=20= b/src/test/modules/test_checksums/meson.build=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..4e57c5de09b=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/meson.build=0A@@=20-0,0=20+1,34=20@@=0A= +#=20Copyright=20(c)=202025,=20PostgreSQL=20Global=20Development=20Group=0A= +=0A+test_checksums_sources=20=3D=20files(=0A+=09'test_checksums.c',=0A= +)=0A+=0A+test_checksums=20=3D=20shared_module('test_checksums',=0A+=09= test_checksums_sources,=0A+=09kwargs:=20pg_test_mod_args,=0A+)=0A= +test_install_libs=20+=3D=20test_checksums=0A+=0A+test_install_data=20+=3D= =20files(=0A+=09'test_checksums.control',=0A+=09= 'test_checksums--1.0.sql',=0A+)=0A+=0A+tests=20+=3D=20{=0A+=20=20'name':=20= 'test_checksums',=0A+=20=20'sd':=20meson.current_source_dir(),=0A+=20=20= 'bd':=20meson.current_build_dir(),=0A+=20=20'tap':=20{=0A+=20=20=20=20= 'env':=20{=0A+=20=20=20=20=20=20=20'enable_injection_points':=20= get_option('injection_points')=20?=20'yes'=20:=20'no',=0A+=09},=0A+=20=20= =20=20'tests':=20[=0A+=20=20=20=20=20=20't/001_basic.pl',=0A+=20=20=20=20= =20=20't/002_restarts.pl',=0A+=20=20=20=20=20=20= 't/003_standby_restarts.pl',=0A+=20=20=20=20=20=20't/004_offline.pl',=0A= +=20=20=20=20=20=20't/005_injection.pl',=0A+=20=20=20=20],=0A+=20=20},=0A= +}=0Adiff=20--git=20a/src/test/modules/test_checksums/t/001_basic.pl=20= b/src/test/modules/test_checksums/t/001_basic.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..728a5c4510c=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/001_basic.pl=0A@@=20-0,0=20+1,63=20= @@=0A+=0A+#=20Copyright=20(c)=202025,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=0A+use=20strict;=0A+use=20= warnings=20FATAL=20=3D>=20'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A= +use=20PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20= FindBin;=0A+use=20lib=20$FindBin::RealBin;=0A+=0A+use=20= DataChecksums::Utils;=0A+=0A+#=20Initialize=20node=20with=20checksums=20= disabled.=0A+my=20$node=20=3D=20PostgreSQL::Test::Cluster->new('main');=0A= +$node->init(no_data_checksums=20=3D>=201);=0A+$node->start;=0A+=0A+#=20= Create=20some=20content=20to=20have=20un-checksummed=20data=20in=20the=20= cluster=0A+$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Ensure=20that=20= checksums=20are=20turned=20off=0A+test_checksum_state($node,=20'off');=0A= +=0A+#=20Enable=20data=20checksums=20and=20wait=20for=20the=20state=20= transition=20to=20'on'=0A+enable_data_checksums($node,=20wait=20=3D>=20= 'on');=0A+=0A+#=20Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20= can=20read=20back=20data=0A+my=20$result=20=3D=0A+=20=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201=20");=0A+is($result,=20'9999',=20'ensure=20checksummed=20pages=20= can=20be=20read=20back');=0A+=0A+#=20Enable=20data=20checksums=20again=20= which=20should=20be=20a=20no-op=20so=20we=20explicitly=20don't=0A+#=20= wait=20for=20any=20state=20transition=20as=20none=20should=20happen=20= here=0A+enable_data_checksums($node);=0A+test_checksum_state($node,=20= 'on');=0A+#=20..and=20make=20sure=20we=20can=20still=20read/write=20data=0A= +$node->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;");=0A+$result=20=3D=20$node->safe_psql('postgres',=20"SELECT=20= count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20'10000',=20= 'ensure=20checksummed=20pages=20can=20be=20read=20back');=0A+=0A+#=20= Disable=20checksums=20again=20and=20wait=20for=20the=20state=20= transition=0A+disable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A= +#=20Test=20reading=20data=20again=0A+$result=20=3D=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+is($result,=20'10000',=20'ensure=20previously=20checksummed=20= pages=20can=20be=20read=20back');=0A+=0A+#=20Re-enable=20checksums=20and=20= make=20sure=20that=20the=20underlying=20data=20has=20changed=20to=0A+#=20= ensure=20that=20checksums=20will=20be=20different.=0A= +$node->safe_psql('postgres',=20"UPDATE=20t=20SET=20a=20=3D=20a=20+=20= 1;");=0A+enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+#=20= Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20can=20read=20= back=20the=20data=0A+$result=20=3D=20$node->safe_psql('postgres',=20= "SELECT=20count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20= '10000',=20'ensure=20checksummed=20pages=20can=20be=20read=20back');=0A+=0A= +$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/002_restarts.pl=20= b/src/test/modules/test_checksums/t/002_restarts.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..26e60ea3ee1=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/002_restarts.pl=0A@@=20-0,0=20+1,110=20= @@=0A+=0A+#=20Copyright=20(c)=202024,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=20a=0A+#=20restart=20= which=20breaks=20processing.=0A+use=20strict;=0A+use=20warnings=20FATAL=20= =3D>=20'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A+use=20= PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20FindBin;=0A= +use=20lib=20$FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A= +#=20Initialize=20node=20with=20checksums=20disabled.=0A+my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('main');=0A+$node->init(no_data_checksums=20= =3D>=201);=0A+$node->start;=0A+=0A+#=20Initialize=20result=20storage=20= for=20queries=0A+my=20$result;=0A+=0A+#=20Create=20some=20content=20to=20= have=20un-checksummed=20data=20in=20the=20cluster=0A= +$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20SELECT=20= generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Ensure=20that=20= checksums=20are=20disabled=0A+test_checksum_state($node,=20'off');=0A+=0A= +SKIP:=0A+{=0A+=09skip=20'Data=20checksum=20delay=20tests=20not=20= enabled=20in=20PG_TEST_EXTRA',=206=0A+=09=20=20if=20= (!$ENV{PG_TEST_EXTRA}=0A+=09=09||=20$ENV{PG_TEST_EXTRA}=20!~=20= /\bchecksum_delays\b/);=0A+=0A+=09#=20Create=20a=20barrier=20for=20= checksumming=20to=20block=20on,=20in=20this=20case=20a=20pre-=0A+=09#=20= existing=20temporary=20table=20which=20is=20kept=20open=20while=20= processing=20is=20started.=0A+=09#=20We=20can=20accomplish=20this=20by=20= setting=20up=20an=20interactive=20psql=20process=20which=0A+=09#=20keeps=20= the=20temporary=20table=20created=20as=20we=20enable=20checksums=20in=20= another=20psql=0A+=09#=20process.=0A+=09#=0A+=09#=20This=20is=20a=20= similar=20test=20to=20the=20synthetic=20variant=20in=20005_injection.pl=0A= +=09#=20which=20fakes=20this=20scenario.=0A+=09my=20$bsession=20=3D=20= $node->background_psql('postgres');=0A+=09$bsession->query_safe('CREATE=20= TEMPORARY=20TABLE=20tt=20(a=20integer);');=0A+=0A+=09#=20In=20another=20= session,=20make=20sure=20we=20can=20see=20the=20blocking=20temp=20table=20= but=0A+=09#=20start=20processing=20anyways=20and=20check=20that=20we=20= are=20blocked=20with=20a=20proper=0A+=09#=20wait=20event.=0A+=09$result=20= =3D=20$node->safe_psql('postgres',=0A+=09=09"SELECT=20relpersistence=20= FROM=20pg_catalog.pg_class=20WHERE=20relname=20=3D=20'tt';"=0A+=09);=0A+=09= is($result,=20't',=20'ensure=20we=20can=20see=20the=20temporary=20= table');=0A+=0A+=09#=20Enabling=20data=20checksums=20shouldn't=20work=20= as=20the=20process=20is=20blocked=20on=20the=0A+=09#=20temporary=20table=20= held=20open=20by=20$bsession.=20Ensure=20that=20we=20reach=20inprogress-=0A= +=09#=20on=20before=20we=20do=20more=20tests.=0A+=09= enable_data_checksums($node,=20wait=20=3D>=20'inprogress-on');=0A+=0A+=09= #=20Wait=20for=20processing=20to=20finish=20and=20the=20worker=20waiting=20= for=20leftover=20temp=0A+=09#=20relations=20to=20be=20able=20to=20= actually=20finish=0A+=09$result=20=3D=20$node->poll_query_until(=0A+=09=09= 'postgres',=0A+=09=09"SELECT=20wait_event=20FROM=20= pg_catalog.pg_stat_activity=20"=0A+=09=09=20=20.=20"WHERE=20backend_type=20= =3D=20'datachecksumsworker=20worker';",=0A+=09=09= 'ChecksumEnableTemptableWait');=0A+=0A+=09#=20The=20datachecksumsworker=20= waits=20for=20temporary=20tables=20to=20disappear=20for=203=0A+=09#=20= seconds=20before=20retrying,=20so=20sleep=20for=204=20seconds=20to=20be=20= guaranteed=20to=20see=0A+=09#=20a=20retry=20cycle=0A+=09sleep(4);=0A+=0A= +=09#=20Re-check=20the=20wait=20event=20to=20ensure=20we=20are=20blocked=20= on=20the=20right=20thing.=0A+=09$result=20=3D=20= $node->safe_psql('postgres',=0A+=09=09=09"SELECT=20wait_event=20FROM=20= pg_catalog.pg_stat_activity=20"=0A+=09=09=20=20.=20"WHERE=20backend_type=20= =3D=20'datachecksumsworker=20worker';");=0A+=09is($result,=20= 'ChecksumEnableTemptableWait',=0A+=09=09'ensure=20the=20correct=20wait=20= condition=20is=20set');=0A+=09test_checksum_state($node,=20= 'inprogress-on');=0A+=0A+=09#=20Stop=20the=20cluster=20while=20bsession=20= is=20still=20attached.=20=20We=20can't=20close=20the=0A+=09#=20session=20= first=20since=20the=20brief=20period=20between=20closing=20and=20= stopping=20might=0A+=09#=20be=20enough=20for=20checksums=20to=20get=20= enabled.=0A+=09$node->stop;=0A+=09$bsession->quit;=0A+=09$node->start;=0A= +=0A+=09#=20Ensure=20the=20checksums=20aren't=20enabled=20across=20the=20= restart.=20=20This=20leaves=20the=0A+=09#=20cluster=20in=20the=20same=20= state=20as=20before=20we=20entered=20the=20SKIP=20block.=0A+=09= test_checksum_state($node,=20'off');=0A+}=0A+=0A= +enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+$result=20=3D=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+is($result,=20'9999',=20'ensure=20checksummed=20pages=20can=20= be=20read=20back');=0A+=0A+$result=20=3D=20$node->poll_query_until(=0A+=09= 'postgres',=0A+=09"SELECT=20count(*)=20FROM=20pg_stat_activity=20WHERE=20= backend_type=20LIKE=20'datachecksumsworker%';",=0A+=09'0');=0A= +is($result,=201,=20'await=20datachecksums=20worker/launcher=20= termination');=0A+=0A+disable_data_checksums($node,=20wait=20=3D>=201);=0A= +=0A+$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/003_standby_restarts.pl=20= b/src/test/modules/test_checksums/t/003_standby_restarts.pl=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..fe34b4d7d05=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/003_standby_restarts.pl=0A@@=20-0,0=20= +1,114=20@@=0A+=0A+#=20Copyright=20(c)=202024,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=0A+#=20streaming=20= replication=0A+use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20'all';=0A= +use=20PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A= +use=20Test::More;=0A+=0A+use=20FindBin;=0A+use=20lib=20= $FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A+#=20= Initialize=20primary=20node=0A+my=20$node_primary=20=3D=20= PostgreSQL::Test::Cluster->new('primary');=0A= +$node_primary->init(allows_streaming=20=3D>=201,=20no_data_checksums=20= =3D>=201);=0A+$node_primary->start;=0A+=0A+my=20$slotname=20=3D=20= 'physical_slot';=0A+$node_primary->safe_psql('postgres',=0A+=09"SELECT=20= pg_create_physical_replication_slot('$slotname')");=0A+=0A+#=20Take=20= backup=0A+my=20$backup_name=20=3D=20'my_backup';=0A= +$node_primary->backup($backup_name);=0A+=0A+#=20Create=20streaming=20= standby=20linking=20to=20primary=0A+my=20$node_standby_1=20=3D=20= PostgreSQL::Test::Cluster->new('standby_1');=0A= +$node_standby_1->init_from_backup($node_primary,=20$backup_name,=0A+=09= has_streaming=20=3D>=201);=0A+$node_standby_1->append_conf(=0A+=09= 'postgresql.conf',=20qq[=0A+primary_slot_name=20=3D=20'$slotname'=0A+]);=0A= +$node_standby_1->start;=0A+=0A+#=20Create=20some=20content=20on=20the=20= primary=20to=20have=20un-checksummed=20data=20in=20the=20cluster=0A= +$node_primary->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20= SELECT=20generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Wait=20for=20= standbys=20to=20catch=20up=0A= +$node_primary->wait_for_catchup($node_standby_1,=20'replay',=0A+=09= $node_primary->lsn('insert'));=0A+=0A+#=20Check=20that=20checksums=20are=20= turned=20off=20on=20all=20nodes=0A+test_checksum_state($node_primary,=20= 'off');=0A+test_checksum_state($node_standby_1,=20'off');=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Enable=20checksums=20for=20the=20cluster,=20and=20make=20sure=20= that=20both=20the=20primary=20and=0A+#=20standby=20change=20state.=0A+#=0A= +=0A+#=20Ensure=20that=20the=20primary=20switches=20to=20"inprogress-on"=0A= +enable_data_checksums($node_primary,=20wait=20=3D>=20'inprogress-on');=0A= +#=20Wait=20for=20checksum=20enable=20to=20be=20replayed=0A= +$node_primary->wait_for_catchup($node_standby_1,=20'replay');=0A+=0A+#=20= Ensure=20that=20the=20standby=20has=20switched=20to=20"inprogress-on"=20= or=20"on".=20=20Normally=20it=0A+#=20would=20be=20"inprogress-on",=20but=20= it=20is=20theoretically=20possible=20for=20the=20primary=20to=0A+#=20= complete=20the=20checksum=20enabling=20*and*=20have=20the=20standby=20= replay=20that=20record=0A+#=20before=20we=20reach=20the=20check=20below.=0A= +my=20$result=20=3D=20$node_standby_1->poll_query_until(=0A+=09= 'postgres',=0A+=09"SELECT=20setting=20=3D=20'off'=20FROM=20= pg_catalog.pg_settings=20WHERE=20name=20=3D=20'data_checksums';",=0A+=09= 'f');=0A+is($result,=201,=20'ensure=20standby=20has=20absorbed=20the=20= inprogress-on=20barrier');=0A+$result=20=3D=20= $node_standby_1->safe_psql('postgres',=0A+=09"SELECT=20setting=20FROM=20= pg_catalog.pg_settings=20WHERE=20name=20=3D=20'data_checksums';"=0A+);=0A= +=0A+is(($result=20eq=20'inprogress-on'=20||=20$result=20eq=20'on'),=0A+=09= 1,=20'ensure=20checksums=20are=20on,=20or=20in=20progress,=20on=20= standby_1');=0A+=0A+#=20Insert=20some=20more=20data=20which=20should=20= be=20checksummed=20on=20INSERT=0A+$node_primary->safe_psql('postgres',=0A= +=09"INSERT=20INTO=20t=20VALUES=20(generate_series(1,=2010000));");=0A+=0A= +#=20Wait=20for=20checksums=20enabled=20on=20the=20primary=20and=20= standby=0A+wait_for_checksum_state($node_primary,=20'on');=0A= +wait_for_checksum_state($node_standby_1,=20'on');=0A+=0A+$result=20=3D=0A= +=20=20$node_primary->safe_psql('postgres',=20"SELECT=20count(a)=20FROM=20= t=20WHERE=20a=20>=201");=0A+is($result,=20'19998',=20'ensure=20we=20can=20= safely=20read=20all=20data=20with=20checksums');=0A+=0A+$result=20=3D=20= $node_primary->poll_query_until(=0A+=09'postgres',=0A+=09"SELECT=20= count(*)=20FROM=20pg_stat_activity=20WHERE=20backend_type=20LIKE=20= 'datachecksumsworker%';",=0A+=09'0');=0A+is($result,=201,=20'await=20= datachecksums=20worker/launcher=20termination');=0A+=0A+#=0A+#=20Disable=20= checksums=20and=20ensure=20it's=20propagated=20to=20standby=20and=20that=20= we=20can=0A+#=20still=20read=20all=20data=0A+#=0A+=0A+#=20Disable=20= checksums=20and=20wait=20for=20the=20operation=20to=20be=20replayed=0A= +disable_data_checksums($node_primary);=0A= +$node_primary->wait_for_catchup($node_standby_1,=20'replay');=0A+#=20= Ensure=20that=20the=20primary=20abd=20standby=20has=20switched=20to=20= off=0A+wait_for_checksum_state($node_primary,=20'off');=0A= +wait_for_checksum_state($node_standby_1,=20'off');=0A+#=20Doublecheck=20= reading=20data=20withourt=20errors=0A+$result=20=3D=0A+=20=20= $node_primary->safe_psql('postgres',=20"SELECT=20count(a)=20FROM=20t=20= WHERE=20a=20>=201");=0A+is($result,=20"19998",=20'ensure=20we=20can=20= safely=20read=20all=20data=20without=20checksums');=0A+=0A= +$node_standby_1->stop;=0A+$node_primary->stop;=0A+done_testing();=0A= diff=20--git=20a/src/test/modules/test_checksums/t/004_offline.pl=20= b/src/test/modules/test_checksums/t/004_offline.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..e9fbcf77eab=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/004_offline.pl=0A@@=20-0,0=20+1,82=20= @@=0A+=0A+#=20Copyright=20(c)=202024,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20offline=20from=20various=20states=0A+#=20of=20= checksum=20processing=0A+use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20= 'all';=0A+=0A+use=20PostgreSQL::Test::Cluster;=0A+use=20= PostgreSQL::Test::Utils;=0A+use=20Test::More;=0A+=0A+use=20FindBin;=0A= +use=20lib=20$FindBin::RealBin;=0A+=0A+use=20DataChecksums::Utils;=0A+=0A= +#=20Initialize=20node=20with=20checksums=20disabled.=0A+my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('main');=0A+$node->init(no_data_checksums=20= =3D>=201);=0A+$node->start;=0A+=0A+#=20Create=20some=20content=20to=20= have=20un-checksummed=20data=20in=20the=20cluster=0A= +$node->safe_psql('postgres',=0A+=09"CREATE=20TABLE=20t=20AS=20SELECT=20= generate_series(1,10000)=20AS=20a;");=0A+=0A+#=20Ensure=20that=20= checksums=20are=20disabled=0A+test_checksum_state($node,=20'off');=0A+=0A= +#=20Enable=20checksums=20offline=20using=20pg_checksums=0A+$node->stop;=0A= +$node->checksum_enable_offline;=0A+$node->start;=0A+=0A+#=20Ensure=20= that=20checksums=20are=20enabled=0A+test_checksum_state($node,=20'on');=0A= +=0A+#=20Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20can=20= read=20back=20some=20data=0A+my=20$result=20=3D=0A+=20=20= $node->safe_psql('postgres',=20"SELECT=20count(*)=20FROM=20t=20WHERE=20a=20= >=201");=0A+is($result,=20'9999',=20'ensure=20checksummed=20pages=20can=20= be=20read=20back');=0A+=0A+#=20Disable=20checksums=20offline=20again=20= using=20pg_checksums=0A+$node->stop;=0A+$node->checksum_disable_offline;=0A= +$node->start;=0A+=0A+#=20Ensure=20that=20checksums=20are=20disabled=0A= +test_checksum_state($node,=20'off');=0A+=0A+#=20Create=20a=20barrier=20= for=20checksumming=20to=20block=20on,=20in=20this=20case=20a=20= pre-existing=0A+#=20temporary=20table=20which=20is=20kept=20open=20while=20= processing=20is=20started.=20We=20can=0A+#=20accomplish=20this=20by=20= setting=20up=20an=20interactive=20psql=20process=20which=20keeps=20the=0A= +#=20temporary=20table=20created=20as=20we=20enable=20checksums=20in=20= another=20psql=20process.=0A+=0A+my=20$bsession=20=3D=20= $node->background_psql('postgres');=0A+$bsession->query_safe('CREATE=20= TEMPORARY=20TABLE=20tt=20(a=20integer);');=0A+=0A+#=20In=20another=20= session,=20make=20sure=20we=20can=20see=20the=20blocking=20temp=20table=20= but=20start=0A+#=20processing=20anyways=20and=20check=20that=20we=20are=20= blocked=20with=20a=20proper=20wait=20event.=0A+$result=20=3D=20= $node->safe_psql('postgres',=0A+=09"SELECT=20relpersistence=20FROM=20= pg_catalog.pg_class=20WHERE=20relname=20=3D=20'tt';");=0A+is($result,=20= 't',=20'ensure=20we=20can=20see=20the=20temporary=20table');=0A+=0A= +enable_data_checksums($node,=20wait=20=3D>=20'inprogress-on');=0A+=0A+#=20= Turn=20the=20cluster=20off=20and=20enable=20checksums=20offline,=20then=20= start=20back=20up=0A+$bsession->quit;=0A+$node->stop;=0A= +$node->checksum_enable_offline;=0A+$node->start;=0A+=0A+#=20Ensure=20= that=20checksums=20are=20now=20enabled=20even=20though=20processing=20= wasn't=0A+#=20restarted=0A+test_checksum_state($node,=20'on');=0A+=0A+#=20= Run=20a=20dummy=20query=20just=20to=20make=20sure=20we=20can=20read=20= back=20some=20data=0A+$result=20=3D=20$node->safe_psql('postgres',=20= "SELECT=20count(*)=20FROM=20t=20WHERE=20a=20>=201");=0A+is($result,=20= '9999',=20'ensure=20checksummed=20pages=20can=20be=20read=20back');=0A+=0A= +$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/005_injection.pl=20= b/src/test/modules/test_checksums/t/005_injection.pl=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..e9f4bfdcd91=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/005_injection.pl=0A@@=20-0,0=20+1,76=20= @@=0A+=0A+#=20Copyright=20(c)=202025,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+#=20Test=20suite=20for=20testing=20enabling=20= data=20checksums=20in=20an=20online=20cluster=20with=0A+#=20injection=20= point=20tests=20injecting=20failures=20into=20the=20processing=0A+=0A= +use=20strict;=0A+use=20warnings=20FATAL=20=3D>=20'all';=0A+=0A+use=20= PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A+use=20= Test::More;=0A+=0A+use=20FindBin;=0A+use=20lib=20$FindBin::RealBin;=0A+=0A= +use=20DataChecksums::Utils;=0A+=0A+if=20($ENV{enable_injection_points}=20= ne=20'yes')=0A+{=0A+=09plan=20skip_all=20=3D>=20'Injection=20points=20= not=20supported=20by=20this=20build';=0A+}=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Test=20cluster=20setup=0A+#=0A+=0A+#=20Initiate=20testcluster=0A= +my=20$node=20=3D=20PostgreSQL::Test::Cluster->new('main');=0A= +$node->init(no_data_checksums=20=3D>=201);=0A+$node->start;=0A+=0A+#=20= Set=20up=20test=20environment=0A+$node->safe_psql('postgres',=20'CREATE=20= EXTENSION=20test_checksums;');=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Inducing=20failures=20in=20processing=0A+=0A+#=20Force=20= enabling=20checksums=20to=20fail=20by=20marking=20one=20of=20the=20= databases=20as=20having=0A+#=20failed=20in=20processing.=0A= +disable_data_checksums($node,=20wait=20=3D>=201);=0A= +$node->safe_psql('postgres',=20'SELECT=20= dcw_inject_fail_database(true);');=0A+enable_data_checksums($node,=20= wait=20=3D>=20'off');=0A+$node->safe_psql('postgres',=20'SELECT=20= dcw_inject_fail_database(false);');=0A+=0A+#=20Force=20the=20enable=20= checksums=20processing=20to=20make=20multiple=20passes=20by=20removing=0A= +#=20one=20database=20from=20the=20list=20in=20the=20first=20pass.=20=20= This=20will=20simulate=20a=20CREATE=0A+#=20DATABASE=20during=20= processing.=20=20Doing=20this=20via=20fault=20injection=20makes=20the=20= test=0A+#=20not=20be=20subject=20to=20exact=20timing.=0A= +$node->safe_psql('postgres',=20'SELECT=20dcw_prune_dblist(true);');=0A= +enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+#=20= --------------------------------------------------------------------------= -=0A+#=20Timing=20and=20retry=20related=20tests=0A+#=0A+SKIP:=0A+{=0A+=09= skip=20'Data=20checksum=20delay=20tests=20not=20enabled=20in=20= PG_TEST_EXTRA',=204=0A+=09=20=20if=20(!$ENV{PG_TEST_EXTRA}=0A+=09=09||=20= $ENV{PG_TEST_EXTRA}=20!~=20/\bchecksum_delays\b/);=0A+=0A+=09#=20Inject=20= a=20delay=20in=20the=20barrier=20for=20enabling=20checksums=0A+=09= disable_data_checksums($node,=20wait=20=3D>=201);=0A+=09= $node->safe_psql('postgres',=20'SELECT=20dcw_inject_delay_barrier();');=0A= +=09enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+=0A+=09#=20= Fake=20the=20existence=20of=20a=20temporary=20table=20at=20the=20start=20= of=20processing,=20which=0A+=09#=20will=20force=20the=20processing=20to=20= wait=20and=20retry=20in=20order=20to=20wait=20for=20it=20to=0A+=09#=20= disappear.=0A+=09disable_data_checksums($node,=20wait=20=3D>=201);=0A+=09= $node->safe_psql('postgres',=20'SELECT=20dcw_fake_temptable(true);');=0A= +=09enable_data_checksums($node,=20wait=20=3D>=20'on');=0A+}=0A+=0A= +$node->stop;=0A+done_testing();=0Adiff=20--git=20= a/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=20= b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..ee2f2a1428f=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm=0A@@=20-0,0=20= +1,185=20@@=0A+=0A+#=20Copyright=20(c)=202025,=20PostgreSQL=20Global=20= Development=20Group=0A+=0A+=3Dpod=0A+=0A+=3Dhead1=20NAME=0A+=0A= +DataChecksums::Utils=20-=20Utility=20functions=20for=20testing=20data=20= checksums=20in=20a=20running=20cluster=0A+=0A+=3Dhead1=20SYNOPSIS=0A+=0A= +=20=20use=20PostgreSQL::Test::Cluster;=0A+=20=20use=20= DataChecksums::Utils=20qw(=20..=20);=0A+=0A+=20=20#=20Create,=20and=20= start,=20a=20new=20cluster=0A+=20=20my=20$node=20=3D=20= PostgreSQL::Test::Cluster->new('primary');=0A+=20=20$node->init;=0A+=20=20= $node->start;=0A+=0A+=20=20test_checksum_state($node,=20'off');=0A+=0A+=20= =20enable_data_checksums($node);=0A+=0A+=20=20= wait_for_checksum_state($node,=20'on');=0A+=0A+=0A+=3Dcut=0A+=0A+package=20= DataChecksums::Utils;=0A+=0A+use=20strict;=0A+use=20warnings=20FATAL=20= =3D>=20'all';=0A+use=20Exporter=20'import';=0A+use=20= PostgreSQL::Test::Cluster;=0A+use=20PostgreSQL::Test::Utils;=0A+use=20= Test::More;=0A+=0A+our=20@EXPORT=20=3D=20qw(=0A+=20=20= test_checksum_state=0A+=20=20wait_for_checksum_state=0A+=20=20= enable_data_checksums=0A+=20=20disable_data_checksums=0A+);=0A+=0A+=3Dpod=0A= +=0A+=3Dhead1=20METHODS=0A+=0A+=3Dover=0A+=0A+=3Ditem=20= test_checksum_state(node,=20state)=0A+=0A+Test=20that=20the=20current=20= value=20of=20the=20data=20checksum=20GUC=20in=20the=20server=20running=0A= +at=20B=20matches=20B.=20=20If=20the=20values=20differ,=20a=20= test=20failure=20is=20logged.=0A+Returns=20True=20if=20the=20values=20= match,=20otherwise=20False.=0A+=0A+=3Dcut=0A+=0A+sub=20= test_checksum_state=0A+{=0A+=09my=20($postgresnode,=20$state)=20=3D=20= @_;=0A+=0A+=09my=20$result=20=3D=20$postgresnode->safe_psql('postgres',=0A= +=09=09"SELECT=20setting=20FROM=20pg_catalog.pg_settings=20WHERE=20name=20= =3D=20'data_checksums';"=0A+=09);=0A+=09is($result,=20$state,=20'ensure=20= checksums=20are=20set=20to=20'=20.=20$state);=0A+=09return=20$result=20= eq=20$state;=0A+}=0A+=0A+=3Ditem=20wait_for_checksum_state(node,=20= state)=0A+=0A+Test=20the=20value=20of=20the=20data=20checksum=20GUC=20in=20= the=20server=20running=20at=20B=0A+repeatedly=20until=20it=20= matches=20B=20or=20times=20out.=20=20Processing=20will=20run=20= for=0A+$PostgreSQL::Test::Utils::timeout_default=20seconds=20before=20= timing=20out.=20=20If=20the=0A+values=20differ=20when=20the=20process=20= times=20out,=20False=20is=20returned=20and=20a=20test=20failure=0A+is=20= logged,=20otherwise=20True.=0A+=0A+=3Dcut=0A+=0A+sub=20= wait_for_checksum_state=0A+{=0A+=09my=20($postgresnode,=20$state)=20=3D=20= @_;=0A+=0A+=09my=20$res=20=3D=20$postgresnode->poll_query_until(=0A+=09=09= 'postgres',=0A+=09=09"SELECT=20setting=20FROM=20pg_catalog.pg_settings=20= WHERE=20name=20=3D=20'data_checksums';",=0A+=09=09$state);=0A+=09= is($res,=201,=20'ensure=20data=20checksums=20are=20transitioned=20to=20'=20= .=20$state);=0A+=09return=20$res=20=3D=3D=201;=0A+}=0A+=0A+=3Ditem=20= enable_data_checksums($node,=20%params)=0A+=0A+Function=20for=20enabling=20= data=20checksums=20in=20the=20cluster=20running=20at=20B.=0A+=0A= +=3Dover=0A+=0A+=3Ditem=20cost_delay=0A+=0A+The=20C=20to=20= use=20when=20enabling=20data=20checksums,=20default=20is=200.=0A+=0A= +=3Ditem=20cost_limit=0A+=0A+The=20C=20to=20use=20when=20= enabling=20data=20checksums,=20default=20is=20100.=0A+=0A+=3Ditem=20fast=0A= +=0A+If=20set=20to=20C=20an=20immediate=20checkpoint=20will=20be=20= issued=20after=20data=0A+checksums=20are=20enabled.=20=20Setting=20this=20= to=20false=20will=20lead=20to=20slower=20tests.=0A+The=20default=20is=20= true.=0A+=0A+=3Ditem=20wait=0A+=0A+If=20defined,=20the=20function=20will=20= wait=20for=20the=20state=20defined=20in=20this=20parameter,=0A+waiting=20= timing=20out,=20before=20returning.=20=20The=20function=20will=20wait=20= for=0A+$PostgreSQL::Test::Utils::timeout_default=20seconds=20before=20= timing=20out.=0A+=0A+=3Dback=0A+=0A+=3Dcut=0A+=0A+sub=20= enable_data_checksums=0A+{=0A+=09my=20$postgresnode=20=3D=20shift;=0A+=09= my=20%params=20=3D=20@_;=0A+=0A+=09#=20Set=20sane=20defaults=20for=20the=20= parameters=0A+=09$params{cost_delay}=20=3D=200=20unless=20= (defined($params{cost_delay}));=0A+=09$params{cost_limit}=20=3D=20100=20= unless=20(defined($params{cost_limit}));=0A+=09$params{fast}=20=3D=20= 'true'=20unless=20(defined($params{fast}));=0A+=0A+=09my=20$query=20=3D=20= <<'EOQ';=0A+SELECT=20pg_enable_data_checksums(%s,=20%s,=20%s);=0A+EOQ=0A= +=0A+=09$postgresnode->safe_psql(=0A+=09=09'postgres',=0A+=09=09= sprintf($query,=0A+=09=09=09$params{cost_delay},=20$params{cost_limit},=20= $params{fast}));=0A+=0A+=09wait_for_checksum_state($postgresnode,=20= $params{wait})=0A+=09=20=20if=20(defined($params{wait}));=0A+}=0A+=0A= +=3Ditem=20disable_data_checksums($node,=20%params)=0A+=0A+Function=20= for=20disabling=20data=20checksums=20in=20the=20cluster=20running=20at=20= B.=0A+=0A+=3Dover=0A+=0A+=3Ditem=20wait=0A+=0A+If=20defined,=20the=20= function=20will=20wait=20for=20the=20state=20to=20turn=20to=20B,=20= or=0A+waiting=20timing=20out,=20before=20returning.=20=20The=20function=20= will=20wait=20for=0A+$PostgreSQL::Test::Utils::timeout_default=20seconds=20= before=20timing=20out.=0A+Unlike=20in=20C=20the=20= value=20of=20the=20parameter=20is=20discarded.=0A+=0A+=3Dback=0A+=0A= +=3Dcut=0A+=0A+sub=20disable_data_checksums=0A+{=0A+=09my=20= $postgresnode=20=3D=20shift;=0A+=09my=20%params=20=3D=20@_;=0A+=0A+=09#=20= Set=20sane=20defaults=20for=20the=20parameters=0A+=09$params{fast}=20=3D=20= 'true'=20unless=20(defined($params{fast}));=0A+=0A+=09my=20$query=20=3D=20= <<'EOQ';=0A+SELECT=20pg_disable_data_checksums(%s);=0A+EOQ=0A+=0A+=09= $postgresnode->safe_psql('postgres',=20sprintf($query,=20= $params{fast}));=0A+=0A+=09wait_for_checksum_state($postgresnode,=20= 'off')=20if=20(defined($params{wait}));=0A+}=0A+=0A+=3Dpod=0A+=0A+=3Dback=0A= +=0A+=3Dcut=0A+=0A+1;=0Adiff=20--git=20= a/src/test/modules/test_checksums/test_checksums--1.0.sql=20= b/src/test/modules/test_checksums/test_checksums--1.0.sql=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..704b45a3186=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/test_checksums--1.0.sql=0A@@=20-0,0=20= +1,20=20@@=0A+/*=20= src/test/modules/test_checksums/test_checksums--1.0.sql=20*/=0A+=0A+--=20= complain=20if=20script=20is=20sourced=20in=20psql,=20rather=20than=20via=20= CREATE=20EXTENSION=0A+\echo=20Use=20"CREATE=20EXTENSION=20= test_checksums"=20to=20load=20this=20file.=20\quit=0A+=0A+CREATE=20= FUNCTION=20dcw_inject_delay_barrier(attach=20boolean=20DEFAULT=20true)=0A= +=09RETURNS=20pg_catalog.void=0A+=09AS=20'MODULE_PATHNAME'=20LANGUAGE=20= C;=0A+=0A+CREATE=20FUNCTION=20dcw_inject_fail_database(attach=20boolean=20= DEFAULT=20true)=0A+=09RETURNS=20pg_catalog.void=0A+=09AS=20= 'MODULE_PATHNAME'=20LANGUAGE=20C;=0A+=0A+CREATE=20FUNCTION=20= dcw_prune_dblist(attach=20boolean=20DEFAULT=20true)=0A+=09RETURNS=20= pg_catalog.void=0A+=09AS=20'MODULE_PATHNAME'=20LANGUAGE=20C;=0A+=0A= +CREATE=20FUNCTION=20dcw_fake_temptable(attach=20boolean=20DEFAULT=20= true)=0A+=09RETURNS=20pg_catalog.void=0A+=09AS=20'MODULE_PATHNAME'=20= LANGUAGE=20C;=0Adiff=20--git=20= a/src/test/modules/test_checksums/test_checksums.c=20= b/src/test/modules/test_checksums/test_checksums.c=0Anew=20file=20mode=20= 100644=0Aindex=2000000000000..26897bff960=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/test_checksums.c=0A@@=20-0,0=20+1,173=20= @@=0A= +/*-----------------------------------------------------------------------= ---=0A+=20*=0A+=20*=20test_checksums.c=0A+=20*=09=09Test=20data=20= checksums=0A+=20*=0A+=20*=20Copyright=20(c)=202025,=20PostgreSQL=20= Global=20Development=20Group=0A+=20*=0A+=20*=20IDENTIFICATION=0A+=20*=09=09= src/test/modules/test_checksums/test_checksums.c=0A+=20*=0A+=20*=20= -------------------------------------------------------------------------=0A= +=20*/=0A+#include=20"postgres.h"=0A+=0A+#include=20"funcapi.h"=0A= +#include=20"miscadmin.h"=0A+#include=20= "postmaster/datachecksumsworker.h"=0A+#include=20"storage/latch.h"=0A= +#include=20"utils/injection_point.h"=0A+#include=20"utils/wait_event.h"=0A= +=0A+#define=20USEC_PER_SEC=20=20=20=201000000=0A+=0A+=0A= +PG_MODULE_MAGIC;=0A+=0A+extern=20PGDLLEXPORT=20void=20= dc_delay_barrier(const=20char=20*name,=20const=20void=20*private_data,=20= void=20*arg);=0A+extern=20PGDLLEXPORT=20void=20dc_fail_database(const=20= char=20*name,=20const=20void=20*private_data,=20void=20*arg);=0A+extern=20= PGDLLEXPORT=20void=20dc_dblist(const=20char=20*name,=20const=20void=20= *private_data,=20void=20*arg);=0A+extern=20PGDLLEXPORT=20void=20= dc_fake_temptable(const=20char=20*name,=20const=20void=20*private_data,=20= void=20*arg);=0A+=0A+/*=0A+=20*=20Test=20for=20delaying=20emission=20of=20= procsignalbarriers.=0A+=20*/=0A+void=0A+dc_delay_barrier(const=20char=20= *name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09(void)=20= name;=0A+=09(void)=20private_data;=0A+=0A+=09(void)=20WaitLatch(MyLatch,=0A= +=09=09=09=09=09=20WL_LATCH_SET=20|=20WL_TIMEOUT=20|=20= WL_EXIT_ON_PM_DEATH,=0A+=09=09=09=09=09=20(3=20*=201000),=0A+=09=09=09=09= =09=20WAIT_EVENT_PG_SLEEP);=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_inject_delay_barrier);=0A+Datum=0A= +dcw_inject_delay_barrier(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksums-enable-checksums-delay",=0A+=09=09=09= =09=09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_delay_barrier",=0A+=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09= =09=200);=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksums-enable-checksums-delay");=0A+#else=0A= +=09elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20intended=20= when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+void=0A+dc_fail_database(const=20char=20= *name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09static=20= bool=20first_pass=20=3D=20true;=0A+=09DataChecksumsWorkerResult=20*res=20= =3D=20(DataChecksumsWorkerResult=20*)=20arg;=0A+=0A+=09if=20(first_pass)=0A= +=09=09*res=20=3D=20DATACHECKSUMSWORKER_FAILED;=0A+=09first_pass=20=3D=20= false;=0A+}=0A+=0A+PG_FUNCTION_INFO_V1(dcw_inject_fail_database);=0A= +Datum=0A+dcw_inject_fail_database(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-fail-db",=0A+=09=09=09=09=09=09= =09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20"dc_fail_database",=0A= +=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09=09=200);=0A+=09else=0A= +=09=09InjectionPointDetach("datachecksumsworker-fail-db");=0A+#else=0A+=09= elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20intended=20= when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+/*=0A+=20*=20Test=20to=20remove=20an=20= entry=20from=20the=20Databaselist=20to=20force=20re-processing=20since=0A= +=20*=20not=20all=20databases=20could=20be=20processed=20in=20the=20= first=20iteration=20of=20the=20loop.=0A+=20*/=0A+void=0A+dc_dblist(const=20= char=20*name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09= static=20bool=20first_pass=20=3D=20true;=0A+=09List=09=20=20=20= *DatabaseList=20=3D=20(List=20*)=20arg;=0A+=0A+=09if=20(first_pass)=0A+=09= =09DatabaseList=20=3D=20list_delete_last(DatabaseList);=0A+=09first_pass=20= =3D=20false;=0A+}=0A+=0A+PG_FUNCTION_INFO_V1(dcw_prune_dblist);=0A+Datum=0A= +dcw_prune_dblist(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-initial-dblist",=0A+=09=09=09=09= =09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20"dc_dblist",=0A= +=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09=09=200);=0A+=09else=0A= +=09=09InjectionPointDetach("datachecksumsworker-initial-dblist");=0A= +#else=0A+=09elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20= intended=20when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0A+=0A+/*=0A+=20*=20Test=20to=20force=20waiting=20= for=20existing=20temptables.=0A+=20*/=0A+void=0A+dc_fake_temptable(const=20= char=20*name,=20const=20void=20*private_data,=20void=20*arg)=0A+{=0A+=09= static=20bool=20first_pass=20=3D=20true;=0A+=09int=09=09=20=20=20= *numleft=20=3D=20(int=20*)=20arg;=0A+=0A+=09if=20(first_pass)=0A+=09=09= *numleft=20=3D=201;=0A+=09first_pass=20=3D=20false;=0A+}=0A+=0A= +PG_FUNCTION_INFO_V1(dcw_fake_temptable);=0A+Datum=0A= +dcw_fake_temptable(PG_FUNCTION_ARGS)=0A+{=0A+#ifdef=20= USE_INJECTION_POINTS=0A+=09bool=09=09attach=20=3D=20PG_GETARG_BOOL(0);=0A= +=0A+=09if=20(attach)=0A+=09=09= InjectionPointAttach("datachecksumsworker-fake-temptable-wait",=0A+=09=09= =09=09=09=09=09=20"test_checksums",=0A+=09=09=09=09=09=09=09=20= "dc_fake_temptable",=0A+=09=09=09=09=09=09=09=20NULL,=0A+=09=09=09=09=09=09= =09=200);=0A+=09else=0A+=09=09= InjectionPointDetach("datachecksumsworker-fake-temptable-wait");=0A= +#else=0A+=09elog(ERROR,=0A+=09=09=20"test=20is=20not=20working=20as=20= intended=20when=20injection=20points=20are=20disabled");=0A+#endif=0A+=09= PG_RETURN_VOID();=0A+}=0Adiff=20--git=20= a/src/test/modules/test_checksums/test_checksums.control=20= b/src/test/modules/test_checksums/test_checksums.control=0Anew=20file=20= mode=20100644=0Aindex=2000000000000..84b4cc035a7=0A---=20/dev/null=0A+++=20= b/src/test/modules/test_checksums/test_checksums.control=0A@@=20-0,0=20= +1,4=20@@=0A+comment=20=3D=20'Test=20code=20for=20data=20checksums'=0A= +default_version=20=3D=20'1.0'=0A+module_pathname=20=3D=20= '$libdir/test_checksums'=0A+relocatable=20=3D=20true=0Adiff=20--git=20= a/src/test/perl/PostgreSQL/Test/Cluster.pm=20= b/src/test/perl/PostgreSQL/Test/Cluster.pm=0Aindex=20= 35413f14019..950433ea929=20100644=0A---=20= a/src/test/perl/PostgreSQL/Test/Cluster.pm=0A+++=20= b/src/test/perl/PostgreSQL/Test/Cluster.pm=0A@@=20-3872,6=20+3872,42=20= @@=20sub=20advance_wal=0A=20=09}=0A=20}=0A=20=0A+=3Ditem=20= $node->checksum_enable_offline()=0A+=0A+Enable=20data=20page=20checksums=20= in=20an=20offline=20cluster=20with=20B.=20The=0A+caller=20= is=20responsible=20for=20ensuring=20that=20the=20cluster=20is=20in=20the=20= right=20state=20for=0A+this=20operation.=0A+=0A+=3Dcut=0A+=0A+sub=20= checksum_enable_offline=0A+{=0A+=09my=20($self)=20=3D=20@_;=0A+=0A+=09= print=20"###=20Enabling=20checksums=20in=20\"$self->data_dir\"\n";=0A+=09= PostgreSQL::Test::Utils::system_or_bail('pg_checksums',=20'-D',=0A+=09=09= $self->data_dir,=20'-e');=0A+=09return;=0A+}=0A+=0A+=3Ditem=20= checksum_disable_offline=0A+=0A+Disable=20data=20page=20checksums=20in=20= an=20offline=20cluster=20with=20B.=20The=0A+caller=20is=20= responsible=20for=20ensuring=20that=20the=20cluster=20is=20in=20the=20= right=20state=20for=0A+this=20operation.=0A+=0A+=3Dcut=0A+=0A+sub=20= checksum_disable_offline=0A+{=0A+=09my=20($self)=20=3D=20@_;=0A+=0A+=09= print=20"###=20Disabling=20checksums=20in=20\"$self->data_dir\"\n";=0A+=09= PostgreSQL::Test::Utils::system_or_bail('pg_checksums',=20'-D',=0A+=09=09= $self->data_dir,=20'-d');=0A+=09return;=0A+}=0A+=0A=20=3Dpod=0A=20=0A=20= =3Dback=0Adiff=20--git=20a/src/test/regress/expected/rules.out=20= b/src/test/regress/expected/rules.out=0Aindex=2035e8aad7701..4b9c5526e50=20= 100644=0A---=20a/src/test/regress/expected/rules.out=0A+++=20= b/src/test/regress/expected/rules.out=0A@@=20-2071,6=20+2071,42=20@@=20= pg_stat_progress_create_index|=20SELECT=20s.pid,=0A=20=20=20=20=20= s.param15=20AS=20partitions_done=0A=20=20=20=20FROM=20= (pg_stat_get_progress_info('CREATE=20INDEX'::text)=20s(pid,=20datid,=20= relid,=20param1,=20param2,=20param3,=20param4,=20param5,=20param6,=20= param7,=20param8,=20param9,=20param10,=20param11,=20param12,=20param13,=20= param14,=20param15,=20param16,=20param17,=20param18,=20param19,=20= param20)=0A=20=20=20=20=20=20LEFT=20JOIN=20pg_database=20d=20ON=20= ((s.datid=20=3D=20d.oid)));=0A+pg_stat_progress_data_checksums|=20SELECT=20= s.pid,=0A+=20=20=20=20s.datid,=0A+=20=20=20=20d.datname,=0A+=20=20=20=20=20= =20=20=20CASE=20s.param1=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=200=20= THEN=20'enabling'::text=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=201=20= THEN=20'disabling'::text=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=202=20= THEN=20'waiting'::text=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=203=20= THEN=20'waiting=20on=20temporary=20tables'::text=0A+=20=20=20=20=20=20=20= =20=20=20=20=20WHEN=204=20THEN=20'waiting=20on=20checkpoint'::text=0A+=20= =20=20=20=20=20=20=20=20=20=20=20WHEN=205=20THEN=20'done'::text=0A+=20=20= =20=20=20=20=20=20=20=20=20=20ELSE=20NULL::text=0A+=20=20=20=20=20=20=20=20= END=20AS=20phase,=0A+=20=20=20=20=20=20=20=20CASE=20s.param2=0A+=20=20=20= =20=20=20=20=20=20=20=20=20WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A= +=20=20=20=20=20=20=20=20=20=20=20=20ELSE=20s.param2=0A+=20=20=20=20=20=20= =20=20END=20AS=20databases_total,=0A+=20=20=20=20s.param3=20AS=20= databases_done,=0A+=20=20=20=20=20=20=20=20CASE=20s.param4=0A+=20=20=20=20= =20=20=20=20=20=20=20=20WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A+=20= =20=20=20=20=20=20=20=20=20=20=20ELSE=20s.param4=0A+=20=20=20=20=20=20=20= =20END=20AS=20relations_total,=0A+=20=20=20=20=20=20=20=20CASE=20= s.param5=0A+=20=20=20=20=20=20=20=20=20=20=20=20WHEN=20'-1'::integer=20= THEN=20NULL::bigint=0A+=20=20=20=20=20=20=20=20=20=20=20=20ELSE=20= s.param5=0A+=20=20=20=20=20=20=20=20END=20AS=20relations_done,=0A+=20=20=20= =20=20=20=20=20CASE=20s.param6=0A+=20=20=20=20=20=20=20=20=20=20=20=20= WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A+=20=20=20=20=20=20=20=20=20= =20=20=20ELSE=20s.param6=0A+=20=20=20=20=20=20=20=20END=20AS=20= blocks_total,=0A+=20=20=20=20=20=20=20=20CASE=20s.param7=0A+=20=20=20=20=20= =20=20=20=20=20=20=20WHEN=20'-1'::integer=20THEN=20NULL::bigint=0A+=20=20= =20=20=20=20=20=20=20=20=20=20ELSE=20s.param7=0A+=20=20=20=20=20=20=20=20= END=20AS=20blocks_done=0A+=20=20=20FROM=20= (pg_stat_get_progress_info('DATACHECKSUMS'::text)=20s(pid,=20datid,=20= relid,=20param1,=20param2,=20param3,=20param4,=20param5,=20param6,=20= param7,=20param8,=20param9,=20param10,=20param11,=20param12,=20param13,=20= param14,=20param15,=20param16,=20param17,=20param18,=20param19,=20= param20)=0A+=20=20=20=20=20LEFT=20JOIN=20pg_database=20d=20ON=20= ((s.datid=20=3D=20d.oid)))=0A+=20=20ORDER=20BY=20s.datid;=0A=20= pg_stat_progress_vacuum|=20SELECT=20s.pid,=0A=20=20=20=20=20s.datid,=0A=20= =20=20=20=20d.datname,=0Adiff=20--git=20= a/src/test/regress/expected/stats.out=20= b/src/test/regress/expected/stats.out=0Aindex=20605f5070376..9042e4d38e3=20= 100644=0A---=20a/src/test/regress/expected/stats.out=0A+++=20= b/src/test/regress/expected/stats.out=0A@@=20-59,6=20+59,22=20@@=20io=20= worker|relation|vacuum=0A=20io=20worker|temp=20relation|normal=0A=20io=20= worker|wal|init=0A=20io=20worker|wal|normal=0A+datachecksumsworker=20= launcher|relation|bulkread=0A+datachecksumsworker=20= launcher|relation|bulkwrite=0A+datachecksumsworker=20= launcher|relation|init=0A+datachecksumsworker=20launcher|relation|normal=0A= +datachecksumsworker=20launcher|relation|vacuum=0A+datachecksumsworker=20= launcher|temp=20relation|normal=0A+datachecksumsworker=20= launcher|wal|init=0A+datachecksumsworker=20launcher|wal|normal=0A= +datachecksumsworker=20worker|relation|bulkread=0A+datachecksumsworker=20= worker|relation|bulkwrite=0A+datachecksumsworker=20worker|relation|init=0A= +datachecksumsworker=20worker|relation|normal=0A+datachecksumsworker=20= worker|relation|vacuum=0A+datachecksumsworker=20worker|temp=20= relation|normal=0A+datachecksumsworker=20worker|wal|init=0A= +datachecksumsworker=20worker|wal|normal=0A=20slotsync=20= worker|relation|bulkread=0A=20slotsync=20worker|relation|bulkwrite=0A=20= slotsync=20worker|relation|init=0A@@=20-95,7=20+111,7=20@@=20= walsummarizer|wal|init=0A=20walsummarizer|wal|normal=0A=20= walwriter|wal|init=0A=20walwriter|wal|normal=0A-(79=20rows)=0A+(87=20= rows)=0A=20\a=0A=20--=20ensure=20that=20both=20seqscan=20and=20indexscan=20= plans=20are=20allowed=0A=20SET=20enable_seqscan=20TO=20on;=0Adiff=20= --git=20a/src/tools/pgindent/typedefs.list=20= b/src/tools/pgindent/typedefs.list=0Aindex=20e6f2e93b2d6..56c2bf09ae4=20= 100644=0A---=20a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-416,6=20+416,7=20@@=20= CheckPointStmt=0A=20CheckpointStatsData=0A=20CheckpointerRequest=0A=20= CheckpointerShmemStruct=0A+ChecksumType=0A=20Chromosome=0A=20= CkptSortItem=0A=20CkptTsStatus=0A@@=20-608,6=20+609,10=20@@=20= DataPageDeleteStack=0A=20DataTypesUsageChecks=0A=20= DataTypesUsageVersionCheck=0A=20DatabaseInfo=0A= +DataChecksumsWorkerDatabase=0A+DataChecksumsWorkerResult=0A= +DataChecksumsWorkerResultEntry=0A+DataChecksumsWorkerShmemStruct=0A=20= DateADT=0A=20DateTimeErrorExtra=0A=20Datum=0A@@=20-4244,6=20+4249,7=20@@=20= xl_btree_split=0A=20xl_btree_unlink_page=0A=20xl_btree_update=0A=20= xl_btree_vacuum=0A+xl_checksum_state=0A=20xl_clog_truncate=0A=20= xl_commit_ts_truncate=0A=20xl_dbase_create_file_copy_rec=0A--=20=0A= 2.39.3=20(Apple=20Git-146)=0A=0A= --Apple-Mail=_3DBBE924-A497-4C66-A4E1-9A3177F6ED66--