Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kX56R-0000j5-KJ for pgsql-hackers@arkaria.postgresql.org; Mon, 26 Oct 2020 16:12:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kX56P-0007im-KQ for pgsql-hackers@arkaria.postgresql.org; Mon, 26 Oct 2020 16:12:09 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kX56O-0007ic-Gf for pgsql-hackers@lists.postgresql.org; Mon, 26 Oct 2020 16:12:09 +0000 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kX56K-0006V8-3o for pgsql-hackers@postgresql.org; Mon, 26 Oct 2020 16:12:06 +0000 Received: by mail-pl1-x635.google.com with SMTP id h2so4942850pll.11 for ; Mon, 26 Oct 2020 09:12:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=eN40FFljcKu+F494xiLrN6wzgXHEoG/qEmyKRf+An/s=; b=WbJa8o8IrFmMz9kGbqAI2GXvxJqotW/6JgtSdbqt7/m5WgRePIslKIPB1GQFKHAtqz fUx+MNpsIA5JouXnKt1iLFw7GJzNMlIP7nn/H/ixk/O+jD6Xs8blfZ0+bmdj+pLbdlDu Nnop7yepikHB865P/2iqdsqPG3uAHAUROznLMAWn+I3cFp+Px9NeMWrXMO8FMxKYgaww xS7HffL7CInOMHtZZxerZeZyY8o6XOkO3gGduvusj6ChXSZH5IU8rPvb/ymqjlAAJadt ctk0qD4zXjw/aYgPMQKwjhXzNuDXnaSIwDuy7EsmfHVH7nXf3+ReMJWLKwjNGP5pjilw MhJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=eN40FFljcKu+F494xiLrN6wzgXHEoG/qEmyKRf+An/s=; b=iddkvJx9Pb379q/iAN/MJNPGLlmVWJBc0zWTP7wcE16P4ALsPscmWdNpQQ60pCcoJ2 16gXeQplXR7v/pA3RyTaalKt5v2iaexLX3YWqr1Lj6CTwYbCbyz8NMLddH6v4JKvY3gq gkm4CjWBkZ8mwdP74tp8IAz5xzf06X/JXH/cRLD5mytqMnqF9U33fMkRaZr5RZgBi4FX yzcKfaVTVRj4hkTkmU+iUrBykKd5QSECiXZD4ZP6SfsMAtTCgV3wjy86b8VahGRm79n1 R07aeR3lRp/V2GJY7/KTHN/6+rag+RY89l4zzsFMSFWqwHGmUKrwTiBucdEy2QsNKtR5 mtyw== X-Gm-Message-State: AOAM531dytriSKGRJUWzRUilpDufB/KSDb7doomLH41EjrdwwR6TBYmL gZCiLwUTHGPsiwML14ceSGnlqPr1B1nvhypprUuHVWKw9231z2txJgYJojt2I4sQml3MWZNvHe1 WCnOyFe0py6kW7gNrkrH0kihX2Bl9SXIsA1aPLbtGf05c6c3No2fhnQUBfTfkak+PRXTbMygiBq CA+A8PE6PoPF2VVMK2kPJ06EoBDyYQf41k0Utp X-Google-Smtp-Source: ABdhPJxzQWQJ0L1uhjsgPzMpY1KcSj0Ya8qE4s6VMRPfuh7TUSlCMLyN/eT7GiGEKKLFC/H27JiFkQ== X-Received: by 2002:a17:90a:f187:: with SMTP id bv7mr21175571pjb.198.1603728722811; Mon, 26 Oct 2020 09:12:02 -0700 (PDT) Received: from [192.168.0.29] (24-113-193-150.wavecable.com. [24.113.193.150]) by smtp.gmail.com with ESMTPSA id x198sm3190915pfd.187.2020.10.26.09.12.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 09:12:01 -0700 (PDT) From: Mark Dilger Message-Id: <85BE72E5-A926-4F23-9FD5-64239C0C6BA5@enterprisedb.com> Content-Type: multipart/mixed; boundary="Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: new heapcheck contrib module Date: Mon, 26 Oct 2020 09:11:59 -0700 In-Reply-To: <153EEBAD-2673-4367-B36D-4A6AE8EE7A9F@enterprisedb.com> Cc: Tom Lane , Peter Geoghegan , "Andrey M. Borodin" , Stephen Frost , Michael Paquier , Amul Sul , Dilip Kumar , Andres Freund , PostgreSQL-development To: Robert Haas References: <12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com> <20200420194210.sevnpi4lyghoanxi@alap3.anarazel.de> <23E60C3C-C20A-4508-84AC-E0CEA59E784A@enterprisedb.com> <5F743835-3399-419C-8324-2D424237E999@enterprisedb.com> <885D2EC9-7D06-4903-A17D-72C184A1E1C5@enterprisedb.com> <5CC85F28-F210-47AC-8DA8-EC4E82AF3FD5@enterprisedb.com> <0C0349EB-DD85-4DA8-82C7-DF70AB714592@enterprisedb.com> <9DF93B83-FE46-4EC4-A99D-77A60729FD2D@enterprisedb.com> <50F6468F-5A5E-412A-86EC-DF24B549D733@enterprisedb.com> <7D603B53-8078-4AF0-851B-803FF4416501@enterprisedb.com> <98265AA9-7BDC-4708-9348-CC1B173BBE1D@enterprisedb.com> <209783.1603313248@sss.pgh.pa.us> <77A60E99-6054-40FD-A5D6-769C287D4A0F@enterprisedb.com> <721BFA83-EA6F-463D-A9DF-CE78A318B73A@enterprisedb.com> <2A7DA1A8-C4AA-43DF-A985-3CA52F4DC775@enterprisedb.com> <146814.1603463293@sss.pgh.pa.us> <221029.1603476265@sss.pgh.pa.us> <153EEBAD-2673-4367-B36D-4A6AE8EE7A9F@enterprisedb.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 26, 2020, at 7:08 AM, Mark Dilger = wrote: >=20 >=20 >=20 >> On Oct 26, 2020, at 7:00 AM, Robert Haas = wrote: >>=20 >> On Mon, Oct 26, 2020 at 9:56 AM Mark Dilger >> wrote: >>> Much of the test in 0002 could be ported to work without committing = the rest of 0002, if the pg_amcheck command line utiilty is not wanted. >>=20 >> How much consensus do we think we have around 0002 at this point? I >> think I remember a vote in favor and no votes against, but I haven't >> been paying a whole lot of attention. >=20 > My sense over the course of the thread is that people were very much = in favor of having heap checking functionality, but quite vague on = whether they wanted the command line interface. I think the interface = is useful, but I'd rather hear from others on this list whether it is = useful enough to justify maintaining it. As the author of it, I'm = biased. Hopefully others with a more objective view of the matter will = read this and vote? >=20 > I don't recall patches 0003 through 0005 getting any votes. 0003 and = 0004, which create and use a non-throwing interface to clog, were = written in response to Andrey's request, so I'm guessing that's kind of = a vote in favor. 0005 was factored out of of 0001 in response to a lack = of agreement about whether verify_heapam should have acl checks. You = seemed in favor, and Peter against, but I don't think we heard other = opinions. The v20 patches 0002, 0003, and 0005 still apply cleanly, but 0004 = required a rebase. (0001 was already committed last week.) Here is a rebased set of 4 patches, numbered 0002..0005 to be consistent = with the previous naming. There are no substantial changes. --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F Content-Disposition: attachment; filename=v24-0002-Adding-contrib-module-pg_amcheck.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v24-0002-Adding-contrib-module-pg_amcheck.patch" Content-Transfer-Encoding: quoted-printable =46rom=20a4a0f6c30f083859f25d28b1df5111cbc7ecd823=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Wed,=2021=20Oct=202020=2020:25:21=20-0700=0ASubject:=20[PATCH=20= v24=201/4]=20Adding=20contrib=20module=20pg_amcheck=0A=0AAdding=20new=20= contrib=20module=20pg_amcheck,=20which=20is=20a=20command=20line=0A= interface=20for=20running=20amcheck's=20verifications=20against=20tables=20= and=0Aindexes.=0A---=0A=20contrib/Makefile=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= contrib/pg_amcheck/.gitignore=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=20=202=20+=0A=20contrib/pg_amcheck/Makefile=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20|=20=20=2028=20+=0A=20= contrib/pg_amcheck/pg_amcheck.c=20=20=20=20=20=20=20=20=20=20=20=20|=20= 1281=20++++++++++++++++++++=0A=20contrib/pg_amcheck/pg_amcheck.control=20= =20=20=20=20=20|=20=20=20=205=20+=0A=20contrib/pg_amcheck/t/001_basic.pl=20= =20=20=20=20=20=20=20=20=20|=20=20=20=209=20+=0A=20= contrib/pg_amcheck/t/002_nonesuch.pl=20=20=20=20=20=20=20|=20=20=2060=20= +=0A=20contrib/pg_amcheck/t/003_check.pl=20=20=20=20=20=20=20=20=20=20|=20= =20231=20++++=0A=20contrib/pg_amcheck/t/004_verify_heapam.pl=20=20|=20=20= 489=20++++++++=0A=20contrib/pg_amcheck/t/005_opclass_damage.pl=20|=20=20=20= 52=20+=0A=20doc/src/sgml/contrib.sgml=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=20=20=201=20+=0A=20doc/src/sgml/filelist.sgml=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= doc/src/sgml/pgamcheck.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20228=20++++=0A=20src/tools/msvc/Mkvcbuild.pm=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20|=20=20=20=206=20+-=0A=20= src/tools/pgindent/typedefs.list=20=20=20=20=20=20=20=20=20=20=20|=20=20=20= =202=20+=0A=2015=20files=20changed,=202393=20insertions(+),=203=20= deletions(-)=0A=20create=20mode=20100644=20contrib/pg_amcheck/.gitignore=0A= =20create=20mode=20100644=20contrib/pg_amcheck/Makefile=0A=20create=20= mode=20100644=20contrib/pg_amcheck/pg_amcheck.c=0A=20create=20mode=20= 100644=20contrib/pg_amcheck/pg_amcheck.control=0A=20create=20mode=20= 100644=20contrib/pg_amcheck/t/001_basic.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/002_nonesuch.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/003_check.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/004_verify_heapam.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/005_opclass_damage.pl=0A=20create=20mode=20100644=20= doc/src/sgml/pgamcheck.sgml=0A=0Adiff=20--git=20a/contrib/Makefile=20= b/contrib/Makefile=0Aindex=207a4866e338..0fd4125902=20100644=0A---=20= a/contrib/Makefile=0A+++=20b/contrib/Makefile=0A@@=20-30,6=20+30,7=20@@=20= SUBDIRS=20=3D=20\=0A=20=09=09old_snapshot=09\=0A=20=09=09pageinspect=09\=0A= =20=09=09passwordcheck=09\=0A+=09=09pg_amcheck=09\=0A=20=09=09= pg_buffercache=09\=0A=20=09=09pg_freespacemap=20\=0A=20=09=09pg_prewarm=09= \=0Adiff=20--git=20a/contrib/pg_amcheck/.gitignore=20= b/contrib/pg_amcheck/.gitignore=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..f8eecf70bf=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/.gitignore=0A@@=20-0,0=20+1,2=20@@=0A+/pg_amcheck=0A= +/tmp_check/=0Adiff=20--git=20a/contrib/pg_amcheck/Makefile=20= b/contrib/pg_amcheck/Makefile=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..74554b9e8d=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/Makefile=0A@@=20-0,0=20+1,28=20@@=0A+#=20= contrib/pg_amcheck/Makefile=0A+=0A+PGFILEDESC=20=3D=20"pg_amcheck=20-=20= detects=20corruption=20within=20database=20relations"=0A+PGAPPICON=20=3D=20= win32=0A+=0A+PROGRAM=20=3D=20pg_amcheck=0A+OBJS=20=3D=20\=0A+=09= $(WIN32RES)=20\=0A+=09pg_amcheck.o=0A+=0A+REGRESS_OPTS=20+=3D=20= --load-extension=3Damcheck=20--load-extension=3Dpageinspect=0A= +EXTRA_INSTALL=20+=3D=20contrib/amcheck=20contrib/pageinspect=0A+=0A= +TAP_TESTS=20=3D=201=0A+=0A+PG_CPPFLAGS=20=3D=20-I$(libpq_srcdir)=0A= +PG_LIBS_INTERNAL=20=3D=20-L$(top_builddir)/src/fe_utils=20-lpgfeutils=20= $(libpq_pgport)=0A+=0A+ifdef=20USE_PGXS=0A+PG_CONFIG=20=3D=20pg_config=0A= +PGXS=20:=3D=20$(shell=20$(PG_CONFIG)=20--pgxs)=0A+include=20$(PGXS)=0A= +else=0A+subdir=20=3D=20contrib/pg_amcheck=0A+top_builddir=20=3D=20../..=0A= +include=20$(top_builddir)/src/Makefile.global=0A+include=20= $(top_srcdir)/contrib/contrib-global.mk=0A+endif=0Adiff=20--git=20= a/contrib/pg_amcheck/pg_amcheck.c=20b/contrib/pg_amcheck/pg_amcheck.c=0A= new=20file=20mode=20100644=0Aindex=200000000000..6d20ff3d78=0A---=20= /dev/null=0A+++=20b/contrib/pg_amcheck/pg_amcheck.c=0A@@=20-0,0=20= +1,1281=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20pg_amcheck.c=0A+=20*=09=09Detects=20corruption=20= within=20database=20relations.=0A+=20*=0A+=20*=20Copyright=20(c)=20= 2017-2020,=20PostgreSQL=20Global=20Development=20Group=0A+=20*=0A+=20*=20= IDENTIFICATION=0A+=20*=09=20=20contrib/pg_amcheck/pg_amcheck.c=0A+=20*=0A= +=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres_fe.h"=0A+=0A+#include=20= "catalog/pg_am.h"=0A+#include=20"catalog/pg_class.h"=0A+#include=20= "common/logging.h"=0A+#include=20"common/username.h"=0A+#include=20= "common/connect.h"=0A+#include=20"common/string.h"=0A+#include=20= "fe_utils/print.h"=0A+#include=20"fe_utils/simple_list.h"=0A+#include=20= "fe_utils/string_utils.h"=0A+#include=20"pg_getopt.h"=0A+=0A+const=20= char=20*usage_text[]=20=3D=20{=0A+=09"pg_amcheck=20is=20the=20PostgreSQL=20= command=20line=20frontend=20for=20the=20amcheck=20database=20corruption=20= checker.",=0A+=09"",=0A+=09"Usage:",=0A+=09"=20=20pg_amcheck=20= [OPTION]...=20[DBNAME=20[USERNAME]]",=0A+=09"",=0A+=09"General=20= options:",=0A+=09"=20=20-V,=20--version=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20output=20version=20information,=20then=20exit",=0A+=09"=20=20= -?,=20--help=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= show=20this=20help,=20then=20exit",=0A+=09"=20=20-s,=20--strict-names=20=20= =20=20=20=20=20=20=20=20=20require=20include=20patterns=20to=20match=20= at=20least=20one=20entity=20each",=0A+=09"=20=20-o,=20--on-error-stop=20=20= =20=20=20=20=20=20=20=20stop=20checking=20at=20end=20of=20first=20= corrupt=20page",=0A+=09"",=0A+=09"Schema=20checking=20options:",=0A+=09"=20= =20-n,=20--schema=3DPATTERN=20=20=20=20=20=20=20=20=20check=20relations=20= in=20the=20specified=20schema(s)=20only",=0A+=09"=20=20-N,=20= --exclude-schema=3DPATTERN=20do=20NOT=20check=20relations=20in=20the=20= specified=20schema(s)",=0A+=09"",=0A+=09"Table=20checking=20options:",=0A= +=09"=20=20-t,=20--table=3DPATTERN=20=20=20=20=20=20=20=20=20=20check=20= the=20specified=20table(s)=20only",=0A+=09"=20=20-T,=20= --exclude-table=3DPATTERN=20=20do=20NOT=20check=20the=20specified=20= table(s)",=0A+=09"=20=20-b,=20--startblock=20=20=20=20=20=20=20=20=20=20=20= =20=20begin=20checking=20table(s)=20at=20the=20given=20starting=20block=20= number",=0A+=09"=20=20-e,=20--endblock=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20check=20table(s)=20only=20up=20to=20the=20given=20ending=20= block=20number",=0A+=09"=20=20-f,=20--skip-all-frozen=20=20=20=20=20=20=20= =20do=20NOT=20check=20blocks=20marked=20as=20all=20frozen",=0A+=09"=20=20= -v,=20--skip-all-visible=20=20=20=20=20=20=20do=20NOT=20check=20blocks=20= marked=20as=20all=20visible",=0A+=09"",=0A+=09"TOAST=20table=20checking=20= options:",=0A+=09"=20=20-z,=20--check-toast=20=20=20=20=20=20=20=20=20=20= =20=20check=20associated=20toast=20tables=20and=20toast=20indexes",=0A+=09= "=20=20-Z,=20--skip-toast=20=20=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20= check=20associated=20toast=20tables=20and=20toast=20indexes",=0A+=09"=20=20= -B,=20--toast-startblock=20=20=20=20=20=20=20begin=20checking=20toast=20= table(s)=20at=20the=20given=20starting=20block",=0A+=09"=20=20-E,=20= --toast-endblock=20=20=20=20=20=20=20=20=20check=20toast=20table(s)=20= only=20up=20to=20the=20given=20ending=20block",=0A+=09"",=0A+=09"Index=20= checking=20options:",=0A+=09"=20=20-x,=20--check-indexes=20=20=20=20=20=20= =20=20=20=20check=20btree=20indexes=20associated=20with=20tables=20being=20= checked",=0A+=09"=20=20-X,=20--skip-indexes=20=20=20=20=20=20=20=20=20=20= =20do=20NOT=20check=20any=20btree=20indexes",=0A+=09"=20=20-i,=20= --index=3DPATTERN=20=20=20=20=20=20=20=20=20=20check=20the=20specified=20= index(es)=20only",=0A+=09"=20=20-I,=20--exclude-index=3DPATTERN=20=20do=20= NOT=20check=20the=20specified=20index(es)",=0A+=09"=20=20-c,=20= --check-corrupt=20=20=20=20=20=20=20=20=20=20check=20indexes=20even=20if=20= their=20associated=20table=20is=20corrupt",=0A+=09"=20=20-C,=20= --skip-corrupt=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20check=20= indexes=20if=20their=20associated=20table=20is=20corrupt",=0A+=09"=20=20= -a,=20--heapallindexed=20=20=20=20=20=20=20=20=20check=20index=20tuples=20= against=20the=20table=20tuples",=0A+=09"=20=20-A,=20--no-heapallindexed=20= =20=20=20=20=20do=20NOT=20check=20index=20tuples=20against=20the=20table=20= tuples",=0A+=09"=20=20-r,=20--rootdescend=20=20=20=20=20=20=20=20=20=20=20= =20search=20from=20the=20root=20page=20for=20each=20index=20tuple",=0A+=09= "=20=20-R,=20--no-rootdescend=20=20=20=20=20=20=20=20=20do=20NOT=20= search=20from=20the=20root=20page=20for=20each=20index=20tuple",=0A+=09= "",=0A+=09"Connection=20options:",=0A+=09"=20=20-d,=20--dbname=3DDBNAME=20= =20=20=20=20=20=20=20=20=20database=20name=20to=20connect=20to",=0A+=09"=20= =20-h,=20--host=3DHOSTNAME=20=20=20=20=20=20=20=20=20=20database=20= server=20host=20or=20socket=20directory",=0A+=09"=20=20-p,=20--port=3DPORT= =20=20=20=20=20=20=20=20=20=20=20=20=20=20database=20server=20port",=0A+=09= "=20=20-U,=20--username=3DUSERNAME=20=20=20=20=20=20database=20user=20= name",=0A+=09"=20=20-w,=20--no-password=20=20=20=20=20=20=20=20=20=20=20=20= never=20prompt=20for=20password",=0A+=09"=20=20-W,=20--password=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20force=20password=20prompt=20(should=20= happen=20automatically)",=0A+=09"",=0A+=09NULL=09=09=09=09=09=09/*=20= sentinel=20*/=0A+};=0A+=0A+typedef=20struct=0A+AmCheckSettings=0A+{=0A+=09= char=09=20=20=20*dbname;=0A+=09char=09=20=20=20*host;=0A+=09char=09=20=20= =20*port;=0A+=09char=09=20=20=20*username;=0A+}=20ConnectOptions;=0A+=0A= +typedef=20enum=20trivalue=0A+{=0A+=09TRI_DEFAULT,=0A+=09TRI_NO,=0A+=09= TRI_YES=0A+}=20trivalue;=0A+=0A+typedef=20struct=0A+{=0A+=09PGconn=09=20=20= =20*db;=09=09=09=09/*=20connection=20to=20backend=20*/=0A+=09bool=09=09= notty;=09=09=09/*=20stdin=20or=20stdout=20is=20not=20a=20tty=20(as=20= determined=0A+=09=09=09=09=09=09=09=09=20*=20on=20startup)=20*/=0A+=09= trivalue=09getPassword;=09/*=20prompt=20for=20a=20username=20and=20= password=20*/=0A+=09const=20char=20*progname;=09=09/*=20in=20case=20you=20= renamed=20pg_amcheck=20*/=0A+=09bool=09=09strict_names;=09/*=20The=20= specified=20names/patterns=20should=20to=0A+=09=09=09=09=09=09=09=09=20*=20= match=20at=20least=20one=20entity=20*/=0A+=09bool=09=09on_error_stop;=09= /*=20The=20checking=20of=20each=20table=20should=20stop=0A+=09=09=09=09=09= =09=09=09=20*=20after=20the=20first=20corrupt=20page=20is=20found.=20*/=0A= +=09bool=09=09skip_frozen;=09/*=20Do=20not=20check=20pages=20marked=20= all=20frozen=20*/=0A+=09bool=09=09skip_visible;=09/*=20Do=20not=20check=20= pages=20marked=20all=20visible=20*/=0A+=09bool=09=09check_indexes;=09/*=20= Check=20btree=20indexes=20*/=0A+=09bool=09=09check_toast;=09/*=20Check=20= associated=20toast=20tables=20and=20indexes=20*/=0A+=09bool=09=09= check_corrupt;=09/*=20Check=20indexes=20even=20if=20table=20is=20corrupt=20= */=0A+=09bool=09=09heapallindexed;=20/*=20Perform=20index=20to=20table=20= reconciling=20checks=20*/=0A+=09bool=09=09rootdescend;=09/*=20Perform=20= index=20rootdescend=20checks=20*/=0A+=09char=09=20=20=20*startblock;=09=09= /*=20Block=20number=20where=20checking=20begins=20*/=0A+=09char=09=20=20=20= *endblock;=09=09/*=20Block=20number=20where=20checking=20ends,=20= inclusive=20*/=0A+=09char=09=20=20=20*toaststart;=09=09/*=20Block=20= number=20where=20toast=20checking=20begins=20*/=0A+=09char=09=20=20=20= *toastend;=09=09/*=20Block=20number=20where=20toast=20checking=20ends,=0A= +=09=09=09=09=09=09=09=09=20*=20inclusive=20*/=0A+}=20AmCheckSettings;=0A= +=0A+static=20AmCheckSettings=20settings;=0A+=0A+/*=0A+=20*=20Object=20= inclusion/exclusion=20lists=0A+=20*=0A+=20*=20The=20string=20lists=20= record=20the=20patterns=20given=20by=20command-line=20switches,=0A+=20*=20= which=20we=20then=20convert=20to=20lists=20of=20Oids=20of=20matching=20= objects.=0A+=20*/=0A+static=20SimpleStringList=20schema_include_patterns=20= =3D=20{NULL,=20NULL};=0A+static=20SimpleOidList=20schema_include_oids=20= =3D=20{NULL,=20NULL};=0A+static=20SimpleStringList=20= schema_exclude_patterns=20=3D=20{NULL,=20NULL};=0A+static=20= SimpleOidList=20schema_exclude_oids=20=3D=20{NULL,=20NULL};=0A+=0A= +static=20SimpleStringList=20table_include_patterns=20=3D=20{NULL,=20= NULL};=0A+static=20SimpleOidList=20table_include_oids=20=3D=20{NULL,=20= NULL};=0A+static=20SimpleStringList=20table_exclude_patterns=20=3D=20= {NULL,=20NULL};=0A+static=20SimpleOidList=20table_exclude_oids=20=3D=20= {NULL,=20NULL};=0A+=0A+static=20SimpleStringList=20= index_include_patterns=20=3D=20{NULL,=20NULL};=0A+static=20SimpleOidList=20= index_include_oids=20=3D=20{NULL,=20NULL};=0A+static=20SimpleStringList=20= index_exclude_patterns=20=3D=20{NULL,=20NULL};=0A+static=20SimpleOidList=20= index_exclude_oids=20=3D=20{NULL,=20NULL};=0A+=0A+/*=0A+=20*=20List=20of=20= tables=20to=20be=20checked,=20compiled=20from=20above=20lists.=0A+=20*/=0A= +static=20SimpleOidList=20checklist=20=3D=20{NULL,=20NULL};=0A+=0A+/*=0A= +=20*=20Strings=20to=20be=20constructed=20once=20upon=20first=20use.=20=20= These=20could=20be=20made=0A+=20*=20string=20constants=20instead,=20but=20= that=20would=20require=20embedding=20knowledge=0A+=20*=20of=20the=20= single=20character=20values=20for=20each=20relkind,=20such=20as=20'm'=20= for=0A+=20*=20materialized=20views,=20which=20we'd=20rather=20not=20= embed=20here.=0A+=20*/=0A+static=20char=20*table_relkind_quals=20=3D=20= NULL;=0A+static=20char=20*index_relkind_quals=20=3D=20NULL;=0A+=0A+/*=0A= +=20*=20Functions=20to=20get=20pointers=20to=20the=20two=20strings,=20= above,=20after=20initializing=0A+=20*=20them=20upon=20the=20first=20call=20= to=20the=20function.=0A+=20*/=0A+static=20const=20char=20= *get_table_relkind_quals(void);=0A+static=20const=20char=20= *get_index_relkind_quals(void);=0A+=0A+/*=0A+=20*=20Functions=20for=20= running=20the=20various=20corruption=20checks.=0A+=20*/=0A+static=20void=20= check_tables(SimpleOidList=20*checklist);=0A+static=20uint64=20= check_toast(Oid=20tbloid);=0A+static=20uint64=20check_table(Oid=20= tbloid,=20const=20char=20*startblock,=0A+=09=09=09=09=09=09=20=20const=20= char=20*endblock,=20bool=20on_error_stop,=0A+=09=09=09=09=09=09=20=20= bool=20check_toast);=0A+static=20uint64=20check_indexes(Oid=20tbloid,=20= const=20SimpleOidList=20*include_oids,=0A+=09=09=09=09=09=09=09const=20= SimpleOidList=20*exclude_oids);=0A+static=20uint64=20check_index(const=20= char=20*idxoid,=20const=20char=20*idxname,=0A+=09=09=09=09=09=09=20=20= const=20char=20*tblname);=0A+=0A+/*=0A+=20*=20Functions=20implementing=20= standard=20command=20line=20behaviors.=0A+=20*/=0A+static=20void=20= parse_cli_options(int=20argc,=20char=20*argv[],=0A+=09=09=09=09=09=09=09=20= =20ConnectOptions=20*connOpts);=0A+static=20void=20usage(void);=0A= +static=20void=20showVersion(void);=0A+static=20void=20= NoticeProcessor(void=20*arg,=20const=20char=20*message);=0A+=0A+/*=0A+=20= *=20Functions=20for=20converting=20command=20line=20options=20that=20= include=20or=20exclude=0A+=20*=20schemas,=20tables,=20and=20indexes=20by=20= pattern=20into=20internally=20useful=20lists=20of=0A+=20*=20Oids=20for=20= objects=20that=20match=20those=20patterns.=0A+=20*/=0A+static=20void=20= expand_schema_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09=09= =09=09=09=09=09=09=09=09const=20SimpleOidList=20*exclude_oids,=0A+=09=09=09= =09=09=09=09=09=09=09SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09=09=09= =09bool=20strict_names);=0A+static=20void=20= expand_relkind_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09= =09=09=09=09=09=09=09=09=09=20const=20SimpleOidList=20*exclude_nsp_oids,=0A= +=09=09=09=09=09=09=09=09=09=09=20const=20SimpleOidList=20*exclude_oids,=0A= +=09=09=09=09=09=09=09=09=09=09=20SimpleOidList=20*oids,=0A+=09=09=09=09=09= =09=09=09=09=09=20bool=20strict_names,=0A+=09=09=09=09=09=09=09=09=09=09=20= const=20char=20*missing_errtext,=0A+=09=09=09=09=09=09=09=09=09=09=20= const=20char=20*relkind_quals);=0A+static=20void=20= expand_table_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09=09= =09=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20*exclude_nsp_oids,=0A= +=09=09=09=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20= *exclude_oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20SimpleOidList=20= *oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20bool=20strict_names);=0A= +static=20void=20expand_index_name_patterns(const=20SimpleStringList=20= *patterns,=0A+=09=09=09=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20= *exclude_nsp_oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20const=20= SimpleOidList=20*exclude_oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20= SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20bool=20= strict_names);=0A+static=20void=20get_table_check_list(const=20= SimpleOidList=20*include_nsp,=0A+=09=09=09=09=09=09=09=09=20const=20= SimpleOidList=20*exclude_nsp,=0A+=09=09=09=09=09=09=09=09=20const=20= SimpleOidList=20*include_tbl,=0A+=09=09=09=09=09=09=09=09=20const=20= SimpleOidList=20*exclude_tbl,=0A+=09=09=09=09=09=09=09=09=20= SimpleOidList=20*checklist);=0A+static=20PGresult=20= *ExecuteSqlQuery(const=20char=20*query,=20char=20**error);=0A+static=20= PGresult=20*ExecuteSqlQueryOrDie(const=20char=20*query);=0A+=0A+static=20= void=20append_csv_oids(PQExpBuffer=20querybuf,=20const=20SimpleOidList=20= *oids);=0A+static=20void=20apply_filter(PQExpBuffer=20querybuf,=20const=20= char=20*lval,=0A+=09=09=09=09=09=09=20const=20SimpleOidList=20*oids,=20= bool=20include);=0A+=0A+#define=20fatal(...)=20do=20{=20= pg_log_error(__VA_ARGS__);=20exit(1);=20}=20while(0)=0A+=0A+/*=20Like=20= fatal(),=20but=20with=20a=20complaint=20about=20a=20particular=20query.=20= */=0A+static=20void=0A+die_on_query_failure(const=20char=20*query)=0A+{=0A= +=09pg_log_error("query=20failed:=20%s",=0A+=09=09=09=09=20= PQerrorMessage(settings.db));=0A+=09fatal("query=20was:=20%s",=20query);=0A= +}=0A+=0A+#define=20EXIT_BADCONN=202=0A+=0A+int=0A+main(int=20argc,=20= char=20**argv)=0A+{=0A+=09ConnectOptions=20connOpts;=0A+=09bool=09=09= have_password=20=3D=20false;=0A+=09char=09=20=20=20*password=20=3D=20= NULL;=0A+=09bool=09=09new_pass;=0A+=0A+=09pg_logging_init(argv[0]);=0A+=09= set_pglocale_pgservice(argv[0],=20PG_TEXTDOMAIN("pg_amcheck"));=0A+=0A+=09= if=20(argc=20>=201)=0A+=09{=0A+=09=09if=20((strcmp(argv[1],=20"-?")=20=3D=3D= =200)=20||=0A+=09=09=09(argc=20=3D=3D=202=20&&=20(strcmp(argv[1],=20= "--help")=20=3D=3D=200)))=0A+=09=09{=0A+=09=09=09usage();=0A+=09=09=09= exit(EXIT_SUCCESS);=0A+=09=09}=0A+=09=09if=20(strcmp(argv[1],=20= "--version")=20=3D=3D=200=20||=20strcmp(argv[1],=20"-V")=20=3D=3D=200)=0A= +=09=09{=0A+=09=09=09showVersion();=0A+=09=09=09exit(EXIT_SUCCESS);=0A+=09= =09}=0A+=09}=0A+=0A+=09memset(&settings,=200,=20sizeof(settings));=0A+=09= settings.progname=20=3D=20get_progname(argv[0]);=0A+=0A+=09settings.db=20= =3D=20NULL;=0A+=09setDecimalLocale();=0A+=0A+=09settings.notty=20=3D=20= (!isatty(fileno(stdin))=20||=20!isatty(fileno(stdout)));=0A+=0A+=09= settings.getPassword=20=3D=20TRI_DEFAULT;=0A+=0A+=09/*=0A+=09=20*=20= Default=20behaviors=20for=20user=20settable=20options.=20=20Note=20that=20= these=20default=0A+=09=20*=20to=20doing=20all=20the=20safe=20checks=20= and=20none=20of=20the=20unsafe=20ones,=20on=20the=20theory=0A+=09=20*=20= that=20if=20a=20user=20says=20"pg_amcheck=20mydb"=20without=20specifying=20= any=20additional=0A+=09=20*=20options,=20we=20should=20check=20= everything=20we=20know=20how=20to=20check=20without=0A+=09=20*=20risking=20= any=20backend=20aborts.=0A+=09=20*/=0A+=0A+=09settings.on_error_stop=20=3D= =20false;=0A+=09settings.skip_frozen=20=3D=20false;=0A+=09= settings.skip_visible=20=3D=20false;=0A+=0A+=09/*=20Index=20checking=20= options=20*/=0A+=09settings.check_indexes=20=3D=20false;=0A+=09= settings.check_corrupt=20=3D=20false;=0A+=09settings.heapallindexed=20=3D=20= false;=0A+=09settings.rootdescend=20=3D=20false;=0A+=0A+=09/*=0A+=09=20*=20= Reconciling=20toasted=20attributes=20from=20the=20main=20table=20against=20= the=20toast=0A+=09=20*=20table=20can=20crash=20the=20backend=20if=20the=20= toast=20table=20or=20index=20are=20corrupt.=0A+=09=20*=20We=20can=20= optionally=20check=20the=20toast=20table=20and=20then=20the=20toast=20= index=20prior=0A+=09=20*=20to=20checking=20the=20main=20table,=20but=20= if=20the=20toast=20table=20or=20index=20are=0A+=09=20*=20concurrently=20= corrupted=20after=20we=20conclude=20they=20are=20valid,=20the=20check=20= of=0A+=09=20*=20the=20main=20table=20can=20crash=20the=20backend.=20=20= The=20oneous=20is=20on=20any=20caller=20who=0A+=09=20*=20enables=20this=20= option=20to=20make=20certain=20the=20environment=20is=20sufficiently=0A+=09= =20*=20stable=20that=20concurrent=20corruptions=20of=20the=20toast=20is=20= not=20possible.=0A+=09=20*/=0A+=09settings.check_toast=20=3D=20false;=0A= +=0A+=09parse_cli_options(argc,=20argv,=20&connOpts);=0A+=0A+=09if=20= (settings.getPassword=20=3D=3D=20TRI_YES)=0A+=09{=0A+=09=09/*=0A+=09=09=20= *=20We=20can't=20be=20sure=20yet=20of=20the=20username=20that=20will=20= be=20used,=20so=20don't=0A+=09=09=20*=20offer=20a=20potentially=20wrong=20= one.=20=20Typical=20uses=20of=20this=20option=20are=0A+=09=09=20*=20= noninteractive=20anyway.=0A+=09=09=20*/=0A+=09=09password=20=3D=20= simple_prompt("Password:=20",=20false);=0A+=09=09have_password=20=3D=20= true;=0A+=09}=0A+=0A+=09/*=20loop=20until=20we=20have=20a=20password=20= if=20requested=20by=20backend=20*/=0A+=09do=0A+=09{=0A+#define=20= ARRAY_SIZE=098=0A+=09=09const=20char=20**keywords=20=3D=20= pg_malloc(ARRAY_SIZE=20*=20sizeof(*keywords));=0A+=09=09const=20char=20= **values=20=3D=20pg_malloc(ARRAY_SIZE=20*=20sizeof(*values));=0A+=0A+=09=09= keywords[0]=20=3D=20"host";=0A+=09=09values[0]=20=3D=20connOpts.host;=0A= +=09=09keywords[1]=20=3D=20"port";=0A+=09=09values[1]=20=3D=20= connOpts.port;=0A+=09=09keywords[2]=20=3D=20"user";=0A+=09=09values[2]=20= =3D=20connOpts.username;=0A+=09=09keywords[3]=20=3D=20"password";=0A+=09=09= values[3]=20=3D=20have_password=20?=20password=20:=20NULL;=0A+=09=09= keywords[4]=20=3D=20"dbname";=20/*=20see=20do_connect()=20*/=0A+=09=09if=20= (connOpts.dbname=20=3D=3D=20NULL)=0A+=09=09{=0A+=09=09=09if=20= (getenv("PGDATABASE"))=0A+=09=09=09=09values[4]=20=3D=20= getenv("PGDATABASE");=0A+=09=09=09else=20if=20(getenv("PGUSER"))=0A+=09=09= =09=09values[4]=20=3D=20getenv("PGUSER");=0A+=09=09=09else=0A+=09=09=09=09= values[4]=20=3D=20"postgres";=0A+=09=09}=0A+=09=09else=0A+=09=09=09= values[4]=20=3D=20connOpts.dbname;=0A+=09=09keywords[5]=20=3D=20= "fallback_application_name";=0A+=09=09values[5]=20=3D=20= settings.progname;=0A+=09=09keywords[6]=20=3D=20"client_encoding";=0A+=09= =09values[6]=20=3D=20(settings.notty=20||=0A+=09=09=09=09=09=20= getenv("PGCLIENTENCODING"))=20?=20NULL=20:=20"auto";=0A+=09=09= keywords[7]=20=3D=20NULL;=0A+=09=09values[7]=20=3D=20NULL;=0A+=0A+=09=09= new_pass=20=3D=20false;=0A+=09=09settings.db=20=3D=20= PQconnectdbParams(keywords,=20values,=20true);=0A+=09=09if=20= (settings.db=20=3D=3D=20NULL)=0A+=09=09{=0A+=09=09=09pg_log_error("no=20= connection=20to=20server=20after=20initial=20attempt");=0A+=09=09=09= exit(EXIT_BADCONN);=0A+=09=09}=0A+=0A+=09=09free(keywords);=0A+=09=09= free(values);=0A+=0A+=09=09if=20(PQstatus(settings.db)=20=3D=3D=20= CONNECTION_BAD=20&&=0A+=09=09=09PQconnectionNeedsPassword(settings.db)=20= &&=0A+=09=09=09!have_password=20&&=0A+=09=09=09settings.getPassword=20!=3D= =20TRI_NO)=0A+=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20Before=20closing=20= the=20old=20PGconn,=20extract=20the=20user=20name=20that=20was=0A+=09=09=09= =20*=20actually=20connected=20with.=0A+=09=09=09=20*/=0A+=09=09=09const=20= char=20*realusername=20=3D=20PQuser(settings.db);=0A+=09=09=09char=09=20=20= =20*password_prompt;=0A+=0A+=09=09=09if=20(realusername=20&&=20= realusername[0])=0A+=09=09=09=09password_prompt=20=3D=20= psprintf("Password=20for=20user=20%s:=20",=0A+=09=09=09=09=09=09=09=09=09= =09=20=20=20realusername);=0A+=09=09=09else=0A+=09=09=09=09= password_prompt=20=3D=20pg_strdup("Password:=20");=0A+=09=09=09= PQfinish(settings.db);=0A+=0A+=09=09=09password=20=3D=20= simple_prompt(password_prompt,=20false);=0A+=09=09=09= free(password_prompt);=0A+=09=09=09have_password=20=3D=20true;=0A+=09=09=09= new_pass=20=3D=20true;=0A+=09=09}=0A+=09}=20while=20(new_pass);=0A+=0A+=09= if=20(!settings.db)=0A+=09{=0A+=09=09pg_log_error("no=20connection=20to=20= server");=0A+=09=09exit(EXIT_BADCONN);=0A+=09}=0A+=0A+=09if=20= (PQstatus(settings.db)=20=3D=3D=20CONNECTION_BAD)=0A+=09{=0A+=09=09= pg_log_error("could=20not=20connect=20to=20server:=20%s",=0A+=09=09=09=09= =09=20PQerrorMessage(settings.db));=0A+=09=09PQfinish(settings.db);=0A+=09= =09exit(EXIT_BADCONN);=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20Expand=20= schema,=20table,=20and=20index=20exclusion=20patterns,=20if=20any.=20=20= Note=20that=0A+=09=20*=20non-matching=20exclusion=20patterns=20are=20not=20= an=20error,=20even=20when=0A+=09=20*=20--strict-names=20was=20specified.=0A= +=09=20*/=0A+=09expand_schema_name_patterns(&schema_exclude_patterns,=20= NULL,=0A+=09=09=09=09=09=09=09=09&schema_exclude_oids,=20false);=0A+=09= expand_table_name_patterns(&table_exclude_patterns,=20NULL,=20NULL,=0A+=09= =09=09=09=09=09=09=20=20=20&table_exclude_oids,=20false);=0A+=09= expand_index_name_patterns(&index_exclude_patterns,=20NULL,=20NULL,=0A+=09= =09=09=09=09=09=09=20=20=20&index_exclude_oids,=20false);=0A+=0A+=09/*=20= Expand=20schema=20selection=20patterns=20into=20Oid=20lists=20*/=0A+=09= if=20(schema_include_patterns.head=20!=3D=20NULL)=0A+=09{=0A+=09=09= expand_schema_name_patterns(&schema_include_patterns,=0A+=09=09=09=09=09=09= =09=09=09&schema_exclude_oids,=0A+=09=09=09=09=09=09=09=09=09= &schema_include_oids,=0A+=09=09=09=09=09=09=09=09=09= settings.strict_names);=0A+=09=09if=20(schema_include_oids.head=20=3D=3D=20= NULL)=0A+=09=09=09fatal("no=20matching=20schemas=20were=20found");=0A+=09= }=0A+=0A+=09/*=20Expand=20table=20selection=20patterns=20into=20Oid=20= lists=20*/=0A+=09if=20(table_include_patterns.head=20!=3D=20NULL)=0A+=09= {=0A+=09=09expand_table_name_patterns(&table_include_patterns,=0A+=09=09=09= =09=09=09=09=09=20=20=20&schema_exclude_oids,=0A+=09=09=09=09=09=09=09=09= =20=20=20&table_exclude_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= &table_include_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= settings.strict_names);=0A+=09=09if=20(table_include_oids.head=20=3D=3D=20= NULL)=0A+=09=09=09fatal("no=20matching=20tables=20were=20found");=0A+=09= }=0A+=0A+=09/*=20Expand=20index=20selection=20patterns=20into=20Oid=20= lists=20*/=0A+=09if=20(index_include_patterns.head=20!=3D=20NULL)=0A+=09= {=0A+=09=09expand_index_name_patterns(&index_include_patterns,=0A+=09=09=09= =09=09=09=09=09=20=20=20&schema_exclude_oids,=0A+=09=09=09=09=09=09=09=09= =20=20=20&index_exclude_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= &index_include_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= settings.strict_names);=0A+=09=09if=20(index_include_oids.head=20=3D=3D=20= NULL)=0A+=09=09=09fatal("no=20matching=20indexes=20were=20found");=0A+=09= }=0A+=0A+=09/*=0A+=09=20*=20Compile=20list=20of=20all=20tables=20to=20be=20= checked=20based=20on=20namespace=20and=20table=0A+=09=20*=20includes=20= and=20excludes.=0A+=09=20*/=0A+=09= get_table_check_list(&schema_include_oids,=20&schema_exclude_oids,=0A+=09= =09=09=09=09=09=20&table_include_oids,=20&table_exclude_oids,=20= &checklist);=0A+=0A+=09PQsetNoticeProcessor(settings.db,=20= NoticeProcessor,=20NULL);=0A+=0A+=09/*=0A+=09=20*=20All=20information=20= about=20corrupt=20indexes=20are=20returned=20via=20ereport,=20not=20as=0A= +=09=20*=20tuples.=20=20We=20want=20all=20the=20details=20to=20report=20= if=20corruption=20exists.=0A+=09=20*/=0A+=09= PQsetErrorVerbosity(settings.db,=20PQERRORS_VERBOSE);=0A+=0A+=09= check_tables(&checklist);=0A+=0A+=09return=200;=0A+}=0A+=0A+/*=0A+=20*=20= Conditionally=20add=20a=20restriction=20to=20a=20query=20such=20that=20= lval=20must=20be=20an=20Oid=20in=0A+=20*=20the=20given=20list=20of=20= Oids,=20except=20that=20for=20a=20null=20or=20empty=20oids=20list=20= argument,=0A+=20*=20no=20filtering=20is=20done=20and=20we=20return=20= without=20having=20modified=20the=20query=20buffer.=0A+=20*=0A+=20*=20= The=20query=20argument=20must=20already=20have=20begun=20the=20WHERE=20= clause=20and=20must=20be=20in=20a=0A+=20*=20state=20where=20we=20can=20= append=20an=20AND=20clause.=20=20No=20checking=20of=20this=20requirement=20= is=0A+=20*=20done=20here.=0A+=20*=0A+=20*=20On=20return,=20the=20query=20= buffer=20will=20be=20extended=20with=20an=20AND=20clause=20that=20= filters=0A+=20*=20only=20those=20rows=20where=20the=20lval=20is=20an=20= Oid=20present=20in=20the=20given=20list=20of=20oids.=0A+=20*/=0A+static=20= inline=20void=0A+include_filter(PQExpBuffer=20querybuf,=20const=20char=20= *lval,=20const=20SimpleOidList=20*oids)=0A+{=0A+=09= apply_filter(querybuf,=20lval,=20oids,=20true);=0A+}=0A+=0A+/*=0A+=20*=20= Same=20as=20include_filter,=20above,=20except=20that=20for=20a=20= non-null,=20non-empty=20oids=0A+=20*=20list,=20the=20lval=20is=20= restricted=20to=20not=20be=20any=20of=20the=20values=20in=20the=20list.=0A= +=20*/=0A+static=20inline=20void=0A+exclude_filter(PQExpBuffer=20= querybuf,=20const=20char=20*lval,=20const=20SimpleOidList=20*oids)=0A+{=0A= +=09apply_filter(querybuf,=20lval,=20oids,=20false);=0A+}=0A+=0A+/*=0A+=20= *=20Check=20each=20table=20from=20the=20given=20checklist=20per=20the=20= user=20specified=20options.=0A+=20*/=0A+static=20void=0A= +check_tables(SimpleOidList=20*checklist)=0A+{=0A+=09const=20= SimpleOidListCell=20*cell;=0A+=0A+=09for=20(cell=20=3D=20= checklist->head;=20cell;=20cell=20=3D=20cell->next)=0A+=09{=0A+=09=09= uint64=09=09corruptions=20=3D=200;=0A+=09=09bool=09=09reconcile_toast;=0A= +=0A+=09=09/*=0A+=09=09=20*=20If=20we=20skip=20checking=20the=20toast=20= table,=20or=20if=20during=20the=20check=20we=0A+=09=09=20*=20detect=20= any=20toast=20table=20corruption,=20the=20main=20table=20checks=20below=20= must=0A+=09=09=20*=20not=20reconcile=20toasted=20attributes=20against=20= the=20toast=20table,=20as=20such=0A+=09=09=20*=20accesses=20to=20the=20= toast=20table=20might=20crash=20the=20backend.=20=20Instead,=20skip=0A+=09= =09=20*=20such=20reconciliations=20for=20this=20table.=0A+=09=09=20*=0A+=09= =09=20*=20This=20protection=20contains=20a=20race=20condition;=20the=20= toast=20table=20or=20index=0A+=09=09=20*=20could=20become=20corrupted=20= concurrently=20with=20our=20checks,=20but=20prevention=0A+=09=09=20*=20= of=20such=20concurrent=20corruption=20is=20documented=20as=20the=20= caller's=0A+=09=09=20*=20reponsibility,=20so=20we=20don't=20worry=20= about=20it=20here.=0A+=09=09=20*/=0A+=09=09reconcile_toast=20=3D=20= false;=0A+=09=09if=20(settings.check_toast)=0A+=09=09{=0A+=09=09=09if=20= (check_toast(cell->val)=20=3D=3D=200)=0A+=09=09=09=09reconcile_toast=20=3D= =20true;=0A+=09=09}=0A+=0A+=09=09corruptions=20=3D=20= check_table(cell->val,=0A+=09=09=09=09=09=09=09=09=20=20= settings.startblock,=0A+=09=09=09=09=09=09=09=09=20=20settings.endblock,=0A= +=09=09=09=09=09=09=09=09=20=20settings.on_error_stop,=0A+=09=09=09=09=09= =09=09=09=20=20reconcile_toast);=0A+=0A+=09=09if=20= (settings.check_indexes)=0A+=09=09{=0A+=09=09=09bool=09=09= old_heapallindexed;=0A+=0A+=09=09=09/*=20Optionally=20skip=20the=20index=20= checks=20for=20a=20corrupt=20table.=20*/=0A+=09=09=09if=20(corruptions=20= &&=20!settings.check_corrupt)=0A+=09=09=09=09continue;=0A+=0A+=09=09=09= /*=0A+=09=09=09=20*=20The=20btree=20checking=20logic=20which=20= optionally=20checks=20the=20contents=0A+=09=09=09=20*=20of=20an=20index=20= against=20the=20corresponding=20table=20has=20not=20yet=20been=0A+=09=09=09= =20*=20sufficiently=20hardened=20against=20corrupt=20tables.=20=20In=20= particular,=0A+=09=09=09=20*=20when=20called=20with=20heapallindexed=20= true,=20it=20segfaults=20if=20the=20file=0A+=09=09=09=20*=20backing=20= the=20table=20relation=20has=20been=20erroneously=20unlinked.=20=20In=0A= +=09=09=09=20*=20any=20event,=20it=20seems=20unwise=20to=20reconcile=20= an=20index=20against=20its=0A+=09=09=09=20*=20table=20when=20we=20= already=20know=20the=20table=20is=20corrupt.=0A+=09=09=09=20*/=0A+=09=09=09= old_heapallindexed=20=3D=20settings.heapallindexed;=0A+=09=09=09if=20= (corruptions)=0A+=09=09=09=09settings.heapallindexed=20=3D=20false;=0A+=0A= +=09=09=09corruptions=20+=3D=20check_indexes(cell->val,=0A+=09=09=09=09=09= =09=09=09=09=09=20&index_include_oids,=0A+=09=09=09=09=09=09=09=09=09=09=20= &index_exclude_oids);=0A+=0A+=09=09=09settings.heapallindexed=20=3D=20= old_heapallindexed;=0A+=09=09}=0A+=09}=0A+}=0A+=0A+/*=0A+=20*=20For=20a=20= given=20main=20table=20relation,=20returns=20the=20associated=20toast=20= table,=0A+=20*=20or=20InvalidOid=20if=20none=20exists.=0A+=20*/=0A= +static=20Oid=0A+get_toast_oid(Oid=20tbloid)=0A+{=0A+=09PQExpBuffer=20= querybuf=20=3D=20createPQExpBuffer();=0A+=09PGresult=20=20=20*res;=0A+=09= char=09=20=20=20*error=20=3D=20NULL;=0A+=09Oid=09=09=09result=20=3D=20= InvalidOid;=0A+=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09=20=20= "SELECT=20c.reltoastrelid"=0A+=09=09=09=09=09=20=20"\nFROM=20= pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=20=20"\nWHERE=20c.oid=20=3D=20= %u",=0A+=09=09=09=09=09=20=20tbloid);=0A+=09res=20=3D=20= ExecuteSqlQuery(querybuf->data,=20&error);=0A+=09if=20= (PQresultStatus(res)=20=3D=3D=20PGRES_TUPLES_OK=20&&=20PQntuples(res)=20= >=200)=0A+=09=09result=20=3D=20atooid(PQgetvalue(res,=200,=200));=0A+=09= else=20if=20(error)=0A+=09=09die_on_query_failure(querybuf->data);=0A+=0A= +=09PQclear(res);=0A+=09destroyPQExpBuffer(querybuf);=0A+=0A+=09return=20= result;=0A+}=0A+=0A+/*=0A+=20*=20For=20the=20given=20main=20table=20= relation,=20checks=20the=20associated=20toast=20table=20and=0A+=20*=20= index,=20in=20any.=20=20This=20should=20be=20performed=20*before*=20= checking=20the=20main=20table=0A+=20*=20relation,=20as=20the=20checks=20= inside=20verify_heapam=20assume=20both=20the=20toast=20table=20and=0A+=20= *=20toast=20index=20are=20usable.=0A+=20*=0A+=20*=20Returns=20the=20= number=20of=20corruptions=20detected.=0A+=20*/=0A+static=20uint64=0A= +check_toast(Oid=20tbloid)=0A+{=0A+=09Oid=09=09=09toastoid;=0A+=09uint64=09= =09corruption_cnt=20=3D=200;=0A+=0A+=09if=20(settings.db=20=3D=3D=20= NULL)=0A+=09=09fatal("no=20connection=20on=20entry=20to=20check_toast");=0A= +=0A+=09toastoid=20=3D=20get_toast_oid(tbloid);=0A+=09if=20= (OidIsValid(toastoid))=0A+=09{=0A+=09=09corruption_cnt=20=3D=20= check_table(toastoid,=20settings.toaststart,=0A+=09=09=09=09=09=09=09=09=09= =20settings.toastend,=20settings.on_error_stop,=0A+=09=09=09=09=09=09=09=09= =09=20false);=0A+=0A+=09=09/*=0A+=09=09=20*=20If=20the=20toast=20table=20= is=20corrupt,=20checking=20the=20index=20is=20not=20safe.=0A+=09=09=20*=20= There=20is=20a=20race=20condition=20here,=20as=20the=20toast=20table=20= could=20be=0A+=09=09=20*=20concurrently=20corrupted,=20but=20preventing=20= concurrent=20corruption=20is=20the=0A+=09=09=20*=20caller's=20= responsibility,=20not=20ours.=0A+=09=09=20*/=0A+=09=09if=20= (corruption_cnt=20=3D=3D=200)=0A+=09=09=09corruption_cnt=20+=3D=20= check_indexes(toastoid,=20NULL,=20NULL);=0A+=09}=0A+=0A+=09return=20= corruption_cnt;=0A+}=0A+=0A+/*=0A+=20*=20Checks=20the=20given=20table=20= for=20corruption,=20returning=20the=20number=20of=20corruptions=0A+=20*=20= detected=20and=20printed=20to=20the=20user.=0A+=20*/=0A+static=20uint64=0A= +check_table(Oid=20tbloid,=20const=20char=20*startblock,=20const=20char=20= *endblock,=0A+=09=09=09bool=20on_error_stop,=20bool=20check_toast)=0A+{=0A= +=09PQExpBuffer=20querybuf;=0A+=09PGresult=20=20=20*res;=0A+=09int=09=09=09= i;=0A+=09char=09=20=20=20*skip;=0A+=09char=09=20=20=20*toast;=0A+=09= const=20char=20*stop;=0A+=09char=09=20=20=20*error=20=3D=20NULL;=0A+=09= uint64=09=09corruption_cnt=20=3D=200;=0A+=0A+=09if=20(settings.db=20=3D=3D= =20NULL)=0A+=09=09fatal("no=20connection=20on=20entry=20to=20= check_table");=0A+=0A+=09if=20(startblock=20=3D=3D=20NULL)=0A+=09=09= startblock=20=3D=20"NULL";=0A+=09if=20(endblock=20=3D=3D=20NULL)=0A+=09=09= endblock=20=3D=20"NULL";=0A+=09if=20(settings.skip_frozen)=0A+=09=09skip=20= =3D=20pg_strdup("'all=20frozen'");=0A+=09else=20if=20= (settings.skip_visible)=0A+=09=09skip=20=3D=20pg_strdup("'all=20= visible'");=0A+=09else=0A+=09=09skip=20=3D=20pg_strdup("'none'");=0A+=09= stop=20=3D=20(on_error_stop)=20?=20"true"=20:=20"false";=0A+=09toast=20=3D= =20(check_toast)=20?=20"true"=20:=20"false";=0A+=0A+=09querybuf=20=3D=20= createPQExpBuffer();=0A+=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09= =09=20=20"SELECT=20c.relname,=20v.blkno,=20v.offnum,=20v.attnum,=20v.msg=20= "=0A+=09=09=09=09=09=20=20"FROM=20verify_heapam("=0A+=09=09=09=09=09=20=20= "relation=20:=3D=20%u,=20"=0A+=09=09=09=09=09=20=20"on_error_stop=20:=3D=20= %s,=20"=0A+=09=09=09=09=09=20=20"skip=20:=3D=20%s,=20"=0A+=09=09=09=09=09= =20=20"check_toast=20:=3D=20%s,=20"=0A+=09=09=09=09=09=20=20"startblock=20= :=3D=20%s,=20"=0A+=09=09=09=09=09=20=20"endblock=20:=3D=20%s)=20v,=20"=0A= +=09=09=09=09=09=20=20"pg_catalog.pg_class=20c=20"=0A+=09=09=09=09=09=20=20= "WHERE=20c.oid=20=3D=20%u",=0A+=09=09=09=09=09=20=20tbloid,=20stop,=20= skip,=20toast,=20startblock,=20endblock,=20tbloid);=0A+=0A+=09res=20=3D=20= ExecuteSqlQuery(querybuf->data,=20&error);=0A+=09if=20= (PQresultStatus(res)=20=3D=3D=20PGRES_TUPLES_OK=20&&=20PQntuples(res)=20= >=200)=0A+=09{=0A+=09=09corruption_cnt=20+=3D=20PQntuples(res);=0A+=09=09= for=20(i=20=3D=200;=20i=20<=20PQntuples(res);=20i++)=0A+=09=09{=0A+=09=09= =09printf("(relname=3D%s,blkno=3D%s,offnum=3D%s,attnum=3D%s)\n%s\n",=0A+=09= =09=09=09=20=20=20PQgetvalue(res,=20i,=200),=09/*=20relname=20*/=0A+=09=09= =09=09=20=20=20PQgetvalue(res,=20i,=201),=09/*=20blkno=20*/=0A+=09=09=09=09= =20=20=20PQgetvalue(res,=20i,=202),=09/*=20offnum=20*/=0A+=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=203),=09/*=20attnum=20*/=0A+=09=09=09=09=20=20= =20PQgetvalue(res,=20i,=204));=09/*=20msg=20*/=0A+=09=09}=0A+=09}=0A+=09= else=20if=20(error)=0A+=09{=0A+=09=09corruption_cnt++;=0A+=09=09= printf("%s\n",=20error);=0A+=09=09pfree(error);=0A+=09}=0A+=0A+=09= PQclear(res);=0A+=09destroyPQExpBuffer(querybuf);=0A+=09return=20= corruption_cnt;=0A+}=0A+=0A+static=20uint64=0A+check_indexes(Oid=20= tbloid,=20const=20SimpleOidList=20*include_oids,=0A+=09=09=09=20=20const=20= SimpleOidList=20*exclude_oids)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09= PGresult=20=20=20*res;=0A+=09int=09=09=09i;=0A+=09char=09=20=20=20*error=20= =3D=20NULL;=0A+=09uint64=09=09corruption_cnt=20=3D=200;=0A+=0A+=09if=20= (settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20= entry=20to=20check_indexes");=0A+=0A+=09querybuf=20=3D=20= createPQExpBuffer();=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09= =20=20"SELECT=20i.indexrelid,=20ci.relname,=20ct.relname"=0A+=09=09=09=09= =09=20=20"\nFROM=20pg_catalog.pg_index=20i,=20pg_catalog.pg_class=20ci,=20= "=0A+=09=09=09=09=09=20=20"pg_catalog.pg_class=20ct"=0A+=09=09=09=09=09=20= =20"\nWHERE=20i.indexrelid=20=3D=20ci.oid"=0A+=09=09=09=09=09=20=20"\n=20= =20=20=20AND=20i.indrelid=20=3D=20ct.oid"=0A+=09=09=09=09=09=20=20"\n=20=20= =20=20AND=20ci.relam=20=3D=20%u"=0A+=09=09=09=09=09=20=20"\n=20=20=20=20= AND=20i.indrelid=20=3D=20%u",=0A+=09=09=09=09=09=20=20BTREE_AM_OID,=20= tbloid);=0A+=09include_filter(querybuf,=20"i.indexrelid",=20= include_oids);=0A+=09exclude_filter(querybuf,=20"i.indexrelid",=20= exclude_oids);=0A+=0A+=09res=20=3D=20ExecuteSqlQuery(querybuf->data,=20= &error);=0A+=09if=20(PQresultStatus(res)=20=3D=3D=20PGRES_TUPLES_OK)=0A+=09= {=0A+=09=09for=20(i=20=3D=200;=20i=20<=20PQntuples(res);=20i++)=0A+=09=09= =09corruption_cnt=20+=3D=20check_index(PQgetvalue(res,=20i,=200),=0A+=09=09= =09=09=09=09=09=09=09=09=20=20PQgetvalue(res,=20i,=201),=0A+=09=09=09=09=09= =09=09=09=09=09=20=20PQgetvalue(res,=20i,=202));=0A+=09}=0A+=09else=20if=20= (error)=0A+=09{=0A+=09=09corruption_cnt++;=0A+=09=09printf("%s\n",=20= error);=0A+=09=09pfree(error);=0A+=09}=0A+=0A+=09PQclear(res);=0A+=09= destroyPQExpBuffer(querybuf);=0A+=0A+=09return=20corruption_cnt;=0A+}=0A= +=0A+static=20uint64=0A+check_index(const=20char=20*idxoid,=20const=20= char=20*idxname,=20const=20char=20*tblname)=0A+{=0A+=09PQExpBuffer=20= querybuf;=0A+=09PGresult=20=20=20*res;=0A+=09uint64=09=09corruption_cnt=20= =3D=200;=0A+=0A+=09if=20(settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20= connection=20on=20entry=20to=20check_index");=0A+=09if=20(idxname=20=3D=3D= =20NULL)=0A+=09=09fatal("no=20index=20name=20on=20entry=20to=20= check_index");=0A+=09if=20(tblname=20=3D=3D=20NULL)=0A+=09=09fatal("no=20= table=20name=20on=20entry=20to=20check_index");=0A+=0A+=09querybuf=20=3D=20= createPQExpBuffer();=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09= =20=20"SELECT=20bt_index_parent_check('%s'::regclass,=20%s,=20%s)",=0A+=09= =09=09=09=09=20=20idxoid,=0A+=09=09=09=09=09=20=20= settings.heapallindexed=20?=20"true"=20:=20"false",=0A+=09=09=09=09=09=20= =20settings.rootdescend=20?=20"true"=20:=20"false");=0A+=09res=20=3D=20= PQexec(settings.db,=20querybuf->data);=0A+=09if=20(PQresultStatus(res)=20= !=3D=20PGRES_TUPLES_OK)=0A+=09{=0A+=09=09corruption_cnt++;=0A+=09=09= printf("index=20check=20failed=20for=20index=20%s=20of=20table=20%s:\n",=0A= +=09=09=09=20=20=20idxname,=20tblname);=0A+=09=09printf("%s",=20= PQerrorMessage(settings.db));=0A+=09}=0A+=0A+=09PQclear(res);=0A+=09= destroyPQExpBuffer(querybuf);=0A+=0A+=09return=20corruption_cnt;=0A+}=0A= +=0A+static=20void=0A+parse_cli_options(int=20argc,=20char=20*argv[],=20= ConnectOptions=20*connOpts)=0A+{=0A+=09static=20struct=20option=20= long_options[]=20=3D=0A+=09{=0A+=09=09{"check-corrupt",=20no_argument,=20= NULL,=20'c'},=0A+=09=09{"check-indexes",=20no_argument,=20NULL,=20'x'},=0A= +=09=09{"check-toast",=20no_argument,=20NULL,=20'z'},=0A+=09=09= {"dbname",=20required_argument,=20NULL,=20'd'},=0A+=09=09{"endblock",=20= required_argument,=20NULL,=20'e'},=0A+=09=09{"exclude-index",=20= required_argument,=20NULL,=20'I'},=0A+=09=09{"exclude-schema",=20= required_argument,=20NULL,=20'N'},=0A+=09=09{"exclude-table",=20= required_argument,=20NULL,=20'T'},=0A+=09=09{"heapallindexed",=20= no_argument,=20NULL,=20'a'},=0A+=09=09{"help",=20optional_argument,=20= NULL,=20'?'},=0A+=09=09{"host",=20required_argument,=20NULL,=20'h'},=0A+=09= =09{"index",=20required_argument,=20NULL,=20'i'},=0A+=09=09= {"no-heapallindexed",=20no_argument,=20NULL,=20'A'},=0A+=09=09= {"no-password",=20no_argument,=20NULL,=20'w'},=0A+=09=09= {"no-rootdescend",=20no_argument,=20NULL,=20'R'},=0A+=09=09= {"on-error-stop",=20no_argument,=20NULL,=20'o'},=0A+=09=09{"password",=20= no_argument,=20NULL,=20'W'},=0A+=09=09{"port",=20required_argument,=20= NULL,=20'p'},=0A+=09=09{"rootdescend",=20no_argument,=20NULL,=20'r'},=0A= +=09=09{"schema",=20required_argument,=20NULL,=20'n'},=0A+=09=09= {"skip-all-frozen",=20no_argument,=20NULL,=20'f'},=0A+=09=09= {"skip-all-visible",=20no_argument,=20NULL,=20'v'},=0A+=09=09= {"skip-corrupt",=20no_argument,=20NULL,=20'C'},=0A+=09=09= {"skip-indexes",=20no_argument,=20NULL,=20'X'},=0A+=09=09{"skip-toast",=20= no_argument,=20NULL,=20'Z'},=0A+=09=09{"startblock",=20= required_argument,=20NULL,=20'b'},=0A+=09=09{"strict-names",=20= no_argument,=20NULL,=20's'},=0A+=09=09{"table",=20required_argument,=20= NULL,=20't'},=0A+=09=09{"toast-endblock",=20required_argument,=20NULL,=20= 'E'},=0A+=09=09{"toast-startblock",=20required_argument,=20NULL,=20'B'},=0A= +=09=09{"username",=20required_argument,=20NULL,=20'U'},=0A+=09=09= {"version",=20no_argument,=20NULL,=20'V'},=0A+=09=09{NULL,=200,=20NULL,=20= 0}=0A+=09};=0A+=0A+=09int=09=09=09optindex;=0A+=09int=09=09=09c;=0A+=0A+=09= memset(connOpts,=200,=20sizeof=20*connOpts);=0A+=0A+=09while=20((c=20=3D=20= getopt_long(argc,=20argv,=20= "aAb:B:cCd:e:E:fh:i:I:n:N:op:rRst:T:U:vVwWxXzZ?1",=0A+=09=09=09=09=09=09=09= long_options,=20&optindex))=20!=3D=20-1)=0A+=09{=0A+=09=09switch=20(c)=0A= +=09=09{=0A+=09=09=09case=20'a':=0A+=09=09=09=09settings.heapallindexed=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'A':=0A+=09=09=09=09= settings.heapallindexed=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'b':=0A+=09=09=09=09settings.startblock=20=3D=20= pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'B':=0A+=09=09= =09=09settings.toaststart=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09= break;=0A+=09=09=09case=20'c':=0A+=09=09=09=09settings.check_corrupt=20=3D= =20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'C':=0A+=09=09=09=09= settings.check_corrupt=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'd':=0A+=09=09=09=09connOpts->dbname=20=3D=20pg_strdup(optarg);=0A= +=09=09=09=09break;=0A+=09=09=09case=20'e':=0A+=09=09=09=09= settings.endblock=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09= =09=09case=20'E':=0A+=09=09=09=09settings.toastend=20=3D=20= pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'f':=0A+=09=09= =09=09settings.skip_frozen=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'h':=0A+=09=09=09=09connOpts->host=20=3D=20pg_strdup(optarg);=0A+=09= =09=09=09break;=0A+=09=09=09case=20'i':=0A+=09=09=09=09= simple_string_list_append(&index_include_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'I':=0A+=09=09=09=09= simple_string_list_append(&index_exclude_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'n':=09=09=09/*=20include=20schema(s)=20*/=0A= +=09=09=09=09simple_string_list_append(&schema_include_patterns,=20= optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'N':=09=09=09/*=20= exclude=20schema(s)=20*/=0A+=09=09=09=09= simple_string_list_append(&schema_exclude_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'o':=0A+=09=09=09=09settings.on_error_stop=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'p':=0A+=09=09=09=09= connOpts->port=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09= =09case=20's':=0A+=09=09=09=09settings.strict_names=20=3D=20true;=0A+=09=09= =09=09break;=0A+=09=09=09case=20'r':=0A+=09=09=09=09settings.rootdescend=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'R':=0A+=09=09=09=09= settings.rootdescend=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20't':=09=09=09/*=20include=20table(s)=20*/=0A+=09=09=09=09= simple_string_list_append(&table_include_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'T':=09=09=09/*=20exclude=20table(s)=20*/=0A= +=09=09=09=09simple_string_list_append(&table_exclude_patterns,=20= optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'U':=0A+=09=09=09=09= connOpts->username=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09= =09=09case=20'v':=0A+=09=09=09=09settings.skip_visible=20=3D=20true;=0A+=09= =09=09=09break;=0A+=09=09=09case=20'V':=0A+=09=09=09=09showVersion();=0A= +=09=09=09=09exit(EXIT_SUCCESS);=0A+=09=09=09case=20'w':=0A+=09=09=09=09= settings.getPassword=20=3D=20TRI_NO;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'W':=0A+=09=09=09=09settings.getPassword=20=3D=20TRI_YES;=0A+=09=09= =09=09break;=0A+=09=09=09case=20'x':=0A+=09=09=09=09= settings.check_indexes=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'X':=0A+=09=09=09=09settings.check_indexes=20=3D=20false;=0A+=09=09= =09=09break;=0A+=09=09=09case=20'z':=0A+=09=09=09=09settings.check_toast=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'Z':=0A+=09=09=09=09= settings.check_toast=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'?':=0A+=09=09=09=09if=20(optind=20<=3D=20argc=20&&=0A+=09=09=09=09= =09strcmp(argv[optind=20-=201],=20"-?")=20=3D=3D=200)=0A+=09=09=09=09{=0A= +=09=09=09=09=09/*=20actual=20help=20option=20given=20*/=0A+=09=09=09=09=09= usage();=0A+=09=09=09=09=09exit(EXIT_SUCCESS);=0A+=09=09=09=09}=0A+=09=09= =09=09else=0A+=09=09=09=09{=0A+=09=09=09=09=09/*=20getopt=20error=20= (unknown=20option=20or=20missing=20argument)=20*/=0A+=09=09=09=09=09goto=20= unknown_option;=0A+=09=09=09=09}=0A+=09=09=09=09break;=0A+=09=09=09case=20= 1:=0A+=09=09=09=09{=0A+=09=09=09=09=09if=20(!optarg=20||=20= strcmp(optarg,=20"options")=20=3D=3D=200)=0A+=09=09=09=09=09=09usage();=0A= +=09=09=09=09=09else=0A+=09=09=09=09=09=09goto=20unknown_option;=0A+=0A+=09= =09=09=09=09exit(EXIT_SUCCESS);=0A+=09=09=09=09}=0A+=09=09=09=09break;=0A= +=09=09=09default:=0A+=09=09unknown_option:=0A+=09=09=09=09= fprintf(stderr,=20"Try=20\"%s=20--help\"=20for=20more=20information.\n",=0A= +=09=09=09=09=09=09settings.progname);=0A+=09=09=09=09= exit(EXIT_FAILURE);=0A+=09=09=09=09break;=0A+=09=09}=0A+=09}=0A+=0A+=09= /*=0A+=09=20*=20if=20we=20still=20have=20arguments,=20use=20it=20as=20= the=20database=20name=20and=20username=0A+=09=20*/=0A+=09while=20(argc=20= -=20optind=20>=3D=201)=0A+=09{=0A+=09=09if=20(!connOpts->dbname)=0A+=09=09= =09connOpts->dbname=20=3D=20argv[optind];=0A+=09=09else=20if=20= (!connOpts->username)=0A+=09=09=09connOpts->username=20=3D=20= argv[optind];=0A+=09=09else=0A+=09=09=09pg_log_warning("extra=20= command-line=20argument=20\"%s\"=20ignored",=0A+=09=09=09=09=09=09=20=20=20= argv[optind]);=0A+=0A+=09=09optind++;=0A+=09}=0A+=0A+}=0A+=0A+/*=0A+=20*=20= usage=0A+=20*=0A+=20*=20print=20out=20command=20line=20arguments=0A+=20= */=0A+static=20void=0A+usage(void)=0A+{=0A+=09int=09=09=09lineno;=0A+=0A= +=09for=20(lineno=20=3D=200;=20usage_text[lineno];=20lineno++)=0A+=09=09= printf("%s\n",=20usage_text[lineno]);=0A+=09printf("Report=20bugs=20to=20= <%s>.\n",=20PACKAGE_BUGREPORT);=0A+=09printf("%s=20home=20page:=20= <%s>\n",=20PACKAGE_NAME,=20PACKAGE_URL);=0A+}=0A+=0A+static=20void=0A= +showVersion(void)=0A+{=0A+=09puts("pg_amcheck=20(PostgreSQL)=20"=20= PG_VERSION);=0A+}=0A+=0A+/*=0A+=20*=20for=20backend=20Notice=20messages=20= (INFO,=20WARNING,=20etc)=0A+=20*/=0A+static=20void=0A= +NoticeProcessor(void=20*arg,=20const=20char=20*message)=0A+{=0A+=09= (void)=20arg;=09=09=09=09=09/*=20not=20used=20*/=0A+=09pg_log_info("%s",=20= message);=0A+}=0A+=0A+/*=0A+=20*=20Helper=20function=20for=20= apply_filter,=20below.=0A+=20*/=0A+static=20void=0A= +append_csv_oids(PQExpBuffer=20querybuf,=20const=20SimpleOidList=20= *oids)=0A+{=0A+=09const=20SimpleOidListCell=20*cell;=0A+=09const=20char=20= *comma;=0A+=0A+=09for=20(comma=20=3D=20"",=20cell=20=3D=20oids->head;=20= cell;=20comma=20=3D=20",=20",=20cell=20=3D=20cell->next)=0A+=09=09= appendPQExpBuffer(querybuf,=20"%s%u",=20comma,=20cell->val);=0A+}=0A+=0A= +/*=0A+=20*=20Internal=20implementation=20of=20include_filter=20and=20= exclude_filter=0A+=20*/=0A+static=20void=0A+apply_filter(PQExpBuffer=20= querybuf,=20const=20char=20*lval,=20const=20SimpleOidList=20*oids,=0A+=09= =09=09=20bool=20include)=0A+{=0A+=09if=20(!oids=20||=20!oids->head)=0A+=09= =09return;=0A+=09if=20(include)=0A+=09=09appendPQExpBuffer(querybuf,=20= "\nAND=20%s=20OPERATOR(pg_catalog.=3D)=20ANY(array[",=20lval);=0A+=09= else=0A+=09=09appendPQExpBuffer(querybuf,=20"\nAND=20%s=20= OPERATOR(pg_catalog.!=3D)=20ALL(array[",=20lval);=0A+=09= append_csv_oids(querybuf,=20oids);=0A+=09appendPQExpBuffer(querybuf,=20= "]::OID[])");=0A+}=0A+=0A+/*=0A+=20*=20Find=20and=20append=20to=20the=20= given=20Oid=20list=20the=20Oids=20of=20all=20schemas=20matching=20the=0A= +=20*=20given=20list=20of=20patterns=20but=20not=20included=20in=20the=20= given=20list=20of=20excluded=20Oids.=0A+=20*/=0A+static=20void=0A= +expand_schema_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09= =09=09=09=09=09=09const=20SimpleOidList=20*exclude_nsp,=0A+=09=09=09=09=09= =09=09SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09bool=20= strict_names)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09PGresult=20=20=20= *res;=0A+=09SimpleStringListCell=20*cell;=0A+=09int=09=09=09i;=0A+=0A+=09= if=20(settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20= entry=20to=20expand_schema_name_patterns");=0A+=0A+=09if=20= (patterns->head=20=3D=3D=20NULL)=0A+=09=09return;=09=09=09=09=09/*=20= nothing=20to=20do=20*/=0A+=0A+=09querybuf=20=3D=20createPQExpBuffer();=0A= +=0A+=09/*=0A+=09=20*=20The=20loop=20below=20runs=20multiple=20SELECTs=20= might=20sometimes=20result=20in=0A+=09=20*=20duplicate=20entries=20in=20= the=20Oid=20list,=20but=20we=20don't=20care.=0A+=09=20*/=0A+=0A+=09for=20= (cell=20=3D=20patterns->head;=20cell;=20cell=20=3D=20cell->next)=0A+=09{=0A= +=09=09appendPQExpBufferStr(querybuf,=0A+=09=09=09=09=09=09=09=20"SELECT=20= oid=20FROM=20pg_catalog.pg_namespace=20n\n");=0A+=09=09= processSQLNamePattern(settings.db,=20querybuf,=20cell->val,=20false,=0A+=09= =09=09=09=09=09=09=20=20false,=20NULL,=20"n.nspname",=20NULL,=20NULL);=0A= +=09=09exclude_filter(querybuf,=20"n.oid",=20exclude_nsp);=0A+=0A+=09=09= res=20=3D=20ExecuteSqlQueryOrDie(querybuf->data);=0A+=09=09if=20= (strict_names=20&&=20PQntuples(res)=20=3D=3D=200)=0A+=09=09=09fatal("no=20= matching=20schemas=20were=20found=20for=20pattern=20\"%s\"",=0A+=09=09=09= =09=20=20cell->val);=0A+=0A+=09=09for=20(i=20=3D=200;=20i=20<=20= PQntuples(res);=20i++)=0A+=09=09{=0A+=09=09=09= simple_oid_list_append(oids,=20atooid(PQgetvalue(res,=20i,=200)));=0A+=09= =09}=0A+=0A+=09=09PQclear(res);=0A+=09=09resetPQExpBuffer(querybuf);=0A+=09= }=0A+=0A+=09destroyPQExpBuffer(querybuf);=0A+}=0A+=0A+/*=0A+=20*=20Find=20= and=20append=20to=20the=20given=20Oid=20list=20the=20Oids=20of=20all=20= relations=20matching=20the=0A+=20*=20given=20list=20of=20patterns=20but=20= not=20included=20in=20the=20given=20list=20of=20excluded=20Oids=0A+=20*=20= nor=20in=20one=20of=20the=20given=20excluded=20namespaces.=20=20The=20= relations=20are=20filtered=20by=0A+=20*=20the=20given=20schema_quals.=20=20= They=20are=20further=20filtered=20by=20the=20given=0A+=20*=20= relkind_quals,=20allowing=20the=20caller=20to=20restrict=20the=20= relations=20to=20just=20indexes=0A+=20*=20or=20tables.=20=20The=20= missing_errtext=20should=20be=20a=20message=20for=20use=20in=20error=0A+=20= *=20messages=20if=20no=20matching=20relations=20are=20found=20and=20= strict_names=20was=20specified.=0A+=20*/=0A+static=20void=0A= +expand_relkind_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09= =09=09=09=09=09=09=20const=20SimpleOidList=20*exclude_nsp_oids,=0A+=09=09= =09=09=09=09=09=20const=20SimpleOidList=20*exclude_oids,=0A+=09=09=09=09=09= =09=09=20SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09=20bool=20= strict_names,=0A+=09=09=09=09=09=09=09=20const=20char=20= *missing_errtext,=0A+=09=09=09=09=09=09=09=20const=20char=20= *relkind_quals)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09PGresult=20=20=20= *res;=0A+=09SimpleStringListCell=20*cell;=0A+=09int=09=09=09i;=0A+=0A+=09= if=20(settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20= entry=20to=20expand_relkind_name_patterns");=0A+=0A+=09if=20= (patterns->head=20=3D=3D=20NULL)=0A+=09=09return;=09=09=09=09=09/*=20= nothing=20to=20do=20*/=0A+=0A+=09querybuf=20=3D=20createPQExpBuffer();=0A= +=0A+=09/*=0A+=09=20*=20this=20might=20sometimes=20result=20in=20= duplicate=20entries=20in=20the=20Oid=20list,=20but=0A+=09=20*=20we=20= don't=20care.=0A+=09=20*/=0A+=0A+=09for=20(cell=20=3D=20patterns->head;=20= cell;=20cell=20=3D=20cell->next)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20= Query=20must=20remain=20ABSOLUTELY=20devoid=20of=20unqualified=20names.=20= =20This=0A+=09=09=20*=20would=20be=20unnecessary=20given=20a=20= pg_table_is_visible()=20variant=20taking=20a=0A+=09=09=20*=20search_path=20= argument.=0A+=09=09=20*/=0A+=09=09appendPQExpBuffer(querybuf,=0A+=09=09=09= =09=09=09=20=20"SELECT=20c.oid"=0A+=09=09=09=09=09=09=20=20"\nFROM=20= pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=09=20=20"\n=20=20=20=20LEFT=20= JOIN=20pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=09=20=20"\n=20=20=20= =20ON=20n.oid=20OPERATOR(pg_catalog.=3D)=20c.relnamespace"=0A+=09=09=09=09= =09=09=20=20"\nWHERE=20c.relkind=20OPERATOR(pg_catalog.=3D)=20%s\n",=0A+=09= =09=09=09=09=09=20=20relkind_quals);=0A+=09=09exclude_filter(querybuf,=20= "c.oid",=20exclude_oids);=0A+=09=09exclude_filter(querybuf,=20"n.oid",=20= exclude_nsp_oids);=0A+=09=09processSQLNamePattern(settings.db,=20= querybuf,=20cell->val,=20true,=0A+=09=09=09=09=09=09=09=20=20false,=20= "n.nspname",=20"c.relname",=20NULL,=20NULL);=0A+=09=09res=20=3D=20= ExecuteSqlQueryOrDie(querybuf->data);=0A+=09=09if=20(strict_names=20&&=20= PQntuples(res)=20=3D=3D=200)=0A+=09=09=09fatal("%s=20\"%s\"",=20= missing_errtext,=20cell->val);=0A+=0A+=09=09for=20(i=20=3D=200;=20i=20<=20= PQntuples(res);=20i++)=0A+=09=09=09simple_oid_list_append(oids,=20= atooid(PQgetvalue(res,=20i,=200)));=0A+=0A+=09=09PQclear(res);=0A+=09=09= resetPQExpBuffer(querybuf);=0A+=09}=0A+=0A+=09= destroyPQExpBuffer(querybuf);=0A+}=0A+=0A+/*=0A+=20*=20Find=20the=20Oids=20= of=20all=20tables=20matching=20the=20given=20list=20of=20patterns,=0A+=20= *=20and=20append=20them=20to=20the=20given=20Oid=20list.=0A+=20*/=0A= +static=20void=0A+expand_table_name_patterns(const=20SimpleStringList=20= *patterns,=20const=20SimpleOidList=20*exclude_nsp_oids,=0A+=09=09=09=09=09= =09=20=20=20const=20SimpleOidList=20*exclude_oids,=20SimpleOidList=20= *oids,=20bool=20strict_names)=0A+{=0A+=09= expand_relkind_name_patterns(patterns,=20exclude_nsp_oids,=20= exclude_oids,=20oids,=20strict_names,=0A+=09=09=09=09=09=09=09=09=20"no=20= matching=20tables=20were=20found=20for=20pattern",=0A+=09=09=09=09=09=09=09= =09=20get_table_relkind_quals());=0A+}=0A+=0A+/*=0A+=20*=20Find=20the=20= Oids=20of=20all=20indexes=20matching=20the=20given=20list=20of=20= patterns,=0A+=20*=20and=20append=20them=20to=20the=20given=20Oid=20list.=0A= +=20*/=0A+static=20void=0A+expand_index_name_patterns(const=20= SimpleStringList=20*patterns,=20const=20SimpleOidList=20= *exclude_nsp_oids,=0A+=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20= *exclude_oids,=20SimpleOidList=20*oids,=20bool=20strict_names)=0A+{=0A+=09= expand_relkind_name_patterns(patterns,=20exclude_nsp_oids,=20= exclude_oids,=20oids,=20strict_names,=0A+=09=09=09=09=09=09=09=09=20"no=20= matching=20indexes=20were=20found=20for=20pattern",=0A+=09=09=09=09=09=09= =09=09=20get_index_relkind_quals());=0A+}=0A+=0A+static=20void=0A= +get_table_check_list(const=20SimpleOidList=20*include_nsp,=20const=20= SimpleOidList=20*exclude_nsp,=0A+=09=09=09=09=09=20const=20SimpleOidList=20= *include_tbl,=20const=20SimpleOidList=20*exclude_tbl,=0A+=09=09=09=09=09=20= SimpleOidList=20*checklist)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09= PGresult=20=20=20*res;=0A+=09int=09=09=09i;=0A+=0A+=09if=20(settings.db=20= =3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20entry=20to=20= get_table_check_list");=0A+=0A+=09querybuf=20=3D=20createPQExpBuffer();=0A= +=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09=20=20"SELECT=20= c.oid"=0A+=09=09=09=09=09=20=20"\nFROM=20pg_catalog.pg_class=20c,=20= pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=20=20"\nWHERE=20n.oid=20= OPERATOR(pg_catalog.=3D)=20c.relnamespace"=0A+=09=09=09=09=09=20=20"\n=09= AND=20c.relkind=20OPERATOR(pg_catalog.=3D)=20%s\n",=0A+=09=09=09=09=09=20= =20get_table_relkind_quals());=0A+=09include_filter(querybuf,=20"n.oid",=20= include_nsp);=0A+=09exclude_filter(querybuf,=20"n.oid",=20exclude_nsp);=0A= +=09include_filter(querybuf,=20"c.oid",=20include_tbl);=0A+=09= exclude_filter(querybuf,=20"c.oid",=20exclude_tbl);=0A+=0A+=09res=20=3D=20= ExecuteSqlQueryOrDie(querybuf->data);=0A+=09for=20(i=20=3D=200;=20i=20<=20= PQntuples(res);=20i++)=0A+=09=09simple_oid_list_append(checklist,=20= atooid(PQgetvalue(res,=20i,=200)));=0A+=0A+=09PQclear(res);=0A+=09= destroyPQExpBuffer(querybuf);=0A+}=0A+=0A+static=20PGresult=20*=0A= +ExecuteSqlQueryOrDie(const=20char=20*query)=0A+{=0A+=09PGresult=20=20=20= *res;=0A+=0A+=09res=20=3D=20PQexec(settings.db,=20query);=0A+=09if=20= (PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09=09= die_on_query_failure(query);=0A+=09return=20res;=0A+}=0A+=0A+/*=0A+=20*=20= Execute=20the=20given=20SQL=20query.=20=20This=20function=20should=20= only=20be=20used=20for=20queries=0A+=20*=20which=20are=20not=20expected=20= to=20fail=20under=20normal=20circumstances,=20as=20failures=20will=0A+=20= *=20result=20in=20the=20printing=20of=20error=20messages,=20which=20will=20= look=20a=20bit=20messy=20when=0A+=20*=20interleaved=20with=20corruption=20= reports.=0A+=20*=0A+=20*=20On=20error,=20use=20the=20supplied=20= error_context=20string=20and=20the=20error=20string=0A+=20*=20returned=20= from=20the=20database=20connection=20to=20print=20an=20error=20message=20= for=20the=0A+=20*=20user.=0A+=20*=0A+=20*=20The=20error_context=20= argument=20is=20pfree'd=20by=20us=20at=20the=20end=20of=20the=20call.=0A= +=20*/=0A+static=20PGresult=20*=0A+ExecuteSqlQuery(const=20char=20= *query,=20char=20**error)=0A+{=0A+=09PGresult=20=20=20*res;=0A+=0A+=09= res=20=3D=20PQexec(settings.db,=20query);=0A+=09if=20= (PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09=09*error=20=3D=20= pstrdup(PQerrorMessage(settings.db));=0A+=09return=20res;=0A+}=0A+=0A+/*=0A= +=20*=20Return=20the=20cached=20relkind=20quals=20string=20for=20tables,=20= computing=20it=20first=20if=20we=0A+=20*=20don't=20have=20one=20cached.=0A= +=20*/=0A+static=20const=20char=20*=0A+get_table_relkind_quals(void)=0A= +{=0A+=09if=20(!table_relkind_quals)=0A+=09=09table_relkind_quals=20=3D=20= psprintf("ANY(array['%c',=20'%c',=20'%c'])",=0A+=09=09=09=09=09=09=09=09=09= =20=20=20RELKIND_RELATION,=20RELKIND_MATVIEW,=0A+=09=09=09=09=09=09=09=09= =09=20=20=20RELKIND_PARTITIONED_TABLE);=0A+=09return=20= table_relkind_quals;=0A+}=0A+=0A+/*=0A+=20*=20Return=20the=20cached=20= relkind=20quals=20string=20for=20indexes,=20computing=20it=20first=20if=20= we=0A+=20*=20don't=20have=20one=20cached.=0A+=20*/=0A+static=20const=20= char=20*=0A+get_index_relkind_quals(void)=0A+{=0A+=09if=20= (!index_relkind_quals)=0A+=09=09index_relkind_quals=20=3D=20= psprintf("'%c'",=20RELKIND_INDEX);=0A+=09return=20index_relkind_quals;=0A= +}=0Adiff=20--git=20a/contrib/pg_amcheck/pg_amcheck.control=20= b/contrib/pg_amcheck/pg_amcheck.control=0Anew=20file=20mode=20100644=0A= index=200000000000..395f368101=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/pg_amcheck.control=0A@@=20-0,0=20+1,5=20@@=0A+#=20= pg_amcheck=20extension=0A+comment=20=3D=20'command-line=20tool=20for=20= verifying=20relation=20integrity'=0A+default_version=20=3D=20'1.3'=0A= +module_pathname=20=3D=20'$libdir/pg_amcheck'=0A+relocatable=20=3D=20= true=0Adiff=20--git=20a/contrib/pg_amcheck/t/001_basic.pl=20= b/contrib/pg_amcheck/t/001_basic.pl=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..dfa0ae9e06=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/001_basic.pl=0A@@=20-0,0=20+1,9=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20TestLib;=0A+use=20Test::More=20= tests=20=3D>=208;=0A+=0A+program_help_ok('pg_amcheck');=0A= +program_version_ok('pg_amcheck');=0A= +program_options_handling_ok('pg_amcheck');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/002_nonesuch.pl=20= b/contrib/pg_amcheck/t/002_nonesuch.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..68be9c6585=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/002_nonesuch.pl=0A@@=20-0,0=20+1,60=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20TestLib;=0A= +use=20Test::More=20tests=20=3D>=2014;=0A+=0A+#=20Test=20set-up=0A+my=20= ($node,=20$port);=0A+$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+$port=20=3D=20$node->port;=0A+=0A+#=20Load=20the=20= amcheck=20extension,=20upon=20which=20pg_amcheck=20depends=0A= +$node->safe_psql('postgres',=20q(CREATE=20EXTENSION=20amcheck));=0A+=0A= +#########################################=0A+#=20Test=20connecting=20to=20= a=20non-existent=20database=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20"$port",=20'qqq'=20],=0A+=09qr/\Qpg_amcheck:=20= error:=20could=20not=20connect=20to=20server:=20FATAL:=20=20database=20= "qqq"=20does=20not=20exist\E/,=0A+=09'connecting=20to=20a=20non-existent=20= database');=0A+=0A+#########################################=0A+#=20Test=20= connecting=20with=20a=20non-existent=20user=0A+=0A+command_fails_like(=0A= +=09[=20'pg_amcheck',=20'-p',=20"$port",=20'-U=3Dno_such_user'=20],=0A+=09= qr/\Qpg_amcheck:=20error:=20could=20not=20connect=20to=20server:=20= FATAL:=20=20role=20"=3Dno_such_user"=20does=20not=20exist\E/,=0A+=09= 'connecting=20with=20a=20non-existent=20user');=0A+=0A= +#########################################=0A+#=20Test=20checking=20a=20= non-existent=20schema,=20table,=20and=20patterns=20with=20--strict-names=0A= +=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20"$port",=20= '-n',=20'nonexistent'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20no=20= matching=20schemas=20were=20found\E/,=0A+=09'checking=20a=20non-existent=20= schema');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20= "$port",=20'-t',=20'nonexistent'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20= no=20matching=20tables=20were=20found\E/,=0A+=09'checking=20a=20= non-existent=20table');=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20"$port",=20'--strict-names',=20'-n',=20= 'nonexistent*'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20no=20matching=20= schemas=20were=20found=20for=20pattern\E/,=0A+=09'no=20matching=20= schemas');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20= "$port",=20'--strict-names',=20'-t',=20'nonexistent*'=20],=0A+=09= qr/\Qpg_amcheck:=20error:=20no=20matching=20tables=20were=20found=20for=20= pattern\E/,=0A+=09'no=20matching=20tables');=0A+=0A+command_fails_like(=0A= +=09[=20'pg_amcheck',=20'-p',=20"$port",=20'--strict-names',=20'-i',=20= 'nonexistent*'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20no=20matching=20= indexes=20were=20found=20for=20pattern\E/,=0A+=09'no=20matching=20= indexes');=0Adiff=20--git=20a/contrib/pg_amcheck/t/003_check.pl=20= b/contrib/pg_amcheck/t/003_check.pl=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..4d8e61d871=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/003_check.pl=0A@@=20-0,0=20+1,231=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20TestLib;=0A= +use=20Test::More=20tests=20=3D>=2039;=0A+=0A+my=20($node,=20$port);=0A+=0A= +#=20Returns=20the=20filesystem=20path=20for=20the=20named=20relation.=0A= +#=0A+#=20Assumes=20the=20test=20node=20is=20running=0A+sub=20= relation_filepath($)=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A+=0A+=09my=20= $pgdata=20=3D=20$node->data_dir;=0A+=09my=20$rel=20=3D=20= $node->safe_psql('postgres',=0A+=09=09=09=09=09=09=09=20=20=20qq(SELECT=20= pg_relation_filepath('$relname')));=0A+=09die=20"path=20not=20found=20= for=20relation=20$relname"=20unless=20defined=20$rel;=0A+=09return=20= "$pgdata/$rel";=0A+}=0A+=0A+#=20Stops=20the=20test=20node,=20corrupts=20= the=20first=20page=20of=20the=20named=20relation,=20and=0A+#=20restarts=20= the=20node.=0A+#=0A+#=20Assumes=20the=20node=20is=20running.=0A+sub=20= corrupt_first_page($)=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A+=09my=20= $relpath=20=3D=20relation_filepath($relname);=0A+=0A+=09$node->stop;=0A+=09= my=20$fh;=0A+=09open($fh,=20'+<',=20$relpath);=0A+=09binmode=20$fh;=0A+=09= seek($fh,=2032,=200);=0A+=09syswrite($fh,=20'\x77\x77\x77\x77',=20500);=0A= +=09close($fh);=0A+=09$node->start;=0A+}=0A+=0A+#=20Stops=20the=20test=20= node,=20unlinks=20the=20file=20from=20the=20filesystem=20that=20backs=20= the=0A+#=20relation,=20and=20restarts=20the=20node.=0A+#=0A+#=20Assumes=20= the=20test=20node=20is=20running=0A+sub=20remove_relation_file($)=0A+{=0A= +=09my=20($relname)=20=3D=20@_;=0A+=09my=20$relpath=20=3D=20= relation_filepath($relname);=0A+=0A+=09$node->stop();=0A+=09= unlink($relpath);=0A+=09$node->start;=0A+}=0A+=0A+#=20Test=20set-up=0A= +$node=20=3D=20get_new_node('test');=0A+$node->init;=0A+$node->start;=0A= +$port=20=3D=20$node->port;=0A+=0A+#=20Load=20the=20amcheck=20extension,=20= upon=20which=20pg_amcheck=20depends=0A+$node->safe_psql('postgres',=20= q(CREATE=20EXTENSION=20amcheck));=0A+=0A+#=20Create=20schemas=20and=20= tables=20for=20checking=20pg_amcheck's=20include=0A+#=20and=20exclude=20= schema=20and=20table=20command=20line=20options=0A= +$node->safe_psql('postgres',=20q(=0A+--=20We'll=20corrupt=20all=20= indexes=20in=20s1=0A+CREATE=20SCHEMA=20s1;=0A+CREATE=20TABLE=20s1.t1=20= (a=20TEXT);=0A+CREATE=20TABLE=20s1.t2=20(a=20TEXT);=0A+CREATE=20INDEX=20= i1=20ON=20s1.t1(a);=0A+CREATE=20INDEX=20i2=20ON=20s1.t2(a);=0A+INSERT=20= INTO=20s1.t1=20(a)=20(SELECT=20gs::TEXT=20FROM=20= generate_series(1,10000)=20AS=20gs);=0A+INSERT=20INTO=20s1.t2=20(a)=20= (SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A+=0A= +--=20We'll=20corrupt=20all=20tables=20in=20s2=0A+CREATE=20SCHEMA=20s2;=0A= +CREATE=20TABLE=20s2.t1=20(a=20TEXT);=0A+CREATE=20TABLE=20s2.t2=20(a=20= TEXT);=0A+CREATE=20INDEX=20i1=20ON=20s2.t1(a);=0A+CREATE=20INDEX=20i2=20= ON=20s2.t2(a);=0A+INSERT=20INTO=20s2.t1=20(a)=20(SELECT=20gs::TEXT=20= FROM=20generate_series(1,10000)=20AS=20gs);=0A+INSERT=20INTO=20s2.t2=20= (a)=20(SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A= +=0A+--=20We'll=20corrupt=20all=20tables=20and=20indexes=20in=20s3=0A= +CREATE=20SCHEMA=20s3;=0A+CREATE=20TABLE=20s3.t1=20(a=20TEXT);=0A+CREATE=20= TABLE=20s3.t2=20(a=20TEXT);=0A+CREATE=20INDEX=20i1=20ON=20s3.t1(a);=0A= +CREATE=20INDEX=20i2=20ON=20s3.t2(a);=0A+INSERT=20INTO=20s3.t1=20(a)=20= (SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A= +INSERT=20INTO=20s3.t2=20(a)=20(SELECT=20gs::TEXT=20FROM=20= generate_series(1,10000)=20AS=20gs);=0A+=0A+--=20We'll=20leave=20= everything=20in=20s4=20uncorrupted=0A+CREATE=20SCHEMA=20s4;=0A+CREATE=20= TABLE=20s4.t1=20(a=20TEXT);=0A+CREATE=20TABLE=20s4.t2=20(a=20TEXT);=0A= +CREATE=20INDEX=20i1=20ON=20s4.t1(a);=0A+CREATE=20INDEX=20i2=20ON=20= s4.t2(a);=0A+INSERT=20INTO=20s4.t1=20(a)=20(SELECT=20gs::TEXT=20FROM=20= generate_series(1,10000)=20AS=20gs);=0A+INSERT=20INTO=20s4.t2=20(a)=20= (SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A+));=0A= +=0A+#=20Corrupt=20indexes=20in=20schema=20"s1"=0A= +remove_relation_file('s1.i1');=0A+corrupt_first_page('s1.i2');=0A+=0A+#=20= Corrupt=20tables=20in=20schema=20"s2"=0A+remove_relation_file('s2.t1');=0A= +corrupt_first_page('s2.t2');=0A+=0A+#=20Corrupt=20tables=20and=20= indexes=20in=20schema=20"s3"=0A+remove_relation_file('s3.i1');=0A= +corrupt_first_page('s3.i2');=0A+remove_relation_file('s3.t1');=0A= +corrupt_first_page('s3.t2');=0A+=0A+#=20Leave=20schema=20"s4"=20alone=0A= +=0A+=0A+#=20The=20pg_amcheck=20command=20itself=20should=20return=20a=20= success=20exit=20status,=20even=0A+#=20though=20tables=20and=20indexes=20= are=20corrupt.=20=20An=20error=20code=20returned=20would=20mean=20the=0A= +#=20pg_amcheck=20command=20itself=20failed,=20for=20example=20because=20= a=20connection=20to=20the=0A+#=20database=20could=20not=20be=20= established.=0A+#=0A+#=20For=20these=20checks,=20we're=20ignoring=20any=20= corruption=20reported=20and=20focusing=0A+#=20exclusively=20on=20the=20= exit=20code=20from=20pg_amcheck.=0A+#=0A+$node->command_ok(=0A+=09[=20= 'pg_amcheck',=20'-x',=20'-p',=20$port,=20'postgres'=20],=0A+=09= 'pg_amcheck=20all=20schemas=20and=20tables');=0A+=0A+$node->command_ok(=0A= +=09[=20'pg_amcheck',=20'-p',=20$port,=20'postgres'=20],=0A+=09= 'pg_amcheck=20all=20schemas,=20tables=20and=20indexes');=0A+=0A= +$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1'=20],=0A+=09'pg_amcheck=20all=20objects=20in=20= schema=20s1');=0A+=0A+$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-p',=20= $port,=20'postgres',=20'-n',=20's*',=20'-t',=20't1'=20],=0A+=09= 'pg_amcheck=20all=20tables=20named=20t1=20and=20their=20indexes');=0A+=0A= +$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20$port,=20= 'postgres',=20'-T',=20't1'=20],=0A+=09'pg_amcheck=20all=20tables=20not=20= named=20t1');=0A+=0A+$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-x',=20= '-p',=20$port,=20'postgres',=20'-N',=20's1',=20'-T',=20't1'=20],=0A+=09= 'pg_amcheck=20all=20tables=20not=20named=20t1=20nor=20in=20schema=20= s1');=0A+=0A+#=20Scans=20of=20indexes=20in=20s1=20should=20detect=20the=20= specific=20corruption=20that=20we=20created=0A+#=20above.=20=20For=20= missing=20relation=20forks,=20we=20know=20what=20the=20error=20message=20= looks=0A+#=20like.=20=20For=20corrupted=20index=20pages,=20the=20error=20= might=20vary=20depending=20on=20how=20the=0A+#=20page=20was=20formatted=20= on=20disk,=20including=20variations=20due=20to=20alignment=20differences=0A= +#=20between=20platforms,=20so=20we=20accept=20any=20non-empty=20error=20= message.=0A+#=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20= '-p',=20$port,=20'postgres',=20'-n',=20's1',=20'-i',=20'i1'=20],=0A+=09= qr/index=20"i1"=20lacks=20a=20main=20relation=20fork/,=0A+=09'pg_amcheck=20= index=20s1.i1=20reports=20missing=20main=20relation=20fork');=0A+=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1',=20'-i',=20'i2'=20],=0A+=09qr/.+/,=09=09=09#=20= Any=20non-empty=20error=20message=20is=20acceptable=0A+=09'pg_amcheck=20= index=20s1.s2=20reports=20index=20corruption');=0A+=0A+=0A+#=20In=20= schema=20s3,=20the=20tables=20and=20indexes=20are=20both=20corrupt.=20=20= Ordinarily,=20checking=0A+#=20of=20indexes=20will=20not=20be=20performed=20= for=20corrupt=20tables,=20but=20the=20--check-corrupt=0A+#=20option=20= (-c)=20forces=20the=20indexes=20to=20also=20be=20checked.=0A+#=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-c',=20'-p',=20= $port,=20'postgres',=20'-n',=20's3',=20'-i',=20'i1'=20],=0A+=09qr/index=20= "i1"=20lacks=20a=20main=20relation=20fork/,=0A+=09'pg_amcheck=20index=20= s3.i1=20reports=20missing=20main=20relation=20fork');=0A+=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-c',=20'-p',=20= $port,=20'postgres',=20'-n',=20's3',=20'-i',=20'i2'=20],=0A+=09qr/.+/,=09= =09=09#=20Any=20non-empty=20error=20message=20is=20acceptable=0A+=09= 'pg_amcheck=20index=20s3.s2=20reports=20index=20corruption');=0A+=0A+=0A= +#=20Check=20that=20'-x'=20and=20'-X'=20work=20as=20expected.=20=20Since=20= only=20index=20corruption=0A+#=20(and=20not=20table=20corruption)=20= exists=20in=20s1,=20'-X'=20should=20give=20no=20errors,=20and=0A+#=20= '-x'=20should=20give=20errors=20about=20index=20corruption.=0A+#=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1'=20],=0A+=09qr/.+/,=09=09=09#=20Any=20= non-empty=20error=20message=20is=20acceptable=0A+=09'pg_amcheck=20over=20= tables=20and=20indexes=20in=20schema=20s1=20reports=20corruption');=0A+=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-X',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1'=20],=0A+=09qr/^$/,=09=09=09#=20Empty=0A+=09= 'pg_amcheck=20over=20only=20tables=20in=20schema=20s1=20reports=20no=20= corruption');=0A+=0A+=0A+#=20Check=20that=20table=20corruption=20is=20= reported=20as=20expected,=20with=20or=20without=0A+#=20index=20checking=0A= +#=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20= $port,=20'postgres',=20'-n',=20's2'=20],=0A+=09qr/could=20not=20open=20= file/,=0A+=09'pg_amcheck=20over=20tables=20in=20schema=20s2=20reports=20= table=20corruption');=0A+=0A+$node->command_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'postgres',=20'-n',=20's2'=20],=0A+=09= qr/could=20not=20open=20file/,=0A+=09'pg_amcheck=20over=20tables=20and=20= indexes=20in=20schema=20s2=20reports=20table=20corruption');=0A+=0A+#=20= Check=20that=20no=20corruption=20is=20reported=20in=20schema=20s4=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's4'=20],=0A+=09qr/^$/,=09=09=09#=20Empty=0A+=09= 'pg_amcheck=20over=20schema=20s4=20reports=20no=20corruption');=0A+=0A+#=20= Check=20that=20no=20corruption=20is=20reported=20if=20we=20exclude=20= corrupt=20schemas=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20= '-p',=20$port,=20'postgres',=20'-N',=20's1',=20'-N',=20's2',=20'-N',=20= 's3'=20],=0A+=09qr/^$/,=09=09=09#=20Empty=0A+=09'pg_amcheck=20excluding=20= corrupt=20schemas=20reports=20no=20corruption');=0A+=0A+#=20Check=20that=20= no=20corruption=20is=20reported=20if=20we=20exclude=20corrupt=20tables=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-T',=20't1',=20'-T',=20't2'=20],=0A+=09qr/^$/,=09=09=09#=20= Empty=0A+=09'pg_amcheck=20excluding=20corrupt=20tables=20reports=20no=20= corruption');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/004_verify_heapam.pl=20= b/contrib/pg_amcheck/t/004_verify_heapam.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..1cc36b25b7=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/004_verify_heapam.pl=0A@@=20-0,0=20+1,489=20@@=0A= +use=20strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20= TestLib;=0A+=0A+use=20Test::More=20tests=20=3D>=2022;=0A+=0A+#=20This=20= regression=20test=20demonstrates=20that=20the=20pg_amcheck=20binary=20= supplied=20with=0A+#=20the=20pg_amcheck=20contrib=20module=20correctly=20= identifies=20specific=20kinds=20of=0A+#=20corruption=20within=20pages.=20= =20To=20test=20this,=20we=20need=20a=20mechanism=20to=20create=20corrupt=0A= +#=20pages=20with=20predictable,=20repeatable=20corruption.=20=20The=20= postgres=20backend=20cannot=0A+#=20be=20expected=20to=20help=20us=20with=20= this,=20as=20its=20design=20is=20not=20consistent=20with=20the=0A+#=20= goal=20of=20intentionally=20corrupting=20pages.=0A+#=0A+#=20Instead,=20= we=20create=20a=20table=20to=20corrupt,=20and=20with=20careful=20= consideration=20of=20how=0A+#=20postgresql=20lays=20out=20heap=20pages,=20= we=20seek=20to=20offsets=20within=20the=20page=20and=0A+#=20overwrite=20= deliberately=20chosen=20bytes=20with=20specific=20values=20calculated=20= to=0A+#=20corrupt=20the=20page=20in=20expected=20ways.=20=20We=20then=20= verify=20that=20pg_amcheck=20reports=0A+#=20the=20corruption,=20and=20= that=20it=20runs=20without=20crashing.=20=20Note=20that=20the=20backend=0A= +#=20cannot=20simply=20be=20started=20to=20run=20queries=20against=20the=20= corrupt=20table,=20as=20the=0A+#=20backend=20will=20crash,=20at=20least=20= for=20some=20of=20the=20corruption=20types=20we=20generate.=0A+#=0A+#=20= Autovacuum=20potentially=20touching=20the=20table=20in=20the=20= background=20makes=20the=20exact=0A+#=20behavior=20of=20this=20test=20= harder=20to=20reason=20about.=20=20We=20turn=20it=20off=20to=20keep=20= things=0A+#=20simpler.=20=20We=20use=20a=20"belt=20and=20suspenders"=20= approach,=20turning=20it=20off=20for=20the=0A+#=20system=20generally=20= in=20postgresql.conf,=20and=20turning=20it=20off=20specifically=20for=20= the=0A+#=20test=20table.=0A+#=0A+#=20This=20test=20depends=20on=20the=20= table=20being=20written=20to=20the=20heap=20file=20exactly=20as=20we=0A= +#=20expect=20it=20to=20be,=20so=20we=20take=20care=20to=20arrange=20the=20= columns=20of=20the=20table,=20and=0A+#=20insert=20rows=20of=20the=20= table,=20that=20give=20predictable=20sizes=20and=20locations=20within=0A= +#=20the=20table=20page.=0A+#=0A+#=20The=20HeapTupleHeaderData=20has=20= 23=20bytes=20of=20fixed=20size=20fields=20before=20the=20variable=0A+#=20= length=20t_bits[]=20array.=20=20We=20have=20exactly=203=20columns=20in=20= the=20table,=20so=20natts=20=3D=203,=0A+#=20t_bits=20is=201=20byte=20= long,=20and=20t_hoff=20=3D=20MAXALIGN(23=20+=201)=20=3D=2024.=0A+#=0A+#=20= We're=20not=20too=20fussy=20about=20which=20datatypes=20we=20use=20for=20= the=20test,=20but=20we=20do=20care=0A+#=20about=20some=20specific=20= properties.=20=20We'd=20like=20to=20test=20both=20fixed=20size=20and=0A= +#=20varlena=20types.=20=20We'd=20like=20some=20varlena=20data=20inline=20= and=20some=20toasted.=20=20And=0A+#=20we'd=20like=20the=20layout=20of=20= the=20table=20such=20that=20the=20datums=20land=20at=20predictable=0A+#=20= offsets=20within=20the=20tuple.=20=20We=20choose=20a=20structure=20= without=20padding=20on=20all=0A+#=20supported=20architectures:=0A+#=0A+#=20= =09a=20BIGINT=0A+#=09b=20TEXT=0A+#=09c=20TEXT=0A+#=0A+#=20We=20always=20= insert=20a=207-ascii=20character=20string=20into=20field=20'b',=20which=20= with=20a=0A+#=201-byte=20varlena=20header=20gives=20an=208=20byte=20= inline=20value.=20=20We=20always=20insert=20a=20long=0A+#=20text=20= string=20in=20field=20'c',=20long=20enough=20to=20force=20toast=20= storage.=0A+#=0A+#=20We=20choose=20to=20read=20and=20write=20binary=20= copies=20of=20our=20table's=20tuples,=20using=20perl's=0A+#=20pack()=20= and=20unpack()=20functions.=20=20Perl=20uses=20a=20packing=20code=20= system=20in=20which:=0A+#=0A+#=09L=20=3D=20"Unsigned=2032-bit=20Long",=0A= +#=09S=20=3D=20"Unsigned=2016-bit=20Short",=0A+#=09C=20=3D=20"Unsigned=20= 8-bit=20Octet",=0A+#=09c=20=3D=20"signed=208-bit=20octet",=0A+#=09q=20=3D=20= "signed=2064-bit=20quadword"=0A+#=0A+#=20Each=20tuple=20in=20our=20table=20= has=20a=20layout=20as=20follows:=0A+#=0A+#=20=20=20=20xx=20xx=20xx=20xx=20= =20=20=20=20=20=20=20=20=20=20=20t_xmin:=20xxxx=09=09offset=20=3D=200=09=09= L=0A+#=20=20=20=20xx=20xx=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20= t_xmax:=20xxxx=09=09offset=20=3D=204=09=09L=0A+#=20=20=20=20xx=20xx=20xx=20= xx=20=20=20=20=20=20=20=20=20=20t_field3:=20xxxx=09=09offset=20=3D=208=09= =09L=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20bi_hi:=20xx=09=09=09offset=20=3D=2012=09=09S=0A+#=20=20=20=20= xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20bi_lo:=20= xx=09=09=09offset=20=3D=2014=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20ip_posid:=20xx=09=09=09offset=20=3D=20= 16=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20= t_infomask2:=20xx=09=09=09offset=20=3D=2018=09=09S=0A+#=20=20=20=20xx=20= xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20t_infomask:=20xx=09=09=09= offset=20=3D=2020=09=09S=0A+#=20=20=20=20xx=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20t_hoff:=20x=09=09=09offset=20=3D=2022=09= =09C=0A+#=20=20=20=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20t_bits:=20x=09=09=09offset=20=3D=2023=09=09C=0A+#=20=20=20=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'a':=20xxxxxxxx=09offset=20= =3D=2024=09=09q=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20= =20'b':=20xxxxxxxx=09offset=20=3D=2032=09=09Cccccccc=0A+#=20=20=20=20xx=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'c':=20xxxxxxxx=09offset=20=3D=20= 40=09=09SSSS=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20= =20=20=20:=20xxxxxxxx=09=20...continued=09SSSS=0A+#=20=20=20=20xx=20xx=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20:=20= xx=20=20=20=20=20=20=09=20...continued=09S=0A+#=0A+#=20We=20could=20= choose=20to=20read=20and=20write=20columns=20'b'=20and=20'c'=20in=20= other=20ways,=20but=0A+#=20it=20is=20convenient=20enough=20to=20do=20it=20= this=20way.=20=20We=20define=20packing=20code=0A+#=20constants=20here,=20= where=20they=20can=20be=20compared=20easily=20against=20the=20layout.=0A= +=0A+use=20constant=20HEAPTUPLE_PACK_CODE=20=3D>=20= 'LLLSSSSSCCqCcccccccSSSSSSSSS';=0A+use=20constant=20= HEAPTUPLE_PACK_LENGTH=20=3D>=2058;=20=20=20=20=20#=20Total=20size=0A+=0A= +#=20Read=20a=20tuple=20of=20our=20table=20from=20a=20heap=20page.=0A+#=0A= +#=20Takes=20an=20open=20filehandle=20to=20the=20heap=20file,=20and=20= the=20offset=20of=20the=20tuple.=0A+#=0A+#=20Rather=20than=20returning=20= the=20binary=20data=20from=20the=20file,=20unpacks=20the=20data=20into=20= a=0A+#=20perl=20hash=20with=20named=20fields.=20=20These=20fields=20= exactly=20match=20the=20ones=20understood=0A+#=20by=20write_tuple(),=20= below.=20=20Returns=20a=20reference=20to=20this=20hash.=0A+#=0A+sub=20= read_tuple=20($$)=0A+{=0A+=09my=20($fh,=20$offset)=20=3D=20@_;=0A+=09my=20= ($buffer,=20%tup);=0A+=09seek($fh,=20$offset,=200);=0A+=09sysread($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=0A+=09@_=20=3D=20= unpack(HEAPTUPLE_PACK_CODE,=20$buffer);=0A+=09%tup=20=3D=20(t_xmin=20=3D>=20= shift,=0A+=09=09=09t_xmax=20=3D>=20shift,=0A+=09=09=09t_field3=20=3D>=20= shift,=0A+=09=09=09bi_hi=20=3D>=20shift,=0A+=09=09=09bi_lo=20=3D>=20= shift,=0A+=09=09=09ip_posid=20=3D>=20shift,=0A+=09=09=09t_infomask2=20=3D>= =20shift,=0A+=09=09=09t_infomask=20=3D>=20shift,=0A+=09=09=09t_hoff=20=3D>= =20shift,=0A+=09=09=09t_bits=20=3D>=20shift,=0A+=09=09=09a=20=3D>=20= shift,=0A+=09=09=09b_header=20=3D>=20shift,=0A+=09=09=09b_body1=20=3D>=20= shift,=0A+=09=09=09b_body2=20=3D>=20shift,=0A+=09=09=09b_body3=20=3D>=20= shift,=0A+=09=09=09b_body4=20=3D>=20shift,=0A+=09=09=09b_body5=20=3D>=20= shift,=0A+=09=09=09b_body6=20=3D>=20shift,=0A+=09=09=09b_body7=20=3D>=20= shift,=0A+=09=09=09c1=20=3D>=20shift,=0A+=09=09=09c2=20=3D>=20shift,=0A+=09= =09=09c3=20=3D>=20shift,=0A+=09=09=09c4=20=3D>=20shift,=0A+=09=09=09c5=20= =3D>=20shift,=0A+=09=09=09c6=20=3D>=20shift,=0A+=09=09=09c7=20=3D>=20= shift,=0A+=09=09=09c8=20=3D>=20shift,=0A+=09=09=09c9=20=3D>=20shift);=0A= +=09#=20Stitch=20together=20the=20text=20for=20column=20'b'=0A+=09= $tup{b}=20=3D=20join('',=20map=20{=20chr($tup{"b_body$_"})=20}=20= (1..7));=0A+=09return=20\%tup;=0A+}=0A+=0A+#=20Write=20a=20tuple=20of=20= our=20table=20to=20a=20heap=20page.=0A+#=0A+#=20Takes=20an=20open=20= filehandle=20to=20the=20heap=20file,=20the=20offset=20of=20the=20tuple,=20= and=20a=0A+#=20reference=20to=20a=20hash=20with=20the=20tuple=20values,=20= as=20returned=20by=20read_tuple().=0A+#=20Writes=20the=20tuple=20fields=20= from=20the=20hash=20into=20the=20heap=20file.=0A+#=0A+#=20The=20purpose=20= of=20this=20function=20is=20to=20write=20a=20tuple=20back=20to=20disk=20= with=20some=0A+#=20subset=20of=20fields=20modified.=20=20The=20function=20= does=20no=20error=20checking.=20=20Use=0A+#=20cautiously.=0A+#=0A+sub=20= write_tuple($$$)=0A+{=0A+=09my=20($fh,=20$offset,=20$tup)=20=3D=20@_;=0A= +=09my=20$buffer=20=3D=20pack(HEAPTUPLE_PACK_CODE,=0A+=09=09=09=09=09= $tup->{t_xmin},=0A+=09=09=09=09=09$tup->{t_xmax},=0A+=09=09=09=09=09= $tup->{t_field3},=0A+=09=09=09=09=09$tup->{bi_hi},=0A+=09=09=09=09=09= $tup->{bi_lo},=0A+=09=09=09=09=09$tup->{ip_posid},=0A+=09=09=09=09=09= $tup->{t_infomask2},=0A+=09=09=09=09=09$tup->{t_infomask},=0A+=09=09=09=09= =09$tup->{t_hoff},=0A+=09=09=09=09=09$tup->{t_bits},=0A+=09=09=09=09=09= $tup->{a},=0A+=09=09=09=09=09$tup->{b_header},=0A+=09=09=09=09=09= $tup->{b_body1},=0A+=09=09=09=09=09$tup->{b_body2},=0A+=09=09=09=09=09= $tup->{b_body3},=0A+=09=09=09=09=09$tup->{b_body4},=0A+=09=09=09=09=09= $tup->{b_body5},=0A+=09=09=09=09=09$tup->{b_body6},=0A+=09=09=09=09=09= $tup->{b_body7},=0A+=09=09=09=09=09$tup->{c1},=0A+=09=09=09=09=09= $tup->{c2},=0A+=09=09=09=09=09$tup->{c3},=0A+=09=09=09=09=09$tup->{c4},=0A= +=09=09=09=09=09$tup->{c5},=0A+=09=09=09=09=09$tup->{c6},=0A+=09=09=09=09= =09$tup->{c7},=0A+=09=09=09=09=09$tup->{c8},=0A+=09=09=09=09=09= $tup->{c9});=0A+=09seek($fh,=20$offset,=200);=0A+=09syswrite($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=09return;=0A+}=0A+=0A+#=20Set=20= umask=20so=20test=20directories=20and=20files=20are=20created=20with=20= default=20permissions=0A+umask(0077);=0A+=0A+#=20Set=20up=20the=20node.=20= =20Once=20we=20create=20and=20corrupt=20the=20table,=0A+#=20autovacuum=20= workers=20visiting=20the=20table=20could=20crash=20the=20backend.=0A+#=20= Disable=20autovacuum=20so=20that=20won't=20happen.=0A+my=20$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A= +$node->append_conf('postgresql.conf',=20'autovacuum=3Doff');=0A+=0A+#=20= Start=20the=20node=20and=20load=20the=20extensions.=20=20We=20depend=20= on=20both=0A+#=20amcheck=20and=20pageinspect=20for=20this=20test.=0A= +$node->start;=0A+my=20$port=20=3D=20$node->port;=0A+my=20$pgdata=20=3D=20= $node->data_dir;=0A+$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20= amcheck");=0A+$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20= pageinspect");=0A+=0A+#=20Get=20a=20non-zero=20datfrozenxid=0A= +$node->safe_psql('postgres',=20qq(VACUUM=20FREEZE));=0A+=0A+#=20Create=20= the=20test=20table=20with=20precisely=20the=20schema=20that=20our=20= corruption=20function=0A+#=20expects.=0A+$node->safe_psql(=0A+=09= 'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20public.test=20(a=20BIGINT,=20= b=20TEXT,=20c=20TEXT);=0A+=09=09ALTER=20TABLE=20public.test=20SET=20= (autovacuum_enabled=3Dfalse);=0A+=09=09ALTER=20TABLE=20public.test=20= ALTER=20COLUMN=20c=20SET=20STORAGE=20EXTERNAL;=0A+=09=09CREATE=20INDEX=20= test_idx=20ON=20public.test(a,=20b);=0A+=09));=0A+=0A+#=20We=20want=20(0=20= <=20datfrozenxid=20<=20test.relfrozenxid).=20=20To=20achieve=20this,=20= we=20freeze=0A+#=20an=20otherwise=20unused=20table,=20public.junk,=20= prior=20to=20inserting=20data=20and=20freezing=0A+#=20public.test=0A= +$node->safe_psql(=0A+=09'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20= public.junk=20AS=20SELECT=20'junk'::TEXT=20AS=20junk_column;=0A+=09=09= ALTER=20TABLE=20public.junk=20SET=20(autovacuum_enabled=3Dfalse);=0A+=09=09= VACUUM=20FREEZE=20public.junk=0A+=09));=0A+=0A+my=20$rel=20=3D=20= $node->safe_psql('postgres',=20qq(SELECT=20= pg_relation_filepath('public.test')));=0A+my=20$relpath=20=3D=20= "$pgdata/$rel";=0A+=0A+#=20Insert=20data=20and=20freeze=20public.test=0A= +use=20constant=20ROWCOUNT=20=3D>=2016;=0A+$node->safe_psql('postgres',=20= qq(=0A+=09INSERT=20INTO=20public.test=20(a,=20b,=20c)=0A+=09=09VALUES=20= (=0A+=09=09=0912345678,=0A+=09=09=09'abcdefg',=0A+=09=09=09repeat('w',=20= 10000)=0A+=09=09);=0A+=09VACUUM=20FREEZE=20public.test=0A+=09))=20for=20= (1..ROWCOUNT);=0A+=0A+my=20$relfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A+=09q(select=20relfrozenxid=20from=20= pg_class=20where=20relname=20=3D=20'test'));=0A+my=20$datfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A+=09q(select=20datfrozenxid=20from=20= pg_database=20where=20datname=20=3D=20'postgres'));=0A+=0A+#=20Find=20= where=20each=20of=20the=20tuples=20is=20located=20on=20the=20page.=0A+my=20= @lp_off;=0A+for=20my=20$tup=20(0..ROWCOUNT-1)=0A+{=0A+=09push=20= (@lp_off,=20$node->safe_psql('postgres',=20qq(=0A+select=20lp_off=20from=20= heap_page_items(get_raw_page('test',=20'main',=200))=0A+=09offset=20$tup=20= limit=201)));=0A+}=0A+=0A+#=20Check=20that=20pg_amcheck=20runs=20against=20= the=20uncorrupted=20table=20without=20error.=0A= +$node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A+=09=09= =09=09=20=20'pg_amcheck=20test=20table,=20prior=20to=20corruption');=0A+=0A= +#=20Check=20that=20pg_amcheck=20runs=20against=20the=20uncorrupted=20= table=20and=20index=20without=20error.=0A= +$node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A+=09=09= =09=09=20=20'pg_amcheck=20test=20table=20and=20index,=20prior=20to=20= corruption');=0A+=0A+$node->stop;=0A+=0A+#=20Sanity=20check=20that=20our=20= 'test'=20table=20has=20a=20relfrozenxid=20newer=20than=20the=0A+#=20= datfrozenxid=20for=20the=20database,=20and=20that=20the=20datfrozenxid=20= is=20greater=20than=20the=0A+#=20first=20normal=20xid.=20=20We=20rely=20= on=20these=20invariants=20in=20some=20of=20our=20tests.=0A+if=20= ($datfrozenxid=20<=3D=203=20||=20$datfrozenxid=20>=3D=20$relfrozenxid)=0A= +{=0A+=09fail('Xid=20thresholds=20not=20as=20expected');=0A+=09= $node->clean_node;=0A+=09exit;=0A+}=0A+=0A+#=20Some=20#define=20= constants=20from=20access/htup_details.h=20for=20use=20while=20= corrupting.=0A+use=20constant=20HEAP_HASNULL=20=20=20=20=20=20=20=20=20=20= =20=20=3D>=200x0001;=0A+use=20constant=20HEAP_XMAX_LOCK_ONLY=20=20=20=20=20= =3D>=200x0080;=0A+use=20constant=20HEAP_XMIN_COMMITTED=20=20=20=20=20=3D>=20= 0x0100;=0A+use=20constant=20HEAP_XMIN_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0200;=0A+use=20constant=20HEAP_XMAX_COMMITTED=20=20=20=20=20=3D>=20= 0x0400;=0A+use=20constant=20HEAP_XMAX_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0800;=0A+use=20constant=20HEAP_NATTS_MASK=20=20=20=20=20=20=20=20=20=3D>= =200x07FF;=0A+use=20constant=20HEAP_XMAX_IS_MULTI=20=20=20=20=20=20=3D>=20= 0x1000;=0A+use=20constant=20HEAP_KEYS_UPDATED=20=20=20=20=20=20=20=3D>=20= 0x2000;=0A+=0A+#=20Helper=20functions=0A+sub=20header=0A+{=0A+=09my=20= ($blkno,=20$offnum,=20$attnum)=20=3D=20@_;=0A+=09= qr/\(relname=3Dtest,blkno=3D$blkno,offnum=3D$offnum,attnum=3D$attnum\)\s+/= ms;=0A+}=0A+=0A+#=20Corrupt=20the=20tuples,=20one=20type=20of=20= corruption=20per=20tuple.=20=20Some=20types=20of=0A+#=20corruption=20= cause=20verify_heapam=20to=20skip=20to=20the=20next=20tuple=20without=0A= +#=20performing=20any=20remaining=20checks,=20so=20we=20can't=20exercise=20= the=20system=20properly=20if=0A+#=20we=20focus=20all=20our=20corruption=20= on=20a=20single=20tuple.=0A+#=0A+my=20@expected;=0A+my=20$file;=0A= +open($file,=20'+<',=20$relpath);=0A+binmode=20$file;=0A+=0A+for=20(my=20= $tupidx=20=3D=200;=20$tupidx=20<=20ROWCOUNT;=20$tupidx++)=0A+{=0A+=09my=20= $offnum=20=3D=20$tupidx=20+=201;=20=20#=20offnum=20is=201-based,=20not=20= zero-based=0A+=09my=20$offset=20=3D=20$lp_off[$tupidx];=0A+=09my=20$tup=20= =3D=20read_tuple($file,=20$offset);=0A+=0A+=09#=20Sanity-check=20that=20= the=20data=20appears=20on=20the=20page=20where=20we=20expect.=0A+=09if=20= ($tup->{a}=20ne=20'12345678'=20||=20$tup->{b}=20ne=20'abcdefg')=0A+=09{=0A= +=09=09fail('Page=20layout=20differs=20from=20our=20expectations');=0A+=09= =09$node->clean_node;=0A+=09=09exit;=0A+=09}=0A+=0A+=09my=20$header=20=3D=20= header(0,=20$offnum,=20'');=0A+=09if=20($offnum=20=3D=3D=201)=0A+=09{=0A= +=09=09#=20Corruptly=20set=20xmin=20<=20relfrozenxid=0A+=09=09my=20$xmin=20= =3D=20$relfrozenxid=20-=201;=0A+=09=09$tup->{t_xmin}=20=3D=20$xmin;=0A+=09= =09$tup->{t_infomask}=20&=3D=20~HEAP_XMIN_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20&=3D=20~HEAP_XMIN_INVALID;=0A+=0A+=09=09#=20= Expected=20corruption=20report=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}xmin=20$xmin=20precedes=20relation=20freeze=20threshold=20= 0:\d+/;=0A+=09}=0A+=09if=20($offnum=20=3D=3D=202)=0A+=09{=0A+=09=09#=20= Corruptly=20set=20xmin=20<=20datfrozenxid=0A+=09=09my=20$xmin=20=3D=203;=0A= +=09=09$tup->{t_xmin}=20=3D=20$xmin;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmin=20$xmin=20precedes=20oldest=20valid=20transaction=20ID=20= 0:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=203)=0A+=09{=0A+=09=09#=20= Corruptly=20set=20xmin=20<=20datfrozenxid,=20further=20back,=20noting=20= circularity=0A+=09=09#=20of=20xid=20comparison.=20=20For=20a=20new=20= cluster=20with=20epoch=20=3D=200,=20the=20corrupt=0A+=09=09#=20xmin=20= will=20be=20interpreted=20as=20in=20the=20future=0A+=09=09$tup->{t_xmin}=20= =3D=204026531839;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmin=204026531839=20equals=20or=20exceeds=20next=20valid=20= transaction=20ID=200:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=204)=0A= +=09{=0A+=09=09#=20Corruptly=20set=20xmax=20<=20relminmxid;=0A+=09=09= $tup->{t_xmax}=20=3D=204026531839;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMAX_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmax=204026531839=20equals=20or=20exceeds=20next=20valid=20= transaction=20ID=200:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=205)=0A= +=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20but=20keep=20it=20= aligned=20properly=0A+=09=09$tup->{t_hoff}=20+=3D=20128;=0A+=0A+=09=09= push=20@expected,=0A+=09=09=09qr/${$header}data=20begins=20at=20offset=20= 152=20beyond=20the=20tuple=20length=2058/,=0A+=09=09=09= qr/${$header}tuple=20data=20should=20begin=20at=20byte=2024,=20but=20= actually=20begins=20at=20byte=20152=20\(3=20attributes,=20no=20nulls\)/;=0A= +=09}=0A+=09elsif=20($offnum=20=3D=3D=206)=0A+=09{=0A+=09=09#=20Corrupt=20= the=20tuple=20t_hoff,=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20+=3D=20= 3;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 27=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=207)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20= underflow=20but=20correct=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=20= 8;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 16=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=208)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20= underflow=20and=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=203;=0A= +=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 21=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=209)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20to=20look=20= like=20it=20has=20lots=20of=20attributes,=20not=20just=203=0A+=09=09= $tup->{t_infomask2}=20|=3D=20HEAP_NATTS_MASK;=0A+=0A+=09=09push=20= @expected,=0A+=09=09=09qr/${$header}number=20of=20attributes=202047=20= exceeds=20maximum=20expected=20for=20table=203/;=0A+=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2010)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20to=20= look=20like=20it=20has=20lots=20of=20attributes,=20some=20of=0A+=09=09#=20= them=20null.=20=20This=20falsely=20creates=20the=20impression=20that=20= the=20t_bits=0A+=09=09#=20array=20is=20longer=20than=20just=20one=20= byte,=20but=20t_hoff=20still=20says=20otherwise.=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09$tup->{t_infomask2}=20= |=3D=20HEAP_NATTS_MASK;=0A+=09=09$tup->{t_bits}=20=3D=200xAA;=0A+=0A+=09=09= push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20should=20begin=20= at=20byte=20280,=20but=20actually=20begins=20at=20byte=2024=20\(2047=20= attributes,=20has=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=20= 11)=0A+=09{=0A+=09=09#=20Same=20as=20above,=20but=20this=20time=20t_hoff=20= plays=20along=0A+=09=09$tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09= $tup->{t_infomask2}=20|=3D=20(HEAP_NATTS_MASK=20&=200x40);=0A+=09=09= $tup->{t_bits}=20=3D=200xAA;=0A+=09=09$tup->{t_hoff}=20=3D=2032;=0A+=0A+=09= =09push=20@expected,=0A+=09=09=09qr/${$header}number=20of=20attributes=20= 67=20exceeds=20maximum=20expected=20for=20table=203/;=0A+=09}=0A+=09= elsif=20($offnum=20=3D=3D=2012)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= bits=20in=20column=20'b'=201-byte=20varlena=20header=0A+=09=09= $tup->{b_header}=20=3D=200x80;=0A+=0A+=09=09$header=20=3D=20header(0,=20= $offnum,=201);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}attribute=201=20with=20length=204294967295=20ends=20at=20= offset=20416848000=20beyond=20total=20tuple=20length=2058/;=0A+=09}=0A+=09= elsif=20($offnum=20=3D=3D=2013)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= bits=20in=20column=20'c'=20toast=20pointer=0A+=09=09$tup->{c6}=20=3D=20= 41;=0A+=09=09$tup->{c7}=20=3D=2041;=0A+=0A+=09=09$header=20=3D=20= header(0,=20$offnum,=202);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}final=20toast=20chunk=20number=200=20differs=20from=20= expected=20value=206/,=0A+=09=09=09qr/${header}toasted=20value=20for=20= attribute=202=20missing=20from=20toast=20table/;=0A+=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2014)=0A+=09{=0A+=09=09#=20Set=20both=20= HEAP_XMAX_LOCK_ONLY=20and=20HEAP_KEYS_UPDATED=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_LOCK_ONLY;=0A+=09=09= $tup->{t_infomask2}=20|=3D=20HEAP_KEYS_UPDATED;=0A+=0A+=09=09push=20= @expected,=0A+=09=09=09qr/${header}tuple=20is=20marked=20as=20only=20= locked,=20but=20also=20claims=20key=20columns=20were=20updated/;=0A+=09}=0A= +=09elsif=20($offnum=20=3D=3D=2015)=0A+=09{=0A+=09=09#=20Set=20both=20= HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_IS_MULTI;=0A+=09=09$tup->{t_xmax}=20= =3D=204;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}multitransaction=20ID=204=20equals=20or=20exceeds=20next=20= valid=20multitransaction=20ID=201/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D= =2016)=09#=20Last=20offnum=20must=20equal=20ROWCOUNT=0A+=09{=0A+=09=09#=20= Set=20both=20HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_IS_MULTI;=0A+=09=09$tup->{t_xmax}=20= =3D=204000000000;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}multitransaction=20ID=204000000000=20precedes=20relation=20= minimum=20multitransaction=20ID=20threshold=201/;=0A+=09}=0A+=09= write_tuple($file,=20$offset,=20$tup);=0A+}=0A+close($file);=0A= +$node->start;=0A+=0A+#=20Run=20pg_amcheck=20against=20the=20corrupt=20= table=20with=20epoch=3D0,=20comparing=20actual=0A+#=20corruption=20= messages=20against=20the=20expected=20messages=0A= +$node->command_checks_all(=0A+=09['pg_amcheck',=20'--check-toast',=20= '--skip-indexes',=20'-p',=20$port,=20'postgres'],=0A+=090,=0A+=09[=20= @expected=20],=0A+=09[=20qr/^$/=20],=0A+=09'Expected=20corruption=20= message=20output');=0A+=0A+$node->teardown_node;=0A+$node->clean_node;=0A= diff=20--git=20a/contrib/pg_amcheck/t/005_opclass_damage.pl=20= b/contrib/pg_amcheck/t/005_opclass_damage.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..fdbb1ea402=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/005_opclass_damage.pl=0A@@=20-0,0=20+1,52=20@@=0A= +#=20This=20regression=20test=20checks=20the=20behavior=20of=20the=20= btree=20validation=20in=20the=0A+#=20presence=20of=20breaking=20sort=20= order=20changes.=0A+#=0A+use=20strict;=0A+use=20warnings;=0A+use=20= PostgresNode;=0A+use=20TestLib;=0A+use=20Test::More=20tests=20=3D>=206;=0A= +=0A+my=20$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+=0A+#=20Create=20a=20custom=20operator=20class=20and=20= an=20index=20which=20uses=20it.=0A+$node->safe_psql('postgres',=20q(=0A+=09= CREATE=20EXTENSION=20amcheck;=0A+=0A+=09CREATE=20FUNCTION=20int4_asc_cmp=20= (a=20int4,=20b=20int4)=20RETURNS=20int=20LANGUAGE=20sql=20AS=20$$=0A+=09=09= SELECT=20CASE=20WHEN=20$1=20=3D=20$2=20THEN=200=20WHEN=20$1=20>=20$2=20= THEN=201=20ELSE=20-1=20END;=20$$;=0A+=0A+=09CREATE=20OPERATOR=20CLASS=20= int4_fickle_ops=20FOR=20TYPE=20int4=20USING=20btree=20AS=0A+=09=20=20=20=20= OPERATOR=201=20<=20(int4,=20int4),=20OPERATOR=202=20<=3D=20(int4,=20= int4),=0A+=09=20=20=20=20OPERATOR=203=20=3D=20(int4,=20int4),=20OPERATOR=20= 4=20>=3D=20(int4,=20int4),=0A+=09=20=20=20=20OPERATOR=205=20>=20(int4,=20= int4),=20FUNCTION=201=20int4_asc_cmp(int4,=20int4);=0A+=0A+=09CREATE=20= TABLE=20int4tbl=20(i=20int4);=0A+=09INSERT=20INTO=20int4tbl=20(SELECT=20= *=20FROM=20generate_series(1,1000)=20gs);=0A+=09CREATE=20INDEX=20= fickleidx=20ON=20int4tbl=20USING=20btree=20(i=20int4_fickle_ops);=0A+));=0A= +=0A+#=20We=20have=20not=20yet=20broken=20the=20index,=20so=20we=20= should=20get=20no=20corruption=0A+$node->command_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$node->port,=20'postgres'=20],=0A+=09qr/^$/,=0A+=09= 'pg_amcheck=20all=20schemas,=20tables=20and=20indexes=20reports=20no=20= corruption');=0A+=0A+#=20Change=20the=20operator=20class=20to=20use=20a=20= function=20which=20sorts=20in=20a=20different=0A+#=20order=20to=20= corrupt=20the=20btree=20index=0A+$node->safe_psql('postgres',=20q(=0A+=09= CREATE=20FUNCTION=20int4_desc_cmp=20(int4,=20int4)=20RETURNS=20int=20= LANGUAGE=20sql=20AS=20$$=0A+=09=09SELECT=20CASE=20WHEN=20$1=20=3D=20$2=20= THEN=200=20WHEN=20$1=20>=20$2=20THEN=20-1=20ELSE=201=20END;=20$$;=0A+=09= UPDATE=20pg_catalog.pg_amproc=0A+=09=09SET=20amproc=20=3D=20= 'int4_desc_cmp'::regproc=0A+=09=09WHERE=20amproc=20=3D=20= 'int4_asc_cmp'::regproc=0A+));=0A+=0A+#=20Index=20corruption=20should=20= now=20be=20reported=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20= '-x',=20'-p',=20$node->port,=20'postgres'=20],=0A+=09qr/item=20order=20= invariant=20violated=20for=20index=20"fickleidx"/,=0A+=09'pg_amcheck=20= all=20schemas,=20tables=20and=20indexes=20reports=20fickleidx=20= corruption'=0A+);=0Adiff=20--git=20a/doc/src/sgml/contrib.sgml=20= b/doc/src/sgml/contrib.sgml=0Aindex=204e833d79ef..1efca8adc4=20100644=0A= ---=20a/doc/src/sgml/contrib.sgml=0A+++=20b/doc/src/sgml/contrib.sgml=0A= @@=20-119,6=20+119,7=20@@=20CREATE=20EXTENSION=20= module_name;=0A=20=20&oldsnapshot;=0A=20=20= &pageinspect;=0A=20=20&passwordcheck;=0A+=20&pgamcheck;=0A=20=20= &pgbuffercache;=0A=20=20&pgcrypto;=0A=20=20&pgfreespacemap;=0Adiff=20= --git=20a/doc/src/sgml/filelist.sgml=20b/doc/src/sgml/filelist.sgml=0A= index=2038e8aa0bbf..a4e1b28b38=20100644=0A---=20= a/doc/src/sgml/filelist.sgml=0A+++=20b/doc/src/sgml/filelist.sgml=0A@@=20= -133,6=20+133,7=20@@=0A=20=0A=20=0A=20=0A+=0A=20=0A=20=0A=20=0Adiff=20--git=20a/doc/src/sgml/pgamcheck.sgml=20= b/doc/src/sgml/pgamcheck.sgml=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..3e059e7753=0A---=20/dev/null=0A+++=20= b/doc/src/sgml/pgamcheck.sgml=0A@@=20-0,0=20+1,228=20@@=0A+=0A+=0A+=0A+=20pg_amcheck=0A+=0A+=20= =0A+=20=20pg_amcheck=0A= +=20=0A+=0A+=20=0A+=20=20The=20= pg_amcheck=20module=20provides=20a=20command=20line=20= interface=0A+=20=20to=20the=20=20corruption=20= checking=20functionality.=0A+=20=0A+=0A+=20=0A+=20=20= pg_amcheck=20is=20a=20regular=0A+=20=20= PostgreSQL=20client=20application.=20You=20= can=20perform=0A+=20=20corruption=20checks=20from=20any=20remote=20host=20= that=20has=20access=20to=20the=20database=0A+=20=20connecting=20as=20a=20= user=20with=20sufficient=20privileges=20to=20check=20tables=20and=20= indexes.=0A+=20=20Currently,=20this=20requires=20execute=20privileges=20= on=20's=0A+=20=20= bt_index_parent_check=20and=20= verify_heapam=0A+=20=20functions.=0A+=20=0A+=0A= +=0A+pg_amcheck=20[OPTION]...=20[DBNAME=20[USERNAME]]=0A+=20=20= General=20options:=0A+=20=20=20=20-V,=20--version=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20output=20version=20information,=20then=20exit=0A= +=20=20=20=20-?,=20--help=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20show=20this=20help,=20then=20exit=0A+=20=20=20=20-s,=20= --strict-names=20=20=20=20=20=20=20=20=20=20=20require=20include=20= patterns=20to=20match=20at=20least=20one=20entity=20each=0A+=20=20=20=20= -o,=20--on-error-stop=20=20=20=20=20=20=20=20=20=20stop=20checking=20at=20= end=20of=20first=20corrupt=20page=0A+=0A+=20=20Schema=20checking=20= options:=0A+=20=20=20=20-n,=20--schema=3DPATTERN=20=20=20=20=20=20=20=20=20= check=20relations=20in=20the=20specified=20schema(s)=20only=0A+=20=20=20=20= -N,=20--exclude-schema=3DPATTERN=20do=20NOT=20check=20relations=20in=20= the=20specified=20schema(s)=0A+=0A+=20=20Table=20checking=20options:=0A+=20= =20=20=20-t,=20--table=3DPATTERN=20=20=20=20=20=20=20=20=20=20check=20= the=20specified=20table(s)=20only=0A+=20=20=20=20-T,=20= --exclude-table=3DPATTERN=20=20do=20NOT=20check=20the=20specified=20= table(s)=0A+=20=20=20=20-b,=20--startblock=20=20=20=20=20=20=20=20=20=20=20= =20=20begin=20checking=20table(s)=20at=20the=20given=20starting=20block=20= number=0A+=20=20=20=20-e,=20--endblock=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20check=20table(s)=20only=20up=20to=20the=20given=20ending=20= block=20number=0A+=20=20=20=20-f,=20--skip-all-frozen=20=20=20=20=20=20=20= =20do=20NOT=20check=20blocks=20marked=20as=20all-frozen=0A+=20=20=20=20= -v,=20--skip-all-visible=20=20=20=20=20=20=20do=20NOT=20check=20blocks=20= marked=20as=20all-visible=0A+=0A+=20=20TOAST=20table=20checking=20= options:=0A+=20=20=20=20-z,=20--check-toast=20=20=20=20=20=20=20=20=20=20= =20=20check=20associated=20toast=20tables=20and=20toast=20indexes=0A+=20=20= =20=20-Z,=20--skip-toast=20=20=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20= check=20associated=20toast=20tables=20and=20toast=20indexes=0A+=20=20=20=20= -B,=20--toast-startblock=20=20=20=20=20=20=20begin=20checking=20toast=20= table(s)=20at=20the=20given=20starting=20block=0A+=20=20=20=20-E,=20= --toast-endblock=20=20=20=20=20=20=20=20=20check=20toast=20table(s)=20= only=20up=20to=20the=20given=20ending=20block=0A+=0A+=20=20Index=20= checking=20options:=0A+=20=20=20=20-x,=20--check-indexes=20=20=20=20=20=20= =20=20=20=20check=20btree=20indexes=20associated=20with=20tables=20being=20= checked=0A+=20=20=20=20-X,=20--skip-indexes=20=20=20=20=20=20=20=20=20=20= =20do=20NOT=20check=20any=20btree=20indexes=0A+=20=20=20=20-i,=20= --index=3DPATTERN=20=20=20=20=20=20=20=20=20=20check=20the=20specified=20= index(es)=20only=0A+=20=20=20=20-I,=20--exclude-index=3DPATTERN=20=20do=20= NOT=20check=20the=20specified=20index(es)=0A+=20=20=20=20-c,=20= --check-corrupt=20=20=20=20=20=20=20=20=20=20check=20indexes=20even=20if=20= their=20associated=20table=20is=20corrupt=0A+=20=20=20=20-C,=20= --skip-corrupt=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20check=20= indexes=20if=20their=20associated=20table=20is=20corrupt=0A+=20=20=20=20= -a,=20--heapallindexed=20=20=20=20=20=20=20=20=20check=20index=20tuples=20= against=20the=20table=20tuples=0A+=20=20=20=20-A,=20--no-heapallindexed=20= =20=20=20=20=20do=20NOT=20check=20index=20tuples=20against=20the=20table=20= tuples=0A+=20=20=20=20-r,=20--rootdescend=20=20=20=20=20=20=20=20=20=20=20= =20search=20from=20the=20root=20page=20for=20each=20index=20tuple=0A+=20=20= =20=20-R,=20--no-rootdescend=20=20=20=20=20=20=20=20=20do=20NOT=20search=20= from=20the=20root=20page=20for=20each=20index=20tuple=0A+=0A+=20=20= Connection=20options:=0A+=20=20=20=20-d,=20--dbname=3DDBNAME=20=20=20=20=20= =20=20=20=20=20database=20name=20to=20connect=20to=0A+=20=20=20=20-h,=20= --host=3DHOSTNAME=20=20=20=20=20=20=20=20=20=20database=20server=20host=20= or=20socket=20directory=0A+=20=20=20=20-p,=20--port=3DPORT=20=20=20=20=20= =20=20=20=20=20=20=20=20=20database=20server=20port=0A+=20=20=20=20-U,=20= --username=3DUSERNAME=20=20=20=20=20=20database=20user=20name=0A+=20=20=20= =20-w,=20--no-password=20=20=20=20=20=20=20=20=20=20=20=20never=20prompt=20= for=20password=0A+=20=20=20=20-W,=20--password=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20force=20password=20prompt=20(should=20happen=20= automatically)=0A+=0A+=0A+=20=0A+=20=20= Options=0A+=0A+=20=20=0A+=20=20=20To=20specify=20= which=20database=20server=20pg_amcheck=20= should=0A+=20=20=20contact,=20use=20the=20command=20line=20options=20= =20or=0A+=20=20=20=20and=20= =20or=0A+=20=20=20.=20The=20= default=20host=20is=20the=20local=20host=0A+=20=20=20or=20whatever=20= your=20PGHOST=20environment=20variable=20specifies.=0A+=20= =20=20Similarly,=20the=20default=20port=20is=20indicated=20by=20the=20= PGPORT=0A+=20=20=20environment=20variable=20or,=20failing=20= that,=20by=20the=20compiled-in=20default.=0A+=20=20=0A+=20=0A+=20=20= =0A+=20=20=20Like=20any=20other=20= PostgreSQL=20client=20application,=0A+=20=20=20= pg_amcheck=20will=20by=20default=20connect=20= with=20the=0A+=20=20=20database=20user=20name=20that=20is=20equal=20to=20= the=20current=20operating=20system=20user=20name.=0A+=20=20=20To=20= override=20this,=20either=20specify=20the=20=20option=20= or=20set=20the=0A+=20=20=20environment=20variable=20= PGUSER.=20Remember=20that=0A+=20=20=20= pg_amcheck=20connections=20are=20subject=20to=20= the=20normal=0A+=20=20=20client=20authentication=20mechanisms=20(which=20= are=20described=20in=20).=0A+=20=20=0A+=20=0A+=20=20= =0A+=20=20=20To=20restrict=20checking=20of=20tables=20and=20= indexes=20to=20specific=20schemas,=20specify=20the=0A+=20=20=20= =20or=20=20option=20with=20a=20= pattern.=0A+=20=20=20To=20exclude=20checking=20of=20tables=20and=20= indexes=20within=20specific=20schemas,=20specify=0A+=20=20=20the=20= =20or=20=20option=20= with=0A+=20=20=20a=20pattern.=0A+=20=20=0A+=20=0A+=20=20=0A= +=20=20=20To=20specify=20which=20tables=20are=20checked,=20specify=20the=0A= +=20=20=20=20or=20=20option=20= with=20a=20pattern.=0A+=20=20=20To=20exclude=20checking=20of=20tables,=20= specify=20the=0A+=20=20=20=20or=20= =20option=20with=20a=0A+=20=20=20= pattern.=0A+=20=20=0A+=20=0A+=20=20=0A+=20=20=20To=20check=20= indexes=20associated=20with=20checked=20tables,=20specify=20the=0A+=20=20= =20=20or=20=20option.=20= =20Only=0A+=20=20=20indexes=20on=20tables=20which=20are=20being=20= checked=20will=20themselves=20be=20checked.=20=20To=0A+=20=20=20check=20= all=20indexes=20in=20a=20database,=20all=20tables=20on=20which=20the=20= indexes=20exist=20must=0A+=20=20=20also=20be=20checked.=20=20This=20= restriction=20may=20be=20relaxed=20in=20the=20future.=0A+=20=20=0A= +=20=0A+=20=20=0A+=20=20=20To=20restrict=20the=20range=20of=20= blocks=20within=20a=20table=20that=20are=20checked,=20specify=20the=0A+=20= =20=20=20or=20=20and/or=0A= +=20=20=20=20or=20=20= options=20with=20numeric=0A+=20=20=20values=20for=20the=20starting=20and=20= ending=20block=20numbers.=20=20Although=20these=20options=0A+=20=20=20= make=20the=20most=20sense=20when=20applied=20to=20a=20single=20table,=20= if=20specified=20along=20with=0A+=20=20=20options=20that=20select=20= multiple=20tables,=20each=20table=20check=20will=20be=20restricted=20to=0A= +=20=20=20the=20specified=20blocks.=20=20If=20= =20is=20omitted,=20checking=0A+=20=20=20= begins=20with=20the=20first=20block.=20If=20=20= is=20omitted,=0A+=20=20=20checking=20continues=20to=20the=20end=20of=20= the=20relation.=0A+=20=20=0A+=20=0A+=20=20=0A+=20=20=20Some=20= users=20may=20wish=20to=20periodically=20check=20tables=20without=20= incurring=20the=20cost=0A+=20=20=20of=20rechecking=20older=20table=20= blocks,=20presumably=20because=20those=20blocks=20have=0A+=20=20=20= already=20been=20checked=20in=20the=20past.=20=20There=20is=20at=20= present=20no=20perfect=20way=20to=20do=0A+=20=20=20this.=20=20Although=20= the=20=20and=20=0A= +=20=20=20options=20can=20be=20used=20to=20restrict=20blocks,=20the=20= user=20is=20not=20expected=20to=20have=0A+=20=20=20perfect=20knowledge=20= of=20which=20blocks=20have=20already=20been=20checked,=20and=20in=20any=0A= +=20=20=20event,=20some=20blocks=20that=20were=20previously=20checked=20= may=20have=20been=20subject=20to=0A+=20=20=20modification=20since=20the=20= last=20check.=20=20As=20an=20approximation=20to=20the=20desired=0A+=20=20= =20functionality,=20one=20can=20specify=20the=0A+=20=20=20= =20or=20=20option,=20= or=0A+=20=20=20alternatively=20the=0A+=20=20=20=20or=20= =20option=20to=20skip=0A+=20=20=20= blocks=20marked=20in=20the=20visibility=20map=20as=20all-frozen=20or=20= all-visible,=0A+=20=20=20respectively.=0A+=20=20=0A+=20=0A= +=0A+=20=0A+=20=20Example=20Usage=0A+=0A+=20=20= =0A+=20=20=20For=20table=20corruption,=20each=20detected=20= corruption=20is=20reported=20on=20two=20lines,=20the=0A+=20=20=20first=20= line=20shows=20the=20location=20and=20the=20second=20line=20shows=20a=20= message=20describing=0A+=20=20=20the=20problem.=0A+=20=20=0A+=0A+=20= =20=0A+=20=20=20Checking=20an=20entire=20database=20which=20= contains=20one=20corrupt=20table,=20"mytable",=0A+=20=20=20along=20with=20= the=20output:=0A+=20=20=0A+=0A+=0A+%=20pg_amcheck=20= --check-toast=20--skip-indexes=20mydb=0A= +(relname=3Dmytable,blkno=3D17,offnum=3D12,attnum=3D)=0A+xmin=20= 4294967295=20precedes=20relation=20freeze=20threshold=2017:1134217582=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D4,attnum=3D)=0A+data=20begins=20= at=20offset=20152=20beyond=20the=20tuple=20length=2058=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D4,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 152=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D5,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 27=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D6,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 16=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D7,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 21=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D2,attnum=3D)=0A+number=20of=20= attributes=202047=20exceeds=20maximum=20expected=20for=20table=203=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D10,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=20280,=20but=20actually=20begins=20at=20byte=20= 24=20(2047=20attributes,=20has=20nulls)=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D15,attnum=3D)=0A+number=20of=20= attributes=2067=20exceeds=20maximum=20expected=20for=20table=203=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D16,attnum=3D1)=0A+attribute=20= 1=20with=20length=204294967295=20ends=20at=20offset=20416848000=20beyond=20= total=20tuple=20length=2058=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D18,attnum=3D2)=0A+final=20= toast=20chunk=20number=200=20differs=20from=20expected=20value=206=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D19,attnum=3D2)=0A+toasted=20= value=20for=20attribute=202=20missing=20from=20toast=20table=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D21,attnum=3D)=0A+tuple=20is=20= marked=20as=20only=20locked,=20but=20also=20claims=20key=20columns=20= were=20updated=0A+(relname=3Dmytable,blkno=3D1147,offnum=3D22,attnum=3D)=0A= +multitransaction=20ID=201775655=20is=20from=20before=20relation=20= cutoff=202355572=0A+=0A+=0A+=20=20=0A+=20=20=20For=20= index=20corruption,=20the=20output=20is=20more=20free-form,=20and=20may=20= span=20differing=0A+=20=20=20numbers=20of=20lines=20per=20corruption=20= detected.=0A+=20=20=0A+=0A+=20=20=0A+=20=20=20Checking=20an=20= entire=20database=20which=20contains=20one=20corrupt=20index,=0A+=20=20=20= "corrupt_index",=20with=20corruption=20in=20the=20page=20header,=20along=20= with=20the=20output:=0A+=20=20=0A+=0A+=0A+%=20pg_amcheck=20= --check-toast=20--check-indexes=20--schema=3Dpublic=20= --table=3Dtable_with_corrupt_index=20mydb=0A+index=20check=20failed=20= for=20index=20corrupt_index=20of=20table=20table_with_corrupt_index:=0A= +ERROR:=20=20XX002:=20index=20"corrupt_index"=20is=20not=20a=20btree=0A= +LOCATION:=20=20_bt_getmeta,=20nbtpage.c:152=0A+=0A+=0A+=20=20= =0A+=20=20=20Checking=20again=20after=20rebuilding=20the=20index=20= but=20corrupting=20the=20contents,=0A+=20=20=20along=20with=20the=20= output:=0A+=20=20=0A+=0A+=0A+%=20pg_amcheck=20= --check-toast=20--check-indexes=20--schema=3Dpublic=20= --table=3Dtable_with_corrupt_index=20mydb=0A+index=20check=20failed=20= for=20index=20corrupt_index=20of=20table=20table_with_corrupt_index:=0A= +ERROR:=20=20XX002:=20index=20tuple=20size=20does=20not=20equal=20lp_len=20= in=20index=20"corrupt_index"=0A+DETAIL:=20=20Index=20tid=3D(39,49)=20= tuple=20size=3D3373=20lp_len=3D24=20page=20lsn=3D0/2B548C0.=0A+HINT:=20=20= This=20could=20be=20a=20torn=20page=20problem.=0A+LOCATION:=20=20= bt_target_page_check,=20verify_nbtree.c:1125=0A+=0A+=0A+=20= =0A+=0Adiff=20--git=20a/src/tools/msvc/Mkvcbuild.pm=20= b/src/tools/msvc/Mkvcbuild.pm=0Aindex=2090594bd41b..ec87fb85b3=20100644=0A= ---=20a/src/tools/msvc/Mkvcbuild.pm=0A+++=20= b/src/tools/msvc/Mkvcbuild.pm=0A@@=20-33,9=20+33,9=20@@=20my=20= @unlink_on_exit;=0A=20=0A=20#=20Set=20of=20variables=20for=20modules=20= in=20contrib/=20and=20src/test/modules/=0A=20my=20$contrib_defines=20=3D=20= {=20'refint'=20=3D>=20'REFINT_VERBOSE'=20};=0A-my=20@contrib_uselibpq=20= =3D=20('dblink',=20'oid2name',=20'postgres_fdw',=20'vacuumlo');=0A-my=20= @contrib_uselibpgport=20=20=20=3D=20('oid2name',=20'pg_standby',=20= 'vacuumlo');=0A-my=20@contrib_uselibpgcommon=20=3D=20('oid2name',=20= 'pg_standby',=20'vacuumlo');=0A+my=20@contrib_uselibpq=20=3D=20= ('dblink',=20'oid2name',=20'pg_amcheck',=20'postgres_fdw',=20= 'vacuumlo');=0A+my=20@contrib_uselibpgport=20=20=20=3D=20('oid2name',=20= 'pg_amcheck',=20'pg_standby',=20'vacuumlo');=0A+my=20= @contrib_uselibpgcommon=20=3D=20('oid2name',=20'pg_amcheck',=20= 'pg_standby',=20'vacuumlo');=0A=20my=20$contrib_extralibs=20=20=20=20=20=20= =3D=20undef;=0A=20my=20$contrib_extraincludes=20=3D=20{=20'dblink'=20=3D>=20= ['src/backend']=20};=0A=20my=20$contrib_extrasource=20=3D=20{=0Adiff=20= --git=20a/src/tools/pgindent/typedefs.list=20= b/src/tools/pgindent/typedefs.list=0Aindex=20ff853634bc..2408bb2bf6=20= 100644=0A---=20a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-102,6=20+102,7=20@@=20= AlterUserMappingStmt=0A=20AlteredTableInfo=0A=20AlternativeSubPlan=0A=20= AlternativeSubPlanState=0A+AmCheckSettings=0A=20= AnalyzeAttrComputeStatsFunc=0A=20AnalyzeAttrFetchFunc=0A=20= AnalyzeForeignTable_function=0A@@=20-403,6=20+404,7=20@@=20ConfigData=0A=20= ConfigVariable=0A=20ConnCacheEntry=0A=20ConnCacheKey=0A+ConnectOptions=0A= =20ConnStatusType=0A=20ConnType=0A=20ConnectionStateEnum=0A--=20=0A= 2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F Content-Disposition: attachment; filename=v24-0003-Creating-non-throwing-interface-to-clog-and-slru.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v24-0003-Creating-non-throwing-interface-to-clog-and-slru.patch" Content-Transfer-Encoding: quoted-printable =46rom=2073c97ae7cd2e39aee80712132654f4745a35bdae=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Wed,=2021=20Oct=202020=2020:26:07=20-0700=0ASubject:=20[PATCH=20= v24=202/4]=20Creating=20non-throwing=20interface=20to=20clog=20and=20= slru.=0A=0A---=0A=20src/backend/access/transam/clog.c=20=20=20=20=20=20|=20= 21=20+++---=0A=20src/backend/access/transam/commit_ts.c=20|=20=204=20+-=0A= =20src/backend/access/transam/multixact.c=20|=2016=20++---=0A=20= src/backend/access/transam/slru.c=20=20=20=20=20=20|=2023=20+++---=0A=20= src/backend/access/transam/subtrans.c=20=20|=20=204=20+-=0A=20= src/backend/access/transam/transam.c=20=20=20|=2098=20= +++++++++-----------------=0A=20src/backend/commands/async.c=20=20=20=20=20= =20=20=20=20=20=20|=20=204=20+-=0A=20= src/backend/storage/lmgr/predicate.c=20=20=20|=20=204=20+-=0A=20= src/include/access/clog.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= 18=20+----=0A=20src/include/access/clogdefs.h=20=20=20=20=20=20=20=20=20=20= |=2033=20+++++++++=0A=20src/include/access/slru.h=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=206=20+-=0A=20src/include/access/transam.h=20=20=20= =20=20=20=20=20=20=20=20|=20=203=20+=0A=2012=20files=20changed,=20122=20= insertions(+),=20112=20deletions(-)=0A=20create=20mode=20100644=20= src/include/access/clogdefs.h=0A=0Adiff=20--git=20= a/src/backend/access/transam/clog.c=20= b/src/backend/access/transam/clog.c=0Aindex=20034349aa7b..a2eb3e2983=20= 100644=0A---=20a/src/backend/access/transam/clog.c=0A+++=20= b/src/backend/access/transam/clog.c=0A@@=20-357,7=20+357,7=20@@=20= TransactionIdSetPageStatusInternal(TransactionId=20xid,=20int=20= nsubxids,=0A=20=09=20*=20write-busy,=20since=20we=20don't=20care=20if=20= the=20update=20reaches=20disk=20sooner=20than=0A=20=09=20*=20we=20think.=0A= =20=09=20*/=0A-=09slotno=20=3D=20SimpleLruReadPage(XactCtl,=20pageno,=20= XLogRecPtrIsInvalid(lsn),=20xid);=0A+=09slotno=20=3D=20= SimpleLruReadPage(XactCtl,=20pageno,=20XLogRecPtrIsInvalid(lsn),=20xid,=20= true);=0A=20=0A=20=09/*=0A=20=09=20*=20Set=20the=20main=20transaction=20= id,=20if=20any.=0A@@=20-631,7=20+631,7=20@@=20= TransactionIdSetStatusBit(TransactionId=20xid,=20XidStatus=20status,=20= XLogRecPtr=20lsn,=20i=0A=20=20*=20for=20most=20uses;=20= TransactionLogFetch()=20in=20transam.c=20is=20the=20intended=20caller.=0A= =20=20*/=0A=20XidStatus=0A-TransactionIdGetStatus(TransactionId=20xid,=20= XLogRecPtr=20*lsn)=0A+TransactionIdGetStatus(TransactionId=20xid,=20= XLogRecPtr=20*lsn,=20bool=20throwError)=0A=20{=0A=20=09int=09=09=09= pageno=20=3D=20TransactionIdToPage(xid);=0A=20=09int=09=09=09byteno=20=3D=20= TransactionIdToByte(xid);=0A@@=20-643,13=20+643,18=20@@=20= TransactionIdGetStatus(TransactionId=20xid,=20XLogRecPtr=20*lsn)=0A=20=0A= =20=09/*=20lock=20is=20acquired=20by=20SimpleLruReadPage_ReadOnly=20*/=0A= =20=0A-=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(XactCtl,=20pageno,=20= xid);=0A-=09byteptr=20=3D=20XactCtl->shared->page_buffer[slotno]=20+=20= byteno;=0A+=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(XactCtl,=20= pageno,=20xid,=20throwError);=0A+=09if=20(slotno=20=3D=3D=20= InvalidSlotNo)=0A+=09=09status=20=3D=20TRANSACTION_STATUS_UNKNOWN;=0A+=09= else=0A+=09{=0A+=09=09byteptr=20=3D=20= XactCtl->shared->page_buffer[slotno]=20+=20byteno;=0A=20=0A-=09status=20= =3D=20(*byteptr=20>>=20bshift)=20&=20CLOG_XACT_BITMASK;=0A+=09=09status=20= =3D=20(*byteptr=20>>=20bshift)=20&=20CLOG_XACT_BITMASK;=0A=20=0A-=09= lsnindex=20=3D=20GetLSNIndex(slotno,=20xid);=0A-=09*lsn=20=3D=20= XactCtl->shared->group_lsn[lsnindex];=0A+=09=09lsnindex=20=3D=20= GetLSNIndex(slotno,=20xid);=0A+=09=09*lsn=20=3D=20= XactCtl->shared->group_lsn[lsnindex];=0A+=09}=0A=20=0A=20=09= LWLockRelease(XactSLRULock);=0A=20=0A@@=20-796,7=20+801,7=20@@=20= TrimCLOG(void)=0A=20=09=09int=09=09=09slotno;=0A=20=09=09char=09=20=20=20= *byteptr;=0A=20=0A-=09=09slotno=20=3D=20SimpleLruReadPage(XactCtl,=20= pageno,=20false,=20xid);=0A+=09=09slotno=20=3D=20= SimpleLruReadPage(XactCtl,=20pageno,=20false,=20xid,=20true);=0A=20=09=09= byteptr=20=3D=20XactCtl->shared->page_buffer[slotno]=20+=20byteno;=0A=20=0A= =20=09=09/*=20Zero=20so-far-unused=20positions=20in=20the=20current=20= byte=20*/=0Adiff=20--git=20a/src/backend/access/transam/commit_ts.c=20= b/src/backend/access/transam/commit_ts.c=0Aindex=20= cb8a968801..98c685405c=20100644=0A---=20= a/src/backend/access/transam/commit_ts.c=0A+++=20= b/src/backend/access/transam/commit_ts.c=0A@@=20-237,7=20+237,7=20@@=20= SetXidCommitTsInPage(TransactionId=20xid,=20int=20nsubxids,=0A=20=0A=20=09= LWLockAcquire(CommitTsSLRULock,=20LW_EXCLUSIVE);=0A=20=0A-=09slotno=20=3D=20= SimpleLruReadPage(CommitTsCtl,=20pageno,=20true,=20xid);=0A+=09slotno=20= =3D=20SimpleLruReadPage(CommitTsCtl,=20pageno,=20true,=20xid,=20true);=0A= =20=0A=20=09TransactionIdSetCommitTs(xid,=20ts,=20nodeid,=20slotno);=0A=20= =09for=20(i=20=3D=200;=20i=20<=20nsubxids;=20i++)=0A@@=20-342,7=20+342,7=20= @@=20TransactionIdGetCommitTsData(TransactionId=20xid,=20TimestampTz=20= *ts,=0A=20=09}=0A=20=0A=20=09/*=20lock=20is=20acquired=20by=20= SimpleLruReadPage_ReadOnly=20*/=0A-=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(CommitTsCtl,=20pageno,=20xid);=0A+=09slotno=20= =3D=20SimpleLruReadPage_ReadOnly(CommitTsCtl,=20pageno,=20xid,=20true);=0A= =20=09memcpy(&entry,=0A=20=09=09=20=20=20= CommitTsCtl->shared->page_buffer[slotno]=20+=0A=20=09=09=20=20=20= SizeOfCommitTimestampEntry=20*=20entryno,=0Adiff=20--git=20= a/src/backend/access/transam/multixact.c=20= b/src/backend/access/transam/multixact.c=0Aindex=20= 43653fe572..ed902a9d64=20100644=0A---=20= a/src/backend/access/transam/multixact.c=0A+++=20= b/src/backend/access/transam/multixact.c=0A@@=20-881,7=20+881,7=20@@=20= RecordNewMultiXact(MultiXactId=20multi,=20MultiXactOffset=20offset,=0A=20= =09=20*=20enough=20that=20a=20MultiXactId=20is=20really=20involved.=20=20= Perhaps=20someday=20we'll=0A=20=09=20*=20take=20the=20trouble=20to=20= generalize=20the=20slru.c=20error=20reporting=20code.=0A=20=09=20*/=0A-=09= slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20= multi);=0A+=09slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20= pageno,=20true,=20multi,=20true);=0A=20=09offptr=20=3D=20= (MultiXactOffset=20*)=20MultiXactOffsetCtl->shared->page_buffer[slotno];=0A= =20=09offptr=20+=3D=20entryno;=0A=20=0A@@=20-914,7=20+914,7=20@@=20= RecordNewMultiXact(MultiXactId=20multi,=20MultiXactOffset=20offset,=0A=20= =0A=20=09=09if=20(pageno=20!=3D=20prev_pageno)=0A=20=09=09{=0A-=09=09=09= slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20true,=20= multi);=0A+=09=09=09slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20= pageno,=20true,=20multi,=20true);=0A=20=09=09=09prev_pageno=20=3D=20= pageno;=0A=20=09=09}=0A=20=0A@@=20-1345,7=20+1345,7=20@@=20retry:=0A=20=09= pageno=20=3D=20MultiXactIdToOffsetPage(multi);=0A=20=09entryno=20=3D=20= MultiXactIdToOffsetEntry(multi);=0A=20=0A-=09slotno=20=3D=20= SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20multi);=0A+=09= slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20= multi,=20true);=0A=20=09offptr=20=3D=20(MultiXactOffset=20*)=20= MultiXactOffsetCtl->shared->page_buffer[slotno];=0A=20=09offptr=20+=3D=20= entryno;=0A=20=09offset=20=3D=20*offptr;=0A@@=20-1377,7=20+1377,7=20@@=20= retry:=0A=20=09=09entryno=20=3D=20MultiXactIdToOffsetEntry(tmpMXact);=0A=20= =0A=20=09=09if=20(pageno=20!=3D=20prev_pageno)=0A-=09=09=09slotno=20=3D=20= SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20tmpMXact);=0A+=09= =09=09slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20= true,=20tmpMXact,=20true);=0A=20=0A=20=09=09offptr=20=3D=20= (MultiXactOffset=20*)=20MultiXactOffsetCtl->shared->page_buffer[slotno];=0A= =20=09=09offptr=20+=3D=20entryno;=0A@@=20-1418,7=20+1418,7=20@@=20retry:=0A= =20=0A=20=09=09if=20(pageno=20!=3D=20prev_pageno)=0A=20=09=09{=0A-=09=09=09= slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20true,=20= multi);=0A+=09=09=09slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20= pageno,=20true,=20multi,=20true);=0A=20=09=09=09prev_pageno=20=3D=20= pageno;=0A=20=09=09}=0A=20=0A@@=20-2063,7=20+2063,7=20@@=20= TrimMultiXact(void)=0A=20=09=09int=09=09=09slotno;=0A=20=09=09= MultiXactOffset=20*offptr;=0A=20=0A-=09=09slotno=20=3D=20= SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20nextMXact);=0A= +=09=09slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20= true,=20nextMXact,=20true);=0A=20=09=09offptr=20=3D=20(MultiXactOffset=20= *)=20MultiXactOffsetCtl->shared->page_buffer[slotno];=0A=20=09=09offptr=20= +=3D=20entryno;=0A=20=0A@@=20-2095,7=20+2095,7=20@@=20= TrimMultiXact(void)=0A=20=09=09int=09=09=09memberoff;=0A=20=0A=20=09=09= memberoff=20=3D=20MXOffsetToMemberOffset(offset);=0A-=09=09slotno=20=3D=20= SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20true,=20offset);=0A+=09= =09slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20= true,=20offset,=20true);=0A=20=09=09xidptr=20=3D=20(TransactionId=20*)=0A= =20=09=09=09(MultiXactMemberCtl->shared->page_buffer[slotno]=20+=20= memberoff);=0A=20=0A@@=20-2749,7=20+2749,7=20@@=20= find_multixact_start(MultiXactId=20multi,=20MultiXactOffset=20*result)=0A= =20=09=09return=20false;=0A=20=0A=20=09/*=20lock=20is=20acquired=20by=20= SimpleLruReadPage_ReadOnly=20*/=0A-=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(MultiXactOffsetCtl,=20pageno,=20multi);=0A+=09= slotno=20=3D=20SimpleLruReadPage_ReadOnly(MultiXactOffsetCtl,=20pageno,=20= multi,=20true);=0A=20=09offptr=20=3D=20(MultiXactOffset=20*)=20= MultiXactOffsetCtl->shared->page_buffer[slotno];=0A=20=09offptr=20+=3D=20= entryno;=0A=20=09offset=20=3D=20*offptr;=0Adiff=20--git=20= a/src/backend/access/transam/slru.c=20= b/src/backend/access/transam/slru.c=0Aindex=2016a7898697..daa145eeff=20= 100644=0A---=20a/src/backend/access/transam/slru.c=0A+++=20= b/src/backend/access/transam/slru.c=0A@@=20-385,14=20+385,15=20@@=20= SimpleLruWaitIO(SlruCtl=20ctl,=20int=20slotno)=0A=20=20*=20The=20= passed-in=20xid=20is=20used=20only=20for=20error=20reporting,=20and=20= may=20be=0A=20=20*=20InvalidTransactionId=20if=20no=20specific=20xid=20= is=20associated=20with=20the=20action.=0A=20=20*=0A-=20*=20Return=20= value=20is=20the=20shared-buffer=20slot=20number=20now=20holding=20the=20= page.=0A-=20*=20The=20buffer's=20LRU=20access=20info=20is=20updated.=0A+=20= *=20On=20error,=20when=20throwError=20is=20false,=20the=20return=20value=20= is=20negative.=0A+=20*=20Otherwise,=20return=20value=20is=20the=20= shared-buffer=20slot=20number=20now=20holding=20the=0A+=20*=20page,=20= and=20the=20buffer's=20LRU=20access=20info=20is=20updated.=0A=20=20*=0A=20= =20*=20Control=20lock=20must=20be=20held=20at=20entry,=20and=20will=20be=20= held=20at=20exit.=0A=20=20*/=0A=20int=0A=20SimpleLruReadPage(SlruCtl=20= ctl,=20int=20pageno,=20bool=20write_ok,=0A-=09=09=09=09=20=20= TransactionId=20xid)=0A+=09=09=09=09=20=20TransactionId=20xid,=20bool=20= throwError)=0A=20{=0A=20=09SlruShared=09shared=20=3D=20ctl->shared;=0A=20= =0A@@=20-465,7=20+466,11=20@@=20SimpleLruReadPage(SlruCtl=20ctl,=20int=20= pageno,=20bool=20write_ok,=0A=20=0A=20=09=09/*=20Now=20it's=20okay=20to=20= ereport=20if=20we=20failed=20*/=0A=20=09=09if=20(!ok)=0A-=09=09=09= SlruReportIOError(ctl,=20pageno,=20xid);=0A+=09=09{=0A+=09=09=09if=20= (throwError)=0A+=09=09=09=09SlruReportIOError(ctl,=20pageno,=20xid);=0A+=09= =09=09return=20InvalidSlotNo;=0A+=09=09}=0A=20=0A=20=09=09= SlruRecentlyUsed(shared,=20slotno);=0A=20=0A@@=20-484,14=20+489,16=20@@=20= SimpleLruReadPage(SlruCtl=20ctl,=20int=20pageno,=20bool=20write_ok,=0A=20= =20*=20The=20passed-in=20xid=20is=20used=20only=20for=20error=20= reporting,=20and=20may=20be=0A=20=20*=20InvalidTransactionId=20if=20no=20= specific=20xid=20is=20associated=20with=20the=20action.=0A=20=20*=0A-=20= *=20Return=20value=20is=20the=20shared-buffer=20slot=20number=20now=20= holding=20the=20page.=0A-=20*=20The=20buffer's=20LRU=20access=20info=20= is=20updated.=0A+=20*=20On=20error,=20when=20throwError=20is=20false,=20= the=20return=20value=20is=20negative.=0A+=20*=20Otherwise,=20return=20= value=20is=20the=20shared-buffer=20slot=20number=20now=20holding=20the=0A= +=20*=20page,=20and=20the=20buffer's=20LRU=20access=20info=20is=20= updated.=0A=20=20*=0A=20=20*=20Control=20lock=20must=20NOT=20be=20held=20= at=20entry,=20but=20will=20be=20held=20at=20exit.=0A=20=20*=20It=20is=20= unspecified=20whether=20the=20lock=20will=20be=20shared=20or=20= exclusive.=0A=20=20*/=0A=20int=0A-SimpleLruReadPage_ReadOnly(SlruCtl=20= ctl,=20int=20pageno,=20TransactionId=20xid)=0A= +SimpleLruReadPage_ReadOnly(SlruCtl=20ctl,=20int=20pageno,=20= TransactionId=20xid,=0A+=09=09=09=09=09=09=20=20=20bool=20throwError)=0A=20= {=0A=20=09SlruShared=09shared=20=3D=20ctl->shared;=0A=20=09int=09=09=09= slotno;=0A@@=20-520,7=20+527,7=20@@=20SimpleLruReadPage_ReadOnly(SlruCtl=20= ctl,=20int=20pageno,=20TransactionId=20xid)=0A=20=09= LWLockRelease(shared->ControlLock);=0A=20=09= LWLockAcquire(shared->ControlLock,=20LW_EXCLUSIVE);=0A=20=0A-=09return=20= SimpleLruReadPage(ctl,=20pageno,=20true,=20xid);=0A+=09return=20= SimpleLruReadPage(ctl,=20pageno,=20true,=20xid,=20throwError);=0A=20}=0A=20= =0A=20/*=0Adiff=20--git=20a/src/backend/access/transam/subtrans.c=20= b/src/backend/access/transam/subtrans.c=0Aindex=200111e867c7..353b946731=20= 100644=0A---=20a/src/backend/access/transam/subtrans.c=0A+++=20= b/src/backend/access/transam/subtrans.c=0A@@=20-83,7=20+83,7=20@@=20= SubTransSetParent(TransactionId=20xid,=20TransactionId=20parent)=0A=20=0A= =20=09LWLockAcquire(SubtransSLRULock,=20LW_EXCLUSIVE);=0A=20=0A-=09= slotno=20=3D=20SimpleLruReadPage(SubTransCtl,=20pageno,=20true,=20xid);=0A= +=09slotno=20=3D=20SimpleLruReadPage(SubTransCtl,=20pageno,=20true,=20= xid,=20true);=0A=20=09ptr=20=3D=20(TransactionId=20*)=20= SubTransCtl->shared->page_buffer[slotno];=0A=20=09ptr=20+=3D=20entryno;=0A= =20=0A@@=20-123,7=20+123,7=20@@=20SubTransGetParent(TransactionId=20xid)=0A= =20=0A=20=09/*=20lock=20is=20acquired=20by=20SimpleLruReadPage_ReadOnly=20= */=0A=20=0A-=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(SubTransCtl,=20= pageno,=20xid);=0A+=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(SubTransCtl,=20pageno,=20xid,=20true);=0A=20=09= ptr=20=3D=20(TransactionId=20*)=20= SubTransCtl->shared->page_buffer[slotno];=0A=20=09ptr=20+=3D=20entryno;=0A= =20=0Adiff=20--git=20a/src/backend/access/transam/transam.c=20= b/src/backend/access/transam/transam.c=0Aindex=20a28918657c..88f867e5ef=20= 100644=0A---=20a/src/backend/access/transam/transam.c=0A+++=20= b/src/backend/access/transam/transam.c=0A@@=20-35,7=20+35,8=20@@=20= static=20XidStatus=20cachedFetchXidStatus;=0A=20static=20XLogRecPtr=20= cachedCommitLSN;=0A=20=0A=20/*=20Local=20functions=20*/=0A-static=20= XidStatus=20TransactionLogFetch(TransactionId=20transactionId);=0A= +static=20XidStatus=20TransactionLogFetch(TransactionId=20transactionId,=0A= +=09=09=09=09=09=09=09=09=09=20bool=20throwError);=0A=20=0A=20=0A=20/*=20= ----------------------------------------------------------------=0A@@=20= -49,7=20+50,7=20@@=20static=20XidStatus=20= TransactionLogFetch(TransactionId=20transactionId);=0A=20=20*=20= TransactionLogFetch=20---=20fetch=20commit=20status=20of=20specified=20= transaction=20id=0A=20=20*/=0A=20static=20XidStatus=0A= -TransactionLogFetch(TransactionId=20transactionId)=0A= +TransactionLogFetch(TransactionId=20transactionId,=20bool=20throwError)=0A= =20{=0A=20=09XidStatus=09xidstatus;=0A=20=09XLogRecPtr=09xidlsn;=0A@@=20= -76,14=20+77,16=20@@=20TransactionLogFetch(TransactionId=20= transactionId)=0A=20=09/*=0A=20=09=20*=20Get=20the=20transaction=20= status.=0A=20=09=20*/=0A-=09xidstatus=20=3D=20= TransactionIdGetStatus(transactionId,=20&xidlsn);=0A+=09xidstatus=20=3D=20= TransactionIdGetStatus(transactionId,=20&xidlsn,=20throwError);=0A=20=0A=20= =09/*=0A=20=09=20*=20Cache=20it,=20but=20DO=20NOT=20cache=20status=20for=20= unfinished=20or=20sub-committed=0A=20=09=20*=20transactions!=20=20We=20= only=20cache=20status=20that=20is=20guaranteed=20not=20to=20change.=0A+=09= =20*=20Likewise,=20DO=20NOT=20cache=20when=20the=20status=20is=20= unknown.=0A=20=09=20*/=0A=20=09if=20(xidstatus=20!=3D=20= TRANSACTION_STATUS_IN_PROGRESS=20&&=0A-=09=09xidstatus=20!=3D=20= TRANSACTION_STATUS_SUB_COMMITTED)=0A+=09=09xidstatus=20!=3D=20= TRANSACTION_STATUS_SUB_COMMITTED=20&&=0A+=09=09xidstatus=20!=3D=20= TRANSACTION_STATUS_UNKNOWN)=0A=20=09{=0A=20=09=09cachedFetchXid=20=3D=20= transactionId;=0A=20=09=09cachedFetchXidStatus=20=3D=20xidstatus;=0A@@=20= -96,6=20+99,7=20@@=20TransactionLogFetch(TransactionId=20transactionId)=0A= =20/*=20----------------------------------------------------------------=0A= =20=20*=09=09=09=09=09=09Interface=20functions=0A=20=20*=0A+=20*=09=09= TransactionIdResolveStatus=0A=20=20*=09=09TransactionIdDidCommit=0A=20=20= *=09=09TransactionIdDidAbort=0A=20=20*=09=09=3D=3D=3D=3D=3D=3D=3D=3D=0A= @@=20-115,24=20+119,17=20@@=20TransactionLogFetch(TransactionId=20= transactionId)=0A=20=20*/=0A=20=0A=20/*=0A-=20*=20TransactionIdDidCommit=0A= -=20*=09=09True=20iff=20transaction=20associated=20with=20the=20= identifier=20did=20commit.=0A-=20*=0A-=20*=20Note:=0A-=20*=09=09Assumes=20= transaction=20identifier=20is=20valid=20and=20exists=20in=20clog.=0A+=20= *=20TransactionIdResolveStatus=0A+=20*=20=09=09Returns=20the=20status=20= of=20the=20transaction=20associated=20with=20the=20identifer,=0A+=20*=09=09= recursively=20resolving=20sub-committed=20transaction=20status=20by=20= checking=0A+=20*=09=09the=20parent=20transaction.=0A=20=20*/=0A-bool=09=09= =09=09=09=09=09/*=20true=20if=20given=20transaction=20committed=20*/=0A= -TransactionIdDidCommit(TransactionId=20transactionId)=0A+XidStatus=0A= +TransactionIdResolveStatus(TransactionId=20transactionId,=20bool=20= throwError)=0A=20{=0A=20=09XidStatus=09xidstatus;=0A=20=0A-=09xidstatus=20= =3D=20TransactionLogFetch(transactionId);=0A-=0A-=09/*=0A-=09=20*=20If=20= it's=20marked=20committed,=20it's=20committed.=0A-=09=20*/=0A-=09if=20= (xidstatus=20=3D=3D=20TRANSACTION_STATUS_COMMITTED)=0A-=09=09return=20= true;=0A+=09xidstatus=20=3D=20TransactionLogFetch(transactionId,=20= throwError);=0A=20=0A=20=09/*=0A=20=09=20*=20If=20it's=20marked=20= subcommitted,=20we=20have=20to=20check=20the=20parent=20recursively.=0A= @@=20-153,21=20+150,31=20@@=20TransactionIdDidCommit(TransactionId=20= transactionId)=0A=20=09=09TransactionId=20parentXid;=0A=20=0A=20=09=09if=20= (TransactionIdPrecedes(transactionId,=20TransactionXmin))=0A-=09=09=09= return=20false;=0A+=09=09=09return=20TRANSACTION_STATUS_ABORTED;=0A=20=09= =09parentXid=20=3D=20SubTransGetParent(transactionId);=0A=20=09=09if=20= (!TransactionIdIsValid(parentXid))=0A=20=09=09{=0A=20=09=09=09= elog(WARNING,=20"no=20pg_subtrans=20entry=20for=20subcommitted=20XID=20= %u",=0A=20=09=09=09=09=20transactionId);=0A-=09=09=09return=20false;=0A+=09= =09=09return=20TRANSACTION_STATUS_ABORTED;=0A=20=09=09}=0A-=09=09return=20= TransactionIdDidCommit(parentXid);=0A+=09=09return=20= TransactionIdResolveStatus(parentXid,=20throwError);=0A=20=09}=0A+=09= return=20xidstatus;=0A+}=0A=20=0A-=09/*=0A-=09=20*=20It's=20not=20= committed.=0A-=09=20*/=0A-=09return=20false;=0A+/*=0A+=20*=20= TransactionIdDidCommit=0A+=20*=09=09True=20iff=20transaction=20= associated=20with=20the=20identifier=20did=20commit.=0A+=20*=0A+=20*=20= Note:=0A+=20*=09=09Assumes=20transaction=20identifier=20is=20valid=20and=20= exists=20in=20clog.=0A+=20*/=0A+bool=09=09=09=09=09=09=09/*=20true=20if=20= given=20transaction=20committed=20*/=0A= +TransactionIdDidCommit(TransactionId=20transactionId)=0A+{=0A+=09return=20= (TransactionIdResolveStatus(transactionId,=20true)=20=3D=3D=0A+=09=09=09= TRANSACTION_STATUS_COMMITTED);=0A=20}=0A=20=0A=20/*=0A@@=20-180,43=20= +187,8=20@@=20TransactionIdDidCommit(TransactionId=20transactionId)=0A=20= bool=09=09=09=09=09=09=09/*=20true=20if=20given=20transaction=20aborted=20= */=0A=20TransactionIdDidAbort(TransactionId=20transactionId)=0A=20{=0A-=09= XidStatus=09xidstatus;=0A-=0A-=09xidstatus=20=3D=20= TransactionLogFetch(transactionId);=0A-=0A-=09/*=0A-=09=20*=20If=20it's=20= marked=20aborted,=20it's=20aborted.=0A-=09=20*/=0A-=09if=20(xidstatus=20= =3D=3D=20TRANSACTION_STATUS_ABORTED)=0A-=09=09return=20true;=0A-=0A-=09= /*=0A-=09=20*=20If=20it's=20marked=20subcommitted,=20we=20have=20to=20= check=20the=20parent=20recursively.=0A-=09=20*=20However,=20if=20it's=20= older=20than=20TransactionXmin,=20we=20can't=20look=20at=0A-=09=20*=20= pg_subtrans;=20instead=20assume=20that=20the=20parent=20crashed=20= without=20cleaning=20up=0A-=09=20*=20its=20children.=0A-=09=20*/=0A-=09= if=20(xidstatus=20=3D=3D=20TRANSACTION_STATUS_SUB_COMMITTED)=0A-=09{=0A-=09= =09TransactionId=20parentXid;=0A-=0A-=09=09if=20= (TransactionIdPrecedes(transactionId,=20TransactionXmin))=0A-=09=09=09= return=20true;=0A-=09=09parentXid=20=3D=20= SubTransGetParent(transactionId);=0A-=09=09if=20= (!TransactionIdIsValid(parentXid))=0A-=09=09{=0A-=09=09=09/*=20see=20= notes=20in=20TransactionIdDidCommit=20*/=0A-=09=09=09elog(WARNING,=20"no=20= pg_subtrans=20entry=20for=20subcommitted=20XID=20%u",=0A-=09=09=09=09=20= transactionId);=0A-=09=09=09return=20true;=0A-=09=09}=0A-=09=09return=20= TransactionIdDidAbort(parentXid);=0A-=09}=0A-=0A-=09/*=0A-=09=20*=20It's=20= not=20aborted.=0A-=09=20*/=0A-=09return=20false;=0A+=09return=20= (TransactionIdResolveStatus(transactionId,=20true)=20=3D=3D=0A+=09=09=09= TRANSACTION_STATUS_ABORTED);=0A=20}=0A=20=0A=20/*=0A@@=20-419,7=20+391,7=20= @@=20TransactionIdGetCommitLSN(TransactionId=20xid)=0A=20=09/*=0A=20=09=20= *=20Get=20the=20transaction=20status.=0A=20=09=20*/=0A-=09(void)=20= TransactionIdGetStatus(xid,=20&result);=0A+=09(void)=20= TransactionIdGetStatus(xid,=20&result,=20true);=0A=20=0A=20=09return=20= result;=0A=20}=0Adiff=20--git=20a/src/backend/commands/async.c=20= b/src/backend/commands/async.c=0Aindex=208dbcace3f9..a49126dba0=20100644=0A= ---=20a/src/backend/commands/async.c=0A+++=20= b/src/backend/commands/async.c=0A@@=20-1477,7=20+1477,7=20@@=20= asyncQueueAddEntries(ListCell=20*nextNotify)=0A=20=09=09slotno=20=3D=20= SimpleLruZeroPage(NotifyCtl,=20pageno);=0A=20=09else=0A=20=09=09slotno=20= =3D=20SimpleLruReadPage(NotifyCtl,=20pageno,=20true,=0A-=09=09=09=09=09=09= =09=09=20=20=20InvalidTransactionId);=0A+=09=09=09=09=09=09=09=09=20=20=20= InvalidTransactionId,=20true);=0A=20=0A=20=09/*=20Note=20we=20mark=20the=20= page=20dirty=20before=20writing=20in=20it=20*/=0A=20=09= NotifyCtl->shared->page_dirty[slotno]=20=3D=20true;=0A@@=20-2010,7=20= +2010,7=20@@=20asyncQueueReadAllNotifications(void)=0A=20=09=09=09=20*=20= part=20of=20the=20page=20we=20will=20actually=20inspect.=0A=20=09=09=09=20= */=0A=20=09=09=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(NotifyCtl,=20= curpage,=0A-=09=09=09=09=09=09=09=09=09=09=09=09InvalidTransactionId);=0A= +=09=09=09=09=09=09=09=09=09=09=09=09InvalidTransactionId,=20true);=0A=20= =09=09=09if=20(curpage=20=3D=3D=20QUEUE_POS_PAGE(head))=0A=20=09=09=09{=0A= =20=09=09=09=09/*=20we=20only=20want=20to=20read=20as=20far=20as=20head=20= */=0Adiff=20--git=20a/src/backend/storage/lmgr/predicate.c=20= b/src/backend/storage/lmgr/predicate.c=0Aindex=208a365b400c..6cf12e46f6=20= 100644=0A---=20a/src/backend/storage/lmgr/predicate.c=0A+++=20= b/src/backend/storage/lmgr/predicate.c=0A@@=20-904,7=20+904,7=20@@=20= SerialAdd(TransactionId=20xid,=20SerCommitSeqNo=20= minConflictCommitSeqNo)=0A=20=09=09slotno=20=3D=20= SimpleLruZeroPage(SerialSlruCtl,=20targetPage);=0A=20=09}=0A=20=09else=0A= -=09=09slotno=20=3D=20SimpleLruReadPage(SerialSlruCtl,=20targetPage,=20= true,=20xid);=0A+=09=09slotno=20=3D=20SimpleLruReadPage(SerialSlruCtl,=20= targetPage,=20true,=20xid,=20true);=0A=20=0A=20=09SerialValue(slotno,=20= xid)=20=3D=20minConflictCommitSeqNo;=0A=20=09= SerialSlruCtl->shared->page_dirty[slotno]=20=3D=20true;=0A@@=20-946,7=20= +946,7=20@@=20SerialGetMinConflictCommitSeqNo(TransactionId=20xid)=0A=20=09= =20*=20but=20will=20return=20with=20that=20lock=20held,=20which=20must=20= then=20be=20released.=0A=20=09=20*/=0A=20=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(SerialSlruCtl,=0A-=09=09=09=09=09=09=09=09=09=09= SerialPage(xid),=20xid);=0A+=09=09=09=09=09=09=09=09=09=09= SerialPage(xid),=20xid,=20true);=0A=20=09val=20=3D=20SerialValue(slotno,=20= xid);=0A=20=09LWLockRelease(SerialSLRULock);=0A=20=09return=20val;=0A= diff=20--git=20a/src/include/access/clog.h=20b/src/include/access/clog.h=0A= index=206c840cbf29..cf299cd8f6=20100644=0A---=20= a/src/include/access/clog.h=0A+++=20b/src/include/access/clog.h=0A@@=20= -11,24=20+11,11=20@@=0A=20#ifndef=20CLOG_H=0A=20#define=20CLOG_H=0A=20=0A= +#include=20"access/clogdefs.h"=0A=20#include=20"access/xlogreader.h"=0A=20= #include=20"storage/sync.h"=0A=20#include=20"lib/stringinfo.h"=0A=20=0A= -/*=0A-=20*=20Possible=20transaction=20statuses=20---=20note=20that=20= all-zeroes=20is=20the=20initial=0A-=20*=20state.=0A-=20*=0A-=20*=20A=20= "subcommitted"=20transaction=20is=20a=20committed=20subtransaction=20= whose=20parent=0A-=20*=20hasn't=20committed=20or=20aborted=20yet.=0A-=20= */=0A-typedef=20int=20XidStatus;=0A-=0A-#define=20= TRANSACTION_STATUS_IN_PROGRESS=09=090x00=0A-#define=20= TRANSACTION_STATUS_COMMITTED=09=090x01=0A-#define=20= TRANSACTION_STATUS_ABORTED=09=09=090x02=0A-#define=20= TRANSACTION_STATUS_SUB_COMMITTED=090x03=0A-=0A=20typedef=20struct=20= xl_clog_truncate=0A=20{=0A=20=09int=09=09=09pageno;=0A@@=20-38,7=20+25,8=20= @@=20typedef=20struct=20xl_clog_truncate=0A=20=0A=20extern=20void=20= TransactionIdSetTreeStatus(TransactionId=20xid,=20int=20nsubxids,=0A=20=09= =09=09=09=09=09=09=09=09=20=20=20TransactionId=20*subxids,=20XidStatus=20= status,=20XLogRecPtr=20lsn);=0A-extern=20XidStatus=20= TransactionIdGetStatus(TransactionId=20xid,=20XLogRecPtr=20*lsn);=0A= +extern=20XidStatus=20TransactionIdGetStatus(TransactionId=20xid,=20= XLogRecPtr=20*lsn,=0A+=09=09=09=09=09=09=09=09=09=09bool=20throwError);=0A= =20=0A=20extern=20Size=20CLOGShmemBuffers(void);=0A=20extern=20Size=20= CLOGShmemSize(void);=0Adiff=20--git=20a/src/include/access/clogdefs.h=20= b/src/include/access/clogdefs.h=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..0f9996bb08=0A---=20/dev/null=0A+++=20= b/src/include/access/clogdefs.h=0A@@=20-0,0=20+1,33=20@@=0A+/*=0A+=20*=20= clogdefs.h=0A+=20*=0A+=20*=20PostgreSQL=20transaction-commit-log=20= manager=0A+=20*=0A+=20*=20Portions=20Copyright=20(c)=201996-2020,=20= PostgreSQL=20Global=20Development=20Group=0A+=20*=20Portions=20Copyright=20= (c)=201994,=20Regents=20of=20the=20University=20of=20California=0A+=20*=0A= +=20*=20src/include/access/clogdefs.h=0A+=20*/=0A+#ifndef=20CLOGDEFS_H=0A= +#define=20CLOGDEFS_H=0A+=0A+/*=0A+=20*=20Possible=20transaction=20= statuses=20---=20note=20that=20all-zeroes=20is=20the=20initial=0A+=20*=20= state.=0A+=20*=0A+=20*=20A=20"subcommitted"=20transaction=20is=20a=20= committed=20subtransaction=20whose=20parent=0A+=20*=20hasn't=20committed=20= or=20aborted=20yet.=0A+=20*=0A+=20*=20An=20"unknown"=20status=20= indicates=20an=20error=20condition,=20such=20as=20when=20the=20clog=20= has=0A+=20*=20been=20erroneously=20truncated=20and=20the=20commit=20= status=20of=20a=20transaction=20cannot=20be=0A+=20*=20determined.=0A+=20= */=0A+typedef=20enum=20XidStatus=20{=0A+=09= TRANSACTION_STATUS_IN_PROGRESS=20=3D=200x00,=0A+=09= TRANSACTION_STATUS_COMMITTED=20=3D=200x01,=0A+=09= TRANSACTION_STATUS_ABORTED=20=3D=200x02,=0A+=09= TRANSACTION_STATUS_SUB_COMMITTED=20=3D=200x03,=0A+=09= TRANSACTION_STATUS_UNKNOWN=20=3D=200x04=09/*=20error=20condition=20*/=0A= +}=20XidStatus;=0A+=0A+#endif=09=09=09=09=09=09=09/*=20CLOG_H=20*/=0A= diff=20--git=20a/src/include/access/slru.h=20b/src/include/access/slru.h=0A= index=20b39b43504d..0b6a5669d8=20100644=0A---=20= a/src/include/access/slru.h=0A+++=20b/src/include/access/slru.h=0A@@=20= -133,6=20+133,8=20@@=20typedef=20struct=20SlruCtlData=0A=20=0A=20typedef=20= SlruCtlData=20*SlruCtl;=0A=20=0A+#define=20InvalidSlotNo=20((int)=20-1)=0A= +=0A=20=0A=20extern=20Size=20SimpleLruShmemSize(int=20nslots,=20int=20= nlsns);=0A=20extern=20void=20SimpleLruInit(SlruCtl=20ctl,=20const=20char=20= *name,=20int=20nslots,=20int=20nlsns,=0A@@=20-140,9=20+142,9=20@@=20= extern=20void=20SimpleLruInit(SlruCtl=20ctl,=20const=20char=20*name,=20= int=20nslots,=20int=20nlsns,=0A=20=09=09=09=09=09=09=20=20= SyncRequestHandler=20sync_handler);=0A=20extern=20int=09= SimpleLruZeroPage(SlruCtl=20ctl,=20int=20pageno);=0A=20extern=20int=09= SimpleLruReadPage(SlruCtl=20ctl,=20int=20pageno,=20bool=20write_ok,=0A-=09= =09=09=09=09=09=09=20=20TransactionId=20xid);=0A+=09=09=09=09=09=09=09=20= =20TransactionId=20xid,=20bool=20throwError);=0A=20extern=20int=09= SimpleLruReadPage_ReadOnly(SlruCtl=20ctl,=20int=20pageno,=0A-=09=09=09=09= =09=09=09=09=09=20=20=20TransactionId=20xid);=0A+=09=09=09=09=09=09=09=09= =09=20=20=20TransactionId=20xid,=20bool=20throwError);=0A=20extern=20= void=20SimpleLruWritePage(SlruCtl=20ctl,=20int=20slotno);=0A=20extern=20= void=20SimpleLruWriteAll(SlruCtl=20ctl,=20bool=20allow_redirtied);=0A=20= extern=20void=20SimpleLruTruncate(SlruCtl=20ctl,=20int=20cutoffPage);=0A= diff=20--git=20a/src/include/access/transam.h=20= b/src/include/access/transam.h=0Aindex=202f1f144db4..7d5e2f614d=20100644=0A= ---=20a/src/include/access/transam.h=0A+++=20= b/src/include/access/transam.h=0A@@=20-14,6=20+14,7=20@@=0A=20#ifndef=20= TRANSAM_H=0A=20#define=20TRANSAM_H=0A=20=0A+#include=20= "access/clogdefs.h"=0A=20#include=20"access/xlogdefs.h"=0A=20=0A=20=0A@@=20= -264,6=20+265,8=20@@=20extern=20PGDLLIMPORT=20VariableCache=20= ShmemVariableCache;=0A=20/*=0A=20=20*=20prototypes=20for=20functions=20= in=20transam/transam.c=0A=20=20*/=0A+extern=20XidStatus=20= TransactionIdResolveStatus(TransactionId=20transactionId,=0A+=09=09=09=09= =09=09=09=09=09=09=09bool=20throwError);=0A=20extern=20bool=20= TransactionIdDidCommit(TransactionId=20transactionId);=0A=20extern=20= bool=20TransactionIdDidAbort(TransactionId=20transactionId);=0A=20extern=20= bool=20TransactionIdIsKnownCompleted(TransactionId=20transactionId);=0A= --=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F Content-Disposition: attachment; filename=v24-0004-Using-non-throwing-clog-interface-from-amcheck.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v24-0004-Using-non-throwing-clog-interface-from-amcheck.patch" Content-Transfer-Encoding: quoted-printable =46rom=20c8bbb5e1d98c46c671b0e40c6f78b7b0e4932cc0=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Mon,=2026=20Oct=202020=2007:34:42=20-0700=0ASubject:=20[PATCH=20= v24=203/4]=20Using=20non-throwing=20clog=20interface=20from=20amcheck=0A=0A= Converting=20the=20heap=20checking=20functions=20to=20use=20the=20= recently=20introduced=0Anon-throwing=20interface=20to=20clog=20when=20= checking=20transaction=20commit=20status,=20and=0Aadding=20corruption=20= reports=20about=20missing=20clog=20rather=20than=20aborting.=0A---=0A=20= contrib/amcheck/verify_heapam.c=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= =2074=20+++++++------=0A=20contrib/pg_amcheck/t/006_clog_truncation.pl=20= |=20111=20++++++++++++++++++++=0A=20src/tools/pgindent/typedefs.list=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=201=20-=0A=203=20files=20changed,=20= 155=20insertions(+),=2031=20deletions(-)=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/006_clog_truncation.pl=0A=0Adiff=20--git=20= a/contrib/amcheck/verify_heapam.c=20b/contrib/amcheck/verify_heapam.c=0A= index=208bb890438a..263cbb37c1=20100644=0A---=20= a/contrib/amcheck/verify_heapam.c=0A+++=20= b/contrib/amcheck/verify_heapam.c=0A@@=20-10,6=20+10,7=20@@=0A=20=20*/=0A= =20#include=20"postgres.h"=0A=20=0A+#include=20"access/clogdefs.h"=0A=20= #include=20"access/detoast.h"=0A=20#include=20"access/genam.h"=0A=20= #include=20"access/heapam.h"=0A@@=20-43,13=20+44,6=20@@=20typedef=20enum=20= XidBoundsViolation=0A=20=09XID_BOUNDS_OK=0A=20}=20XidBoundsViolation;=0A=20= =0A-typedef=20enum=20XidCommitStatus=0A-{=0A-=09XID_COMMITTED,=0A-=09= XID_IN_PROGRESS,=0A-=09XID_ABORTED=0A-}=20XidCommitStatus;=0A-=0A=20= typedef=20enum=20SkipPages=0A=20{=0A=20=09SKIP_PAGES_ALL_FROZEN,=0A@@=20= -83,7=20+77,7=20@@=20typedef=20struct=20HeapCheckContext=0A=20=09=20*=20= Cached=20copies=20of=20the=20most=20recently=20checked=20xid=20and=20its=20= status.=0A=20=09=20*/=0A=20=09TransactionId=20cached_xid;=0A-=09= XidCommitStatus=20cached_status;=0A+=09XidStatus=09cached_status;=0A=20=0A= =20=09/*=20Values=20concerning=20the=20heap=20relation=20being=20checked=20= */=0A=20=09Relation=09rel;=0A@@=20-148,7=20+142,7=20@@=20static=20= XidBoundsViolation=20check_mxid_valid_in_rel(MultiXactId=20mxid,=0A=20=09= =09=09=09=09=09=09=09=09=09=09=09=20=20HeapCheckContext=20*ctx);=0A=20= static=20XidBoundsViolation=20get_xid_status(TransactionId=20xid,=0A=20=09= =09=09=09=09=09=09=09=09=09=20HeapCheckContext=20*ctx,=0A-=09=09=09=09=09= =09=09=09=09=09=20XidCommitStatus=20*status);=0A+=09=09=09=09=09=09=09=09= =09=09=20XidStatus=20*status);=0A=20=0A=20/*=0A=20=20*=20Scan=20and=20= report=20corruption=20in=20heap=20pages,=20optionally=20reconciling=20= toasted=0A@@=20-675,7=20+669,7=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09else=20if=20(infomask=20&=20= HEAP_MOVED_OFF=20||=0A=20=09=09=09=09=20infomask=20&=20HEAP_MOVED_IN)=0A=20= =09=09{=0A-=09=09=09XidCommitStatus=20status;=0A+=09=09=09XidStatus=09= status;=0A=20=09=09=09TransactionId=20xvac=20=3D=20= HeapTupleHeaderGetXvac(tuphdr);=0A=20=0A=20=09=09=09switch=20= (get_xid_status(xvac,=20ctx,=20&status))=0A@@=20-710,17=20+704,25=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09=09=09case=20XID_BOUNDS_OK:=0A=20=09=09= =09=09=09switch=20(status)=0A=20=09=09=09=09=09{=0A-=09=09=09=09=09=09= case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_IN_PROGRESS:=0A=20=09=09=09=09=09=09=09return=20true;=09= /*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A-=09=09=09=09=09=09case=20= XID_COMMITTED:=0A-=09=09=09=09=09=09case=20XID_ABORTED:=0A+=09=09=09=09=09= =09case=20TRANSACTION_STATUS_COMMITTED:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_ABORTED:=0A=20=09=09=09=09=09=09=09return=20false;=09= /*=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_UNKNOWN:=0A+=09=09=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20= psprintf("old-style=20VACUUM=20FULL=20transaction=20ID=20%u=20= transaction=20status=20is=20lost",=0A+=09=09=09=09=09=09=09=09=09=09=09=09= =09=20=20=20xvac));=0A+=09=09=09=09=09=09=09return=20false;=09/*=20= corruption=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_SUB_COMMITTED:=0A+=09=09=09=09=09=09=09elog(ERROR,=20= "get_xid_status=20failed=20to=20resolve=20parent=20transaction=20= status");=0A+=09=09=09=09=09=09=09return=20false;=09/*=20not=20reached=20= */=0A=20=09=09=09=09=09}=0A=20=09=09=09}=0A=20=09=09}=0A=20=09=09else=0A=20= =09=09{=0A-=09=09=09XidCommitStatus=20status;=0A+=09=09=09XidStatus=09= status;=0A=20=0A=20=09=09=09switch=20(get_xid_status(raw_xmin,=20ctx,=20= &status))=0A=20=09=09=09{=0A@@=20-752,12=20+754,20=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09=09=09case=20XID_BOUNDS_OK:=0A=20=09=09= =09=09=09switch=20(status)=0A=20=09=09=09=09=09{=0A-=09=09=09=09=09=09= case=20XID_COMMITTED:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_COMMITTED:=0A=20=09=09=09=09=09=09=09break;=0A-=09=09=09= =09=09=09case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_IN_PROGRESS:=0A=20=09=09=09=09=09=09=09return=20true;=09= /*=20insert=20or=20delete=20in=20progress=20*/=0A-=09=09=09=09=09=09case=20= XID_ABORTED:=0A+=09=09=09=09=09=09case=20TRANSACTION_STATUS_ABORTED:=0A=20= =09=09=09=09=09=09=09return=20false;=09/*=20HEAPTUPLE_DEAD=20*/=0A+=09=09= =09=09=09=09case=20TRANSACTION_STATUS_UNKNOWN:=0A+=09=09=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20= psprintf("raw=20xmin=20%u=20transaction=20status=20is=20lost",=0A+=09=09=09= =09=09=09=09=09=09=09=09=09=09=20=20=20raw_xmin));=0A+=09=09=09=09=09=09=09= return=20false;=09/*=20corruption=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_SUB_COMMITTED:=0A+=09=09=09=09=09=09=09elog(ERROR,=20= "get_xid_status=20failed=20to=20resolve=20parent=20transaction=20= status");=0A+=09=09=09=09=09=09=09return=20false;=09/*=20not=20reached=20= */=0A=20=09=09=09=09=09}=0A=20=09=09=09}=0A=20=09=09}=0A@@=20-767,7=20= +777,7=20@@=20check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09{=0A=20=09=09if=20(infomask=20&=20= HEAP_XMAX_IS_MULTI)=0A=20=09=09{=0A-=09=09=09XidCommitStatus=20status;=0A= +=09=09=09XidStatus=09status;=0A=20=09=09=09TransactionId=20xmax=20=3D=20= HeapTupleGetUpdateXid(tuphdr);=0A=20=0A=20=09=09=09switch=20= (get_xid_status(xmax,=20ctx,=20&status))=0A@@=20-801,12=20+811,20=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09=09=09case=20XID_BOUNDS_OK:=0A=20=09=09= =09=09=09switch=20(status)=0A=20=09=09=09=09=09{=0A-=09=09=09=09=09=09= case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_IN_PROGRESS:=0A=20=09=09=09=09=09=09=09return=20true;=09= /*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A-=09=09=09=09=09=09case=20= XID_COMMITTED:=0A-=09=09=09=09=09=09case=20XID_ABORTED:=0A+=09=09=09=09=09= =09case=20TRANSACTION_STATUS_COMMITTED:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_ABORTED:=0A=20=09=09=09=09=09=09=09return=20false;=09= /*=20HEAPTUPLE_RECENTLY_DEAD=20or=0A=20=09=09=09=09=09=09=09=09=09=09=09=20= *=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_UNKNOWN:=0A+=09=09=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20= psprintf("xmax=20%u=20transaction=20status=20is=20lost",=0A+=09=09=09=09=09= =09=09=09=09=09=09=09=09=20=20=20xmax));=0A+=09=09=09=09=09=09=09return=20= false;=09/*=20corruption=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_SUB_COMMITTED:=0A+=09=09=09=09=09=09=09elog(ERROR,=20= "get_xid_status=20failed=20to=20resolve=20parent=20transaction=20= status");=0A+=09=09=09=09=09=09=09return=20false;=09/*=20not=20reached=20= */=0A=20=09=09=09=09=09}=0A=20=09=09=09}=0A=20=0A@@=20-1401,7=20+1419,7=20= @@=20check_mxid_valid_in_rel(MultiXactId=20mxid,=20HeapCheckContext=20= *ctx)=0A=20=20*/=0A=20static=20XidBoundsViolation=0A=20= get_xid_status(TransactionId=20xid,=20HeapCheckContext=20*ctx,=0A-=09=09=09= =20=20=20XidCommitStatus=20*status)=0A+=09=09=09=20=20=20XidStatus=20= *status)=0A=20{=0A=20=09FullTransactionId=20fxid;=0A=20=09= FullTransactionId=20clog_horizon;=0A@@=20-1412,7=20+1430,7=20@@=20= get_xid_status(TransactionId=20xid,=20HeapCheckContext=20*ctx,=0A=20=09= else=20if=20(xid=20=3D=3D=20BootstrapTransactionId=20||=20xid=20=3D=3D=20= FrozenTransactionId)=0A=20=09{=0A=20=09=09if=20(status=20!=3D=20NULL)=0A= -=09=09=09*status=20=3D=20XID_COMMITTED;=0A+=09=09=09*status=20=3D=20= TRANSACTION_STATUS_COMMITTED;=0A=20=09=09return=20XID_BOUNDS_OK;=0A=20=09= }=0A=20=0A@@=20-1447,7=20+1465,7=20@@=20get_xid_status(TransactionId=20= xid,=20HeapCheckContext=20*ctx,=0A=20=09=09return=20XID_BOUNDS_OK;=0A=20=09= }=0A=20=0A-=09*status=20=3D=20XID_COMMITTED;=0A+=09*status=20=3D=20= TRANSACTION_STATUS_COMMITTED;=0A=20=09LWLockAcquire(XactTruncationLock,=20= LW_SHARED);=0A=20=09clog_horizon=20=3D=0A=20=09=09= FullTransactionIdFromXidAndCtx(ShmemVariableCache->oldestClogXid,=0A@@=20= -1455,13=20+1473,9=20@@=20get_xid_status(TransactionId=20xid,=20= HeapCheckContext=20*ctx,=0A=20=09if=20= (FullTransactionIdPrecedesOrEquals(clog_horizon,=20fxid))=0A=20=09{=0A=20= =09=09if=20(TransactionIdIsCurrentTransactionId(xid))=0A-=09=09=09= *status=20=3D=20XID_IN_PROGRESS;=0A-=09=09else=20if=20= (TransactionIdDidCommit(xid))=0A-=09=09=09*status=20=3D=20XID_COMMITTED;=0A= -=09=09else=20if=20(TransactionIdDidAbort(xid))=0A-=09=09=09*status=20=3D=20= XID_ABORTED;=0A+=09=09=09*status=20=3D=20TRANSACTION_STATUS_IN_PROGRESS;=0A= =20=09=09else=0A-=09=09=09*status=20=3D=20XID_IN_PROGRESS;=0A+=09=09=09= *status=20=3D=20TransactionIdResolveStatus(xid,=20false);=0A=20=09}=0A=20= =09LWLockRelease(XactTruncationLock);=0A=20=09ctx->cached_xid=20=3D=20= xid;=0Adiff=20--git=20a/contrib/pg_amcheck/t/006_clog_truncation.pl=20= b/contrib/pg_amcheck/t/006_clog_truncation.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..f205ae7ede=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/006_clog_truncation.pl=0A@@=20-0,0=20+1,111=20@@=0A= +#=20This=20regression=20test=20checks=20the=20behavior=20of=20the=20= heap=20validation=20in=20the=0A+#=20presence=20of=20clog=20corruption.=0A= +=0A+use=20strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20= TestLib;=0A+=0A+use=20Test::More=20tests=20=3D>=203;=0A+=0A+my=20($node,=20= $pgdata,=20$clogdir);=0A+=0A+sub=20count_clog_files=0A+{=0A+=09my=20= $result=20=3D=200;=0A+=09opendir(DIR,=20$clogdir)=20or=20die=20"Cannot=20= opendir=20$clogdir:=20$!";=0A+=09while=20(my=20$fname=20=3D=20= readdir(DIR))=0A+=09{=0A+=09=09$result++=20if=20(-f=20= "$clogdir/$fname");=0A+=09}=0A+=09closedir(DIR);=0A+=09return=20$result;=0A= +}=0A+=0A+#=20Burn=20through=20enough=20xids=20that=20at=20least=20three=20= clog=20files=20exists=20in=20pg_xact/=0A+sub=20create_three_clog_files=0A= +{=0A+=09print=20STDERR=20"Generating=20clog=20entries....\n";=0A+=0A+=09= $node->safe_psql('postgres',=20q(=0A+=09=09CREATE=20PROCEDURE=20= burn_xids=20()=0A+=09=09LANGUAGE=20plpgsql=0A+=09=09AS=20$$=0A+=09=09= DECLARE=0A+=09=09=09loopcnt=20BIGINT;=0A+=09=09BEGIN=0A+=09=09=09FOR=20= loopcnt=20IN=201..32768=0A+=09=09=09LOOP=0A+=09=09=09=09PERFORM=20= txid_current();=0A+=09=09=09=09COMMIT;=0A+=09=09=09END=20LOOP;=0A+=09=09= END;=0A+=09=09$$;=0A+=09));=0A+=0A+=09do=20{=0A+=09=09= $node->safe_psql('postgres',=20'INSERT=20INTO=20test_0=20(i)=20VALUES=20= (0)');=0A+=09=09$node->safe_psql('postgres',=20'CALL=20burn_xids()');=0A= +=09=09print=20STDERR=20"Burned=20transaction=20ids...\n";=0A+=09=09= $node->safe_psql('postgres',=20'INSERT=20INTO=20test_1=20(i)=20VALUES=20= (1)');=0A+=09}=20while=20(count_clog_files()=20<=203);=0A+}=0A+=0A+#=20= Of=20the=20clog=20files=20in=20pg_xact,=20remove=20the=20second=20one,=20= sorted=20by=20name=20order.=0A+#=20This=20function,=20used=20along=20= with=20create_three_clog_files(),=20is=20intended=20to=0A+#=20remove=20= neither=20the=20newest=20nor=20the=20oldest=20clog=20file.=20=20= Experimentation=20shows=0A+#=20that=20removing=20the=20newest=20clog=20= file=20works=20ok,=20but=20for=20future-proofing,=20remove=0A+#=20one=20= less=20likely=20to=20be=20checked=20at=20server=20startup.=0A+sub=20= unlink_second_clog_file=0A+{=0A+=09my=20@paths;=0A+=09opendir(DIR,=20= $clogdir)=20or=20die=20"Cannot=20opendir=20$clogdir:=20$!";=0A+=09while=20= (my=20$fname=20=3D=20readdir(DIR))=0A+=09{=0A+=09=09my=20$path=20=3D=20= "$clogdir/$fname";=0A+=09=09next=20unless=20-f=20$path;=0A+=09=09push=20= @paths,=20$path;=0A+=09}=0A+=09closedir(DIR);=0A+=0A+=09my=20@ordered=20= =3D=20sort=20{=20$a=20cmp=20$b=20}=20@paths;=0A+=09unlink=20$ordered[1];=0A= +}=0A+=0A+#=20Set=20umask=20so=20test=20directories=20and=20files=20are=20= created=20with=20default=20permissions=0A+umask(0077);=0A+=0A+#=20Set=20= up=20the=20node.=20=20Once=20we=20corrupt=20clog,=20autovacuum=20workers=20= visiting=20tables=0A+#=20could=20crash=20the=20backend.=20=20Disable=20= autovacuum=20so=20that=20won't=20happen.=0A+$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A= +$node->append_conf('postgresql.conf',=20'autovacuum=3Doff');=0A+$pgdata=20= =3D=20$node->data_dir;=0A+$clogdir=20=3D=20join('/',=20$pgdata,=20= 'pg_xact');=0A+$node->start;=0A+$node->safe_psql('postgres',=20"CREATE=20= EXTENSION=20amcheck");=0A+$node->safe_psql('postgres',=20"CREATE=20TABLE=20= test_0=20(i=20INTEGER)");=0A+$node->safe_psql('postgres',=20"CREATE=20= TABLE=20test_1=20(i=20INTEGER)");=0A+$node->safe_psql('postgres',=20= "VACUUM=20FREEZE");=0A+=0A+create_three_clog_files();=0A+=0A+#=20= Corruptly=20delete=20a=20clog=20file=0A+$node->stop;=0A= +unlink_second_clog_file();=0A+$node->start;=0A+=0A+my=20$port=20=3D=20= $node->port;=0A+=0A+#=20Run=20pg_amcheck=20against=20the=20corrupt=20= database,=20looking=20for=20clog=20related=0A+#=20corruption=20messages=0A= +$node->command_checks_all(=0A+=09['pg_amcheck',=20'--check-toast',=20= '--skip-indexes',=20'-p',=20$port,=20'postgres'],=0A+=090,=0A+=09[=20= qr/transaction=20status=20is=20lost/=20],=0A+=09[=20qr/^$/=20],=0A+=09= 'Expected=20corruption=20message=20output');=0A+=0A= +$node->teardown_node;=0A+$node->clean_node;=0Adiff=20--git=20= a/src/tools/pgindent/typedefs.list=20b/src/tools/pgindent/typedefs.list=0A= index=202408bb2bf6..c1144c1a92=20100644=0A---=20= a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-2796,7=20+2796,6=20@@=20= XactCallbackItem=0A=20XactEvent=0A=20XactLockTableWaitInfo=0A=20= XidBoundsViolation=0A-XidCommitStatus=0A=20XidHorizonPrefetchState=0A=20= XidStatus=0A=20XmlExpr=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F Content-Disposition: attachment; filename=v24-0005-Adding-ACL-checks-for-verify_heapam.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v24-0005-Adding-ACL-checks-for-verify_heapam.patch" Content-Transfer-Encoding: quoted-printable =46rom=2057e184b4d1577c7846b08603e5ad2f052f98b133=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Wed,=2021=20Oct=202020=2020:27:23=20-0700=0ASubject:=20[PATCH=20= v24=204/4]=20Adding=20ACL=20checks=20for=20verify_heapam=0A=0ARequiring=20= select=20privileges=20tables=20scanned=20by=20verify_heapam,=20in=0A= addition=20to=20the=20already=20required=20execute=20privileges=20on=20= the=20function.=0A---=0A=20contrib/amcheck/expected/check_heap.out=20|=20= 6=20++++++=0A=20contrib/amcheck/sql/check_heap.sql=20=20=20=20=20=20|=20= 7=20+++++++=0A=20contrib/amcheck/verify_heapam.c=20=20=20=20=20=20=20=20=20= |=208=20++++++++=0A=20doc/src/sgml/pgamcheck.sgml=20=20=20=20=20=20=20=20= =20=20=20=20=20|=202=20+-=0A=204=20files=20changed,=2022=20= insertions(+),=201=20deletion(-)=0A=0Adiff=20--git=20= a/contrib/amcheck/expected/check_heap.out=20= b/contrib/amcheck/expected/check_heap.out=0Aindex=20= 882f853d56..41cdc6435c=20100644=0A---=20= a/contrib/amcheck/expected/check_heap.out=0A+++=20= b/contrib/amcheck/expected/check_heap.out=0A@@=20-95,6=20+95,12=20@@=20= SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest');=0A=20= ERROR:=20=20permission=20denied=20for=20function=20verify_heapam=0A=20= RESET=20ROLE;=0A=20GRANT=20EXECUTE=20ON=20FUNCTION=20= verify_heapam(regclass,=20boolean,=20boolean,=20text,=20bigint,=20= bigint)=20TO=20regress_heaptest_role;=0A+--=20verify=20permissions=20are=20= checked=20(error=20due=20to=20no=20select=20privileges=20on=20relation)=0A= +SET=20ROLE=20regress_heaptest_role;=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest');=0A+ERROR:=20=20permission=20= denied=20for=20table=20heaptest=0A+RESET=20ROLE;=0A+GRANT=20SELECT=20ON=20= heaptest=20TO=20regress_heaptest_role;=0A=20--=20verify=20permissions=20= are=20now=20sufficient=0A=20SET=20ROLE=20regress_heaptest_role;=0A=20= SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest');=0Adiff=20= --git=20a/contrib/amcheck/sql/check_heap.sql=20= b/contrib/amcheck/sql/check_heap.sql=0Aindex=20c10a25f21c..c8397a46f0=20= 100644=0A---=20a/contrib/amcheck/sql/check_heap.sql=0A+++=20= b/contrib/amcheck/sql/check_heap.sql=0A@@=20-41,6=20+41,13=20@@=20RESET=20= ROLE;=0A=20=0A=20GRANT=20EXECUTE=20ON=20FUNCTION=20= verify_heapam(regclass,=20boolean,=20boolean,=20text,=20bigint,=20= bigint)=20TO=20regress_heaptest_role;=0A=20=0A+--=20verify=20permissions=20= are=20checked=20(error=20due=20to=20no=20select=20privileges=20on=20= relation)=0A+SET=20ROLE=20regress_heaptest_role;=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest');=0A+RESET=20ROLE;=0A+=0A= +GRANT=20SELECT=20ON=20heaptest=20TO=20regress_heaptest_role;=0A+=0A=20= --=20verify=20permissions=20are=20now=20sufficient=0A=20SET=20ROLE=20= regress_heaptest_role;=0A=20SELECT=20*=20FROM=20verify_heapam(relation=20= :=3D=20'heaptest');=0Adiff=20--git=20a/contrib/amcheck/verify_heapam.c=20= b/contrib/amcheck/verify_heapam.c=0Aindex=20263cbb37c1..f3fe9c44a4=20= 100644=0A---=20a/contrib/amcheck/verify_heapam.c=0A+++=20= b/contrib/amcheck/verify_heapam.c=0A@@=20-23,6=20+23,7=20@@=0A=20= #include=20"miscadmin.h"=0A=20#include=20"storage/bufmgr.h"=0A=20= #include=20"storage/procarray.h"=0A+#include=20"utils/acl.h"=0A=20= #include=20"utils/builtins.h"=0A=20#include=20"utils/fmgroids.h"=0A=20=0A= @@=20-478,6=20+479,8=20@@=20verify_heapam(PG_FUNCTION_ARGS)=0A=20static=20= void=0A=20sanity_check_relation(Relation=20rel)=0A=20{=0A+=09AclResult=09= aclresult;=0A+=0A=20=09if=20(rel->rd_rel->relkind=20!=3D=20= RELKIND_RELATION=20&&=0A=20=09=09rel->rd_rel->relkind=20!=3D=20= RELKIND_MATVIEW=20&&=0A=20=09=09rel->rd_rel->relkind=20!=3D=20= RELKIND_TOASTVALUE)=0A@@=20-489,6=20+492,11=20@@=20= sanity_check_relation(Relation=20rel)=0A=20=09=09ereport(ERROR,=0A=20=09=09= =09=09(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),=0A=20=09=09=09=09=20= errmsg("only=20heap=20AM=20is=20supported")));=0A+=09aclresult=20=3D=20= pg_class_aclcheck(rel->rd_id,=20GetUserId(),=20ACL_SELECT);=0A+=09if=20= (aclresult=20!=3D=20ACLCHECK_OK)=0A+=09=09aclcheck_error(aclresult,=0A+=09= =09=09=09=09=20=20=20get_relkind_objtype(rel->rd_rel->relkind),=0A+=09=09= =09=09=09=20=20=20RelationGetRelationName(rel));=0A=20}=0A=20=0A=20/*=0A= diff=20--git=20a/doc/src/sgml/pgamcheck.sgml=20= b/doc/src/sgml/pgamcheck.sgml=0Aindex=203e059e7753..fc36447dda=20100644=0A= ---=20a/doc/src/sgml/pgamcheck.sgml=0A+++=20= b/doc/src/sgml/pgamcheck.sgml=0A@@=20-19,7=20+19,7=20@@=0A=20=20=20= connecting=20as=20a=20user=20with=20sufficient=20privileges=20to=20check=20= tables=20and=20indexes.=0A=20=20=20Currently,=20this=20requires=20= execute=20privileges=20on=20's=0A=20=20=20= bt_index_parent_check=20and=20= verify_heapam=0A-=20=20functions.=0A+=20=20= functions,=20and=20on=20having=20privileges=20to=20access=20the=20= relations=20being=20checked.=0A=20=20=0A=20=0A=20=0A--=20= =0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 =E2=80=94 Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --Apple-Mail=_E73598DC-278E-453A-BB0B-6FA2286DD77F--