Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kPwG7-0000aT-46 for pgsql-hackers@arkaria.postgresql.org; Tue, 06 Oct 2020 23:20:39 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kPwG5-00084w-Qj for pgsql-hackers@arkaria.postgresql.org; Tue, 06 Oct 2020 23:20:37 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kPwG5-00084n-7A for pgsql-hackers@lists.postgresql.org; Tue, 06 Oct 2020 23:20:37 +0000 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kPwFv-0004gQ-UI for pgsql-hackers@postgresql.org; Tue, 06 Oct 2020 23:20:36 +0000 Received: by mail-pg1-x52e.google.com with SMTP id x16so210589pgj.3 for ; Tue, 06 Oct 2020 16:20:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=MFlKEBRw4SEbDKdzDGgRbgb2zPbL8IVLt88koObQBlc=; b=DZ+C0VPjNkEqmwnOWnXOieCeXM2Eifin385hCsiod2gHynGqXxRnz3KnPQ2szgxEmJ 8GdAnnwTChtXjLL2AGCpPE9dvJ+lS9VVty+18GGfxc38lwuM+n/romc7TJWVshVhUFmi dHzJDOLxPd2Hwu/NU9+Sg5c1JAYhxAzVKNSIWS6VTXvJBLZBeXHeIfDG230n+pe727xj Z2FunzttTayMDcPaZXKOkT9A/BWhhBlp+M0esxVoreSAgXRFeGoWjZhF70wGBtW2YefF DZkX8uu4+lEcJxjzAgmGDTZ51Lr15gPQZsCZ9UariF0R4QYqqzBBWhG/PEJBaa7BICP7 +TUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=MFlKEBRw4SEbDKdzDGgRbgb2zPbL8IVLt88koObQBlc=; b=BlMoJLNfNHZgt2H+YrJmrSqO5nL9jWr5NReFhDvKn6ze1Jzp/+FIoZq2bEyjrdzNvf VQGzWVIYKdAZ12OUO2wFJkaADbGc17/MjnS1DQP3gfPzTIKUvQPaO63yt/zx1DuFblRk CzUnm2tvcCNx11aVIbvI7Q2RXtwpeerMy8DylvWYB6E4q8Mc8rTBQGegTC3Up12lFhtB 8jd7JOmIOv6DeLcKU/CCx2OZEarQCX/iLtADL3ixnLmmQ3Cc8uOSpo9J2D6g03eHBU7p fyVbxkfMSYVdRIa7dmzTaQiTl3JqLMIP9Qt9CDskeCOuVvX/m3Dp3J78EE+zHuYL+eG2 C7DQ== X-Gm-Message-State: AOAM531KCNFriu0EKCH9iNdjYncMT3dJmyS523I7fxbrjkaYBI6T3ALn e9LwDTINolkeNwobprsCU4cG6wLSyE3CBn3QGtJjLKjp+ymsL87tJEd9nK+yiCj17vcH9WBw+aj fm6MaXRM1Ol6PUIQAclnm+uQqrE9QB6P4q9ydIrRAHv/HBTzu6122FzIjYOlSNyTX9YCq2vqe5w lvhvjmuqy5Rc9TNSjcOPH+B6b1DmUW4qXsVGgs X-Google-Smtp-Source: ABdhPJzqo0TNk0iNqoy+VPn5EbrMQ1X+jXSRBY+qeOCd7y8Zm3d1GnLY2XtCQBWlw+EhjSIyekoFYg== X-Received: by 2002:a63:1644:: with SMTP id 4mr497011pgw.232.1602026425190; Tue, 06 Oct 2020 16:20:25 -0700 (PDT) Received: from [192.168.0.29] (24-113-193-150.wavecable.com. [24.113.193.150]) by smtp.gmail.com with ESMTPSA id w23sm264061pfn.142.2020.10.06.16.20.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Oct 2020 16:20:24 -0700 (PDT) From: Mark Dilger Message-Id: <3894CAFE-E391-4EE0-9599-37C5D90BE740@enterprisedb.com> Content-Type: multipart/mixed; boundary="Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: new heapcheck contrib module Date: Tue, 6 Oct 2020 16:20:22 -0700 In-Reply-To: Cc: Amul Sul , Dilip Kumar , Andres Freund , Tom Lane , PostgreSQL-development To: Robert Haas , Peter Geoghegan , "Andrey M. Borodin" , Stephen Frost , Michael Paquier References: <12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com> <20200420194210.sevnpi4lyghoanxi@alap3.anarazel.de> <23E60C3C-C20A-4508-84AC-E0CEA59E784A@enterprisedb.com> <5F743835-3399-419C-8324-2D424237E999@enterprisedb.com> <885D2EC9-7D06-4903-A17D-72C184A1E1C5@enterprisedb.com> <5CC85F28-F210-47AC-8DA8-EC4E82AF3FD5@enterprisedb.com> <0C0349EB-DD85-4DA8-82C7-DF70AB714592@enterprisedb.com> <9DF93B83-FE46-4EC4-A99D-77A60729FD2D@enterprisedb.com> <50F6468F-5A5E-412A-86EC-DF24B549D733@enterprisedb.com> <7D603B53-8078-4AF0-851B-803FF4416501@enterprisedb.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 5, 2020, at 5:24 PM, Mark Dilger = wrote: >=20 > - This version does not change clog handling, which leaves Andrey's = concern unaddressed. Peter also showed some support for (or perhaps = just a lack of opposition to) doing more of what Andrey suggests. I may = come back to this issue, depending on time available and further = feedback. Attached is a patch set that includes the clog handling as discussed. = The 0001 and 0002 are effectively unchanged since version 16 posted = yesterday, but this now includes 0003 which creates a non-throwing = interface to clog, and 0004 which uses the non-throwing interface from = within amcheck's heap checking functions. I think this is a pretty good sketch for discussion, though I am = unsatisfied with the lack of regression test coverage of verify_heapam = in the presence of clog truncation. I was hoping to have that as part = of v17, but since it is taking a bit longer than I anticipated, I'll = have to come back with that in a later patch. --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479 Content-Disposition: attachment; filename=v17-0001-Adding-function-verify_heapam-to-amcheck-module.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v17-0001-Adding-function-verify_heapam-to-amcheck-module.patch" Content-Transfer-Encoding: quoted-printable =46rom=2019f858fb20f29cd716a0ce55f65193a0bde7c0c8=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Mon,=205=20Oct=202020=2015:42:18=20-0700=0ASubject:=20[PATCH=20= v17=201/4]=20Adding=20function=20verify_heapam=20to=20amcheck=20module=0A= =0AAdding=20new=20function=20verify_heapam=20for=20checking=20a=20heap=20= relation=20and=0Aoptionally=20its=20associated=20toast=20relation,=20if=20= any.=0A---=0A=20contrib/amcheck/Makefile=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=207=20+-=0A=20= contrib/amcheck/amcheck--1.2--1.3.sql=20=20=20|=20=20=2030=20+=0A=20= contrib/amcheck/amcheck.control=20=20=20=20=20=20=20=20=20|=20=20=20=202=20= +-=0A=20contrib/amcheck/expected/check_heap.out=20|=20=20200=20+++=0A=20= contrib/amcheck/sql/check_heap.sql=20=20=20=20=20=20|=20=20123=20++=0A=20= contrib/amcheck/t/001_verify_heapam.pl=20=20|=20=20242=20++++=0A=20= contrib/amcheck/verify_heapam.c=20=20=20=20=20=20=20=20=20|=201537=20= +++++++++++++++++++++++=0A=20doc/src/sgml/amcheck.sgml=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20|=20=20237=20+++-=0A=20= src/backend/access/heap/hio.c=20=20=20=20=20=20=20=20=20=20=20|=20=20=20= 11=20+=0A=20src/backend/access/transam/multixact.c=20=20|=20=20=2019=20+=0A= =20src/include/access/multixact.h=20=20=20=20=20=20=20=20=20=20|=20=20=20= =201=20+=0A=20src/tools/pgindent/typedefs.list=20=20=20=20=20=20=20=20|=20= =20=20=204=20+=0A=2012=20files=20changed,=202404=20insertions(+),=209=20= deletions(-)=0A=20create=20mode=20100644=20= contrib/amcheck/amcheck--1.2--1.3.sql=0A=20create=20mode=20100644=20= contrib/amcheck/expected/check_heap.out=0A=20create=20mode=20100644=20= contrib/amcheck/sql/check_heap.sql=0A=20create=20mode=20100644=20= contrib/amcheck/t/001_verify_heapam.pl=0A=20create=20mode=20100644=20= contrib/amcheck/verify_heapam.c=0A=0Adiff=20--git=20= a/contrib/amcheck/Makefile=20b/contrib/amcheck/Makefile=0Aindex=20= a2b1b1036b..b82f221e50=20100644=0A---=20a/contrib/amcheck/Makefile=0A+++=20= b/contrib/amcheck/Makefile=0A@@=20-3,13=20+3,16=20@@=0A=20MODULE_big=09=3D= =20amcheck=0A=20OBJS=20=3D=20\=0A=20=09$(WIN32RES)=20\=0A+=09= verify_heapam.o=20\=0A=20=09verify_nbtree.o=0A=20=0A=20EXTENSION=20=3D=20= amcheck=0A-DATA=20=3D=20amcheck--1.1--1.2.sql=20amcheck--1.0--1.1.sql=20= amcheck--1.0.sql=0A+DATA=20=3D=20amcheck--1.2--1.3.sql=20= amcheck--1.1--1.2.sql=20amcheck--1.0--1.1.sql=20amcheck--1.0.sql=0A=20= PGFILEDESC=20=3D=20"amcheck=20-=20function=20for=20verifying=20relation=20= integrity"=0A=20=0A-REGRESS=20=3D=20check=20check_btree=0A+REGRESS=20=3D=20= check=20check_btree=20check_heap=0A+=0A+TAP_TESTS=20=3D=201=0A=20=0A=20= ifdef=20USE_PGXS=0A=20PG_CONFIG=20=3D=20pg_config=0Adiff=20--git=20= a/contrib/amcheck/amcheck--1.2--1.3.sql=20= b/contrib/amcheck/amcheck--1.2--1.3.sql=0Anew=20file=20mode=20100644=0A= index=200000000000..7237ab738c=0A---=20/dev/null=0A+++=20= b/contrib/amcheck/amcheck--1.2--1.3.sql=0A@@=20-0,0=20+1,30=20@@=0A+/*=20= contrib/amcheck/amcheck--1.2--1.3.sql=20*/=0A+=0A+--=20complain=20if=20= script=20is=20sourced=20in=20psql,=20rather=20than=20via=20CREATE=20= EXTENSION=0A+\echo=20Use=20"ALTER=20EXTENSION=20amcheck=20UPDATE=20TO=20= '1.3'"=20to=20load=20this=20file.=20\quit=0A+=0A+--=0A+--=20= verify_heapam()=0A+--=0A+CREATE=20FUNCTION=20verify_heapam(relation=20= regclass,=0A+=09=09=09=09=09=09=09=20=20on_error_stop=20boolean=20= default=20false,=0A+=09=09=09=09=09=09=09=20=20check_toast=20boolean=20= default=20false,=0A+=09=09=09=09=09=09=09=20=20skip=20text=20default=20= 'none',=0A+=09=09=09=09=09=09=09=20=20startblock=20bigint=20default=20= null,=0A+=09=09=09=09=09=09=09=20=20endblock=20bigint=20default=20null,=0A= +=09=09=09=09=09=09=09=20=20blkno=20OUT=20bigint,=0A+=09=09=09=09=09=09=09= =20=20offnum=20OUT=20integer,=0A+=09=09=09=09=09=09=09=20=20attnum=20OUT=20= integer,=0A+=09=09=09=09=09=09=09=20=20msg=20OUT=20text)=0A+RETURNS=20= SETOF=20record=0A+AS=20'MODULE_PATHNAME',=20'verify_heapam'=0A+LANGUAGE=20= C;=0A+=0A+--=20Don't=20want=20this=20to=20be=20available=20to=20public=0A= +REVOKE=20ALL=20ON=20FUNCTION=20verify_heapam(regclass,=0A+=09=09=09=09=09= =09=09=09=09=20boolean,=0A+=09=09=09=09=09=09=09=09=09=20boolean,=0A+=09=09= =09=09=09=09=09=09=09=20text,=0A+=09=09=09=09=09=09=09=09=09=20bigint,=0A= +=09=09=09=09=09=09=09=09=09=20bigint)=0A+FROM=20PUBLIC;=0Adiff=20--git=20= a/contrib/amcheck/amcheck.control=20b/contrib/amcheck/amcheck.control=0A= index=20c6e310046d..ab50931f75=20100644=0A---=20= a/contrib/amcheck/amcheck.control=0A+++=20= b/contrib/amcheck/amcheck.control=0A@@=20-1,5=20+1,5=20@@=0A=20#=20= amcheck=20extension=0A=20comment=20=3D=20'functions=20for=20verifying=20= relation=20integrity'=0A-default_version=20=3D=20'1.2'=0A= +default_version=20=3D=20'1.3'=0A=20module_pathname=20=3D=20= '$libdir/amcheck'=0A=20relocatable=20=3D=20true=0Adiff=20--git=20= a/contrib/amcheck/expected/check_heap.out=20= b/contrib/amcheck/expected/check_heap.out=0Anew=20file=20mode=20100644=0A= index=200000000000..41cdc6435c=0A---=20/dev/null=0A+++=20= b/contrib/amcheck/expected/check_heap.out=0A@@=20-0,0=20+1,200=20@@=0A= +CREATE=20TABLE=20heaptest=20(a=20integer,=20b=20text);=0A+REVOKE=20ALL=20= ON=20heaptest=20FROM=20PUBLIC;=0A+--=20Check=20that=20invalid=20skip=20= option=20is=20rejected=0A+SELECT=20*=20FROM=20verify_heapam(relation=20= :=3D=20'heaptest',=20skip=20:=3D=20'rope');=0A+ERROR:=20=20invalid=20= skip=20option=0A+HINT:=20=20Valid=20skip=20options=20are=20= "all-visible",=20"all-frozen",=20and=20"none".=0A+--=20Check=20= specifying=20invalid=20block=20ranges=20when=20verifying=20an=20empty=20= table=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20= startblock=20:=3D=200,=20endblock=20:=3D=200);=0A+=20blkno=20|=20offnum=20= |=20attnum=20|=20msg=20=0A+-------+--------+--------+-----=0A+(0=20rows)=0A= +=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20= startblock=20:=3D=205,=20endblock=20:=3D=208);=0A+=20blkno=20|=20offnum=20= |=20attnum=20|=20msg=20=0A+-------+--------+--------+-----=0A+(0=20rows)=0A= +=0A+--=20Check=20that=20valid=20options=20are=20not=20rejected=20nor=20= corruption=20reported=0A+--=20for=20an=20empty=20table,=20and=20that=20= skip=20enum-like=20parameter=20is=20case-insensitive=0A+SELECT=20*=20= FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'none');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-frozen');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-visible');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20'None');=0A+=20= blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'All-Frozen');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'All-Visible');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20'NONE');=0A+=20= blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'ALL-FROZEN');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'ALL-VISIBLE');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+--=20Add=20some=20= data=20so=20subsequent=20tests=20are=20not=20entirely=20trivial=0A= +INSERT=20INTO=20heaptest=20(a,=20b)=0A+=09(SELECT=20gs,=20repeat('x',=20= gs)=0A+=09=09FROM=20generate_series(1,50)=20gs);=0A+--=20Check=20that=20= valid=20options=20are=20not=20rejected=20nor=20corruption=20reported=0A= +--=20for=20a=20non-empty=20table=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20'none');=0A+=20= blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-frozen');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-visible');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=200,=20= endblock=20:=3D=200);=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+CREATE=20ROLE=20= regress_heaptest_role;=0A+--=20verify=20permissions=20are=20checked=20= (error=20due=20to=20function=20not=20callable)=0A+SET=20ROLE=20= regress_heaptest_role;=0A+SELECT=20*=20FROM=20verify_heapam(relation=20= :=3D=20'heaptest');=0A+ERROR:=20=20permission=20denied=20for=20function=20= verify_heapam=0A+RESET=20ROLE;=0A+GRANT=20EXECUTE=20ON=20FUNCTION=20= verify_heapam(regclass,=20boolean,=20boolean,=20text,=20bigint,=20= bigint)=20TO=20regress_heaptest_role;=0A+--=20verify=20permissions=20are=20= checked=20(error=20due=20to=20no=20select=20privileges=20on=20relation)=0A= +SET=20ROLE=20regress_heaptest_role;=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest');=0A+ERROR:=20=20permission=20= denied=20for=20table=20heaptest=0A+RESET=20ROLE;=0A+GRANT=20SELECT=20ON=20= heaptest=20TO=20regress_heaptest_role;=0A+--=20verify=20permissions=20= are=20now=20sufficient=0A+SET=20ROLE=20regress_heaptest_role;=0A+SELECT=20= *=20FROM=20verify_heapam(relation=20:=3D=20'heaptest');=0A+=20blkno=20|=20= offnum=20|=20attnum=20|=20msg=20=0A+-------+--------+--------+-----=0A= +(0=20rows)=0A+=0A+RESET=20ROLE;=0A+--=20Check=20specifying=20invalid=20= block=20ranges=20when=20verifying=20a=20non-empty=20table.=0A+SELECT=20*=20= FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=20= 0,=20endblock=20:=3D=2010000);=0A+ERROR:=20=20ending=20block=20number=20= must=20be=20between=200=20and=200=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=2010000,=20= endblock=20:=3D=2011000);=0A+ERROR:=20=20starting=20block=20number=20= must=20be=20between=200=20and=200=0A+--=20Vacuum=20freeze=20to=20change=20= the=20xids=20encountered=20in=20subsequent=20tests=0A+VACUUM=20FREEZE=20= heaptest;=0A+--=20Check=20that=20valid=20options=20are=20not=20rejected=20= nor=20corruption=20reported=0A+--=20for=20a=20non-empty=20frozen=20table=0A= +SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20skip=20= :=3D=20'none');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-frozen');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-visible');=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=200,=20= endblock=20:=3D=200);=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+--=20Check=20that=20= partitioned=20tables=20(the=20parent=20ones)=20which=20don't=20have=20= visibility=0A+--=20maps=20are=20rejected=0A+CREATE=20TABLE=20= test_partitioned=20(a=20int,=20b=20text=20default=20repeat('x',=205000))=0A= +=09=09=09=20PARTITION=20BY=20list=20(a);=0A+SELECT=20*=20FROM=20= verify_heapam('test_partitioned',=0A+=09=09=09=09=09=09=09startblock=20= :=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+ERROR:=20= =20"test_partitioned"=20is=20not=20a=20table,=20materialized=20view,=20= or=20TOAST=20table=0A+--=20Check=20that=20valid=20options=20are=20not=20= rejected=20nor=20corruption=20reported=0A+--=20for=20an=20empty=20= partition=20table=20(the=20child=20one)=0A+CREATE=20TABLE=20= test_partition=20partition=20OF=20test_partitioned=20FOR=20VALUES=20IN=20= (1);=0A+SELECT=20*=20FROM=20verify_heapam('test_partition',=0A+=09=09=09=09= =09=09=09startblock=20:=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D= =20NULL);=0A+=20blkno=20|=20offnum=20|=20attnum=20|=20msg=20=0A= +-------+--------+--------+-----=0A+(0=20rows)=0A+=0A+--=20Check=20that=20= valid=20options=20are=20not=20rejected=20nor=20corruption=20reported=0A= +--=20for=20a=20non-empty=20partition=20table=20(the=20child=20one)=0A= +INSERT=20INTO=20test_partitioned=20(a)=20(SELECT=201=20FROM=20= generate_series(1,1000)=20gs);=0A+SELECT=20*=20FROM=20= verify_heapam('test_partition',=0A+=09=09=09=09=09=09=09startblock=20:=3D=20= NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+=20blkno=20|=20= offnum=20|=20attnum=20|=20msg=20=0A+-------+--------+--------+-----=0A= +(0=20rows)=0A+=0A+--=20Check=20that=20indexes=20are=20rejected=0A= +CREATE=20INDEX=20test_index=20ON=20test_partition=20(a);=0A+SELECT=20*=20= FROM=20verify_heapam('test_index',=0A+=09=09=09=09=09=09=09startblock=20= :=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+ERROR:=20= =20"test_index"=20is=20not=20a=20table,=20materialized=20view,=20or=20= TOAST=20table=0A+--=20Check=20that=20views=20are=20rejected=0A+CREATE=20= VIEW=20test_view=20AS=20SELECT=201;=0A+SELECT=20*=20FROM=20= verify_heapam('test_view',=0A+=09=09=09=09=09=09=09startblock=20:=3D=20= NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+ERROR:=20=20= "test_view"=20is=20not=20a=20table,=20materialized=20view,=20or=20TOAST=20= table=0A+--=20Check=20that=20sequences=20are=20rejected=0A+CREATE=20= SEQUENCE=20test_sequence;=0A+SELECT=20*=20FROM=20= verify_heapam('test_sequence',=0A+=09=09=09=09=09=09=09startblock=20:=3D=20= NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+ERROR:=20=20= "test_sequence"=20is=20not=20a=20table,=20materialized=20view,=20or=20= TOAST=20table=0A+--=20Check=20that=20foreign=20tables=20are=20rejected=0A= +CREATE=20FOREIGN=20DATA=20WRAPPER=20dummy;=0A+CREATE=20SERVER=20= dummy_server=20FOREIGN=20DATA=20WRAPPER=20dummy;=0A+CREATE=20FOREIGN=20= TABLE=20test_foreign_table=20()=20SERVER=20dummy_server;=0A+SELECT=20*=20= FROM=20verify_heapam('test_foreign_table',=0A+=09=09=09=09=09=09=09= startblock=20:=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20= NULL);=0A+ERROR:=20=20"test_foreign_table"=20is=20not=20a=20table,=20= materialized=20view,=20or=20TOAST=20table=0A+--=20cleanup=0A+DROP=20= TABLE=20heaptest;=0A+DROP=20TABLE=20test_partition;=0A+DROP=20TABLE=20= test_partitioned;=0A+DROP=20OWNED=20BY=20regress_heaptest_role;=20--=20= permissions=0A+DROP=20ROLE=20regress_heaptest_role;=0Adiff=20--git=20= a/contrib/amcheck/sql/check_heap.sql=20= b/contrib/amcheck/sql/check_heap.sql=0Anew=20file=20mode=20100644=0A= index=200000000000..c8397a46f0=0A---=20/dev/null=0A+++=20= b/contrib/amcheck/sql/check_heap.sql=0A@@=20-0,0=20+1,123=20@@=0A+CREATE=20= TABLE=20heaptest=20(a=20integer,=20b=20text);=0A+REVOKE=20ALL=20ON=20= heaptest=20FROM=20PUBLIC;=0A+=0A+--=20Check=20that=20invalid=20skip=20= option=20is=20rejected=0A+SELECT=20*=20FROM=20verify_heapam(relation=20= :=3D=20'heaptest',=20skip=20:=3D=20'rope');=0A+=0A+--=20Check=20= specifying=20invalid=20block=20ranges=20when=20verifying=20an=20empty=20= table=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20= startblock=20:=3D=200,=20endblock=20:=3D=200);=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=205,=20= endblock=20:=3D=208);=0A+=0A+--=20Check=20that=20valid=20options=20are=20= not=20rejected=20nor=20corruption=20reported=0A+--=20for=20an=20empty=20= table,=20and=20that=20skip=20enum-like=20parameter=20is=20= case-insensitive=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20= 'heaptest',=20skip=20:=3D=20'none');=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-frozen');=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20= 'heaptest',=20skip=20:=3D=20'all-visible');=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20'None');=0A= +SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20skip=20= :=3D=20'All-Frozen');=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D= =20'heaptest',=20skip=20:=3D=20'All-Visible');=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20'NONE');=0A= +SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20skip=20= :=3D=20'ALL-FROZEN');=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D= =20'heaptest',=20skip=20:=3D=20'ALL-VISIBLE');=0A+=0A+--=20Add=20some=20= data=20so=20subsequent=20tests=20are=20not=20entirely=20trivial=0A= +INSERT=20INTO=20heaptest=20(a,=20b)=0A+=09(SELECT=20gs,=20repeat('x',=20= gs)=0A+=09=09FROM=20generate_series(1,50)=20gs);=0A+=0A+--=20Check=20= that=20valid=20options=20are=20not=20rejected=20nor=20corruption=20= reported=0A+--=20for=20a=20non-empty=20table=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20'none');=0A= +SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20skip=20= :=3D=20'all-frozen');=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D= =20'heaptest',=20skip=20:=3D=20'all-visible');=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=200,=20= endblock=20:=3D=200);=0A+=0A+CREATE=20ROLE=20regress_heaptest_role;=0A+=0A= +--=20verify=20permissions=20are=20checked=20(error=20due=20to=20= function=20not=20callable)=0A+SET=20ROLE=20regress_heaptest_role;=0A= +SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest');=0A= +RESET=20ROLE;=0A+=0A+GRANT=20EXECUTE=20ON=20FUNCTION=20= verify_heapam(regclass,=20boolean,=20boolean,=20text,=20bigint,=20= bigint)=20TO=20regress_heaptest_role;=0A+=0A+--=20verify=20permissions=20= are=20checked=20(error=20due=20to=20no=20select=20privileges=20on=20= relation)=0A+SET=20ROLE=20regress_heaptest_role;=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest');=0A+RESET=20ROLE;=0A+=0A= +GRANT=20SELECT=20ON=20heaptest=20TO=20regress_heaptest_role;=0A+=0A+--=20= verify=20permissions=20are=20now=20sufficient=0A+SET=20ROLE=20= regress_heaptest_role;=0A+SELECT=20*=20FROM=20verify_heapam(relation=20= :=3D=20'heaptest');=0A+RESET=20ROLE;=0A+=0A+--=20Check=20specifying=20= invalid=20block=20ranges=20when=20verifying=20a=20non-empty=20table.=0A= +SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20'heaptest',=20= startblock=20:=3D=200,=20endblock=20:=3D=2010000);=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=2010000,=20= endblock=20:=3D=2011000);=0A+=0A+--=20Vacuum=20freeze=20to=20change=20= the=20xids=20encountered=20in=20subsequent=20tests=0A+VACUUM=20FREEZE=20= heaptest;=0A+=0A+--=20Check=20that=20valid=20options=20are=20not=20= rejected=20nor=20corruption=20reported=0A+--=20for=20a=20non-empty=20= frozen=20table=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20= 'heaptest',=20skip=20:=3D=20'none');=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20skip=20:=3D=20= 'all-frozen');=0A+SELECT=20*=20FROM=20verify_heapam(relation=20:=3D=20= 'heaptest',=20skip=20:=3D=20'all-visible');=0A+SELECT=20*=20FROM=20= verify_heapam(relation=20:=3D=20'heaptest',=20startblock=20:=3D=200,=20= endblock=20:=3D=200);=0A+=0A+--=20Check=20that=20partitioned=20tables=20= (the=20parent=20ones)=20which=20don't=20have=20visibility=0A+--=20maps=20= are=20rejected=0A+CREATE=20TABLE=20test_partitioned=20(a=20int,=20b=20= text=20default=20repeat('x',=205000))=0A+=09=09=09=20PARTITION=20BY=20= list=20(a);=0A+SELECT=20*=20FROM=20verify_heapam('test_partitioned',=0A+=09= =09=09=09=09=09=09startblock=20:=3D=20NULL,=0A+=09=09=09=09=09=09=09= endblock=20:=3D=20NULL);=0A+=0A+--=20Check=20that=20valid=20options=20= are=20not=20rejected=20nor=20corruption=20reported=0A+--=20for=20an=20= empty=20partition=20table=20(the=20child=20one)=0A+CREATE=20TABLE=20= test_partition=20partition=20OF=20test_partitioned=20FOR=20VALUES=20IN=20= (1);=0A+SELECT=20*=20FROM=20verify_heapam('test_partition',=0A+=09=09=09=09= =09=09=09startblock=20:=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D= =20NULL);=0A+=0A+--=20Check=20that=20valid=20options=20are=20not=20= rejected=20nor=20corruption=20reported=0A+--=20for=20a=20non-empty=20= partition=20table=20(the=20child=20one)=0A+INSERT=20INTO=20= test_partitioned=20(a)=20(SELECT=201=20FROM=20generate_series(1,1000)=20= gs);=0A+SELECT=20*=20FROM=20verify_heapam('test_partition',=0A+=09=09=09=09= =09=09=09startblock=20:=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D= =20NULL);=0A+=0A+--=20Check=20that=20indexes=20are=20rejected=0A+CREATE=20= INDEX=20test_index=20ON=20test_partition=20(a);=0A+SELECT=20*=20FROM=20= verify_heapam('test_index',=0A+=09=09=09=09=09=09=09startblock=20:=3D=20= NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+=0A+--=20Check=20= that=20views=20are=20rejected=0A+CREATE=20VIEW=20test_view=20AS=20SELECT=20= 1;=0A+SELECT=20*=20FROM=20verify_heapam('test_view',=0A+=09=09=09=09=09=09= =09startblock=20:=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20= NULL);=0A+=0A+--=20Check=20that=20sequences=20are=20rejected=0A+CREATE=20= SEQUENCE=20test_sequence;=0A+SELECT=20*=20FROM=20= verify_heapam('test_sequence',=0A+=09=09=09=09=09=09=09startblock=20:=3D=20= NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+=0A+--=20Check=20= that=20foreign=20tables=20are=20rejected=0A+CREATE=20FOREIGN=20DATA=20= WRAPPER=20dummy;=0A+CREATE=20SERVER=20dummy_server=20FOREIGN=20DATA=20= WRAPPER=20dummy;=0A+CREATE=20FOREIGN=20TABLE=20test_foreign_table=20()=20= SERVER=20dummy_server;=0A+SELECT=20*=20FROM=20= verify_heapam('test_foreign_table',=0A+=09=09=09=09=09=09=09startblock=20= :=3D=20NULL,=0A+=09=09=09=09=09=09=09endblock=20:=3D=20NULL);=0A+=0A+--=20= cleanup=0A+DROP=20TABLE=20heaptest;=0A+DROP=20TABLE=20test_partition;=0A= +DROP=20TABLE=20test_partitioned;=0A+DROP=20OWNED=20BY=20= regress_heaptest_role;=20--=20permissions=0A+DROP=20ROLE=20= regress_heaptest_role;=0Adiff=20--git=20= a/contrib/amcheck/t/001_verify_heapam.pl=20= b/contrib/amcheck/t/001_verify_heapam.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..e7526c17b8=0A---=20/dev/null=0A+++=20= b/contrib/amcheck/t/001_verify_heapam.pl=0A@@=20-0,0=20+1,242=20@@=0A= +use=20strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20= TestLib;=0A+=0A+use=20Test::More=20tests=20=3D>=2065;=0A+=0A+my=20= ($node,=20$result);=0A+=0A+#=0A+#=20Test=20set-up=0A+#=0A+$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A= +$node->append_conf('postgresql.conf',=20'autovacuum=3Doff');=0A= +$node->start;=0A+$node->safe_psql('postgres',=20q(CREATE=20EXTENSION=20= amcheck));=0A+=0A+#=0A+#=20Check=20a=20table=20with=20data=20loaded=20= but=20no=20corruption,=20freezing,=20etc.=0A+#=0A= +fresh_test_table('test');=0A+check_all_options_uncorrupted('test',=20= 'plain');=0A+=0A+#=0A+#=20Check=20a=20corrupt=20table=0A+#=0A= +fresh_test_table('test');=0A+corrupt_first_page('test');=0A= +detects_corruption(=0A+=09"verify_heapam('test')",=0A+=09"plain=20= corrupted=20table");=0A+detects_corruption(=0A+=09"verify_heapam('test',=20= skip=20:=3D=20'all-visible')",=0A+=09"plain=20corrupted=20table=20= skipping=20all-visible");=0A+detects_corruption(=0A+=09= "verify_heapam('test',=20skip=20:=3D=20'all-frozen')",=0A+=09"plain=20= corrupted=20table=20skipping=20all-frozen");=0A+detects_corruption(=0A+=09= "verify_heapam('test',=20check_toast=20:=3D=20false)",=0A+=09"plain=20= corrupted=20table=20skipping=20toast");=0A+detects_corruption(=0A+=09= "verify_heapam('test',=20startblock=20:=3D=200,=20endblock=20:=3D=200)",=0A= +=09"plain=20corrupted=20table=20checking=20only=20block=20zero");=0A+=0A= +#=0A+#=20Check=20a=20corrupt=20table=20with=20all-frozen=20data=0A+#=0A= +fresh_test_table('test');=0A+$node->safe_psql('postgres',=20q(VACUUM=20= FREEZE=20test));=0A+corrupt_first_page('test');=0A+detects_corruption(=0A= +=09"verify_heapam('test')",=0A+=09"all-frozen=20corrupted=20table");=0A= +detects_no_corruption(=0A+=09"verify_heapam('test',=20skip=20:=3D=20= 'all-frozen')",=0A+=09"all-frozen=20corrupted=20table=20skipping=20= all-frozen");=0A+=0A+#=0A+#=20Check=20a=20corrupt=20table=20with=20= corrupt=20page=20header=0A+#=0A+fresh_test_table('test');=0A= +corrupt_first_page_and_header('test');=0A+detects_corruption(=0A+=09= "verify_heapam('test')",=0A+=09"corrupted=20test=20table=20with=20bad=20= page=20header");=0A+=0A+#=0A+#=20Check=20an=20uncorrupted=20table=20with=20= corrupt=20toast=20page=20header=0A+#=0A+fresh_test_table('test');=0A+my=20= $toast=20=3D=20get_toast_for('test');=0A= +corrupt_first_page_and_header($toast);=0A+detects_corruption(=0A+=09= "verify_heapam('test',=20check_toast=20:=3D=20true)",=0A+=09"table=20= with=20corrupted=20toast=20page=20header=20checking=20toast");=0A= +detects_no_corruption(=0A+=09"verify_heapam('test',=20check_toast=20:=3D=20= false)",=0A+=09"table=20with=20corrupted=20toast=20page=20header=20= skipping=20toast");=0A+detects_corruption(=0A+=09= "verify_heapam('$toast')",=0A+=09"corrupted=20toast=20page=20header");=0A= +=0A+#=0A+#=20Check=20an=20uncorrupted=20table=20with=20corrupt=20toast=0A= +#=0A+fresh_test_table('test');=0A+$toast=20=3D=20get_toast_for('test');=0A= +corrupt_first_page($toast);=0A+detects_corruption(=0A+=09= "verify_heapam('test',=20check_toast=20:=3D=20true)",=0A+=09"table=20= with=20corrupted=20toast=20checking=20toast");=0A+detects_no_corruption(=0A= +=09"verify_heapam('test',=20check_toast=20:=3D=20false)",=0A+=09"table=20= with=20corrupted=20toast=20skipping=20toast");=0A+detects_corruption(=0A= +=09"verify_heapam('$toast')",=0A+=09"corrupted=20toast=20table");=0A+=0A= +#=0A+#=20Check=20an=20uncorrupted=20all-frozen=20table=20with=20corrupt=20= toast=0A+#=0A+fresh_test_table('test');=0A+$node->safe_psql('postgres',=20= q(VACUUM=20FREEZE=20test));=0A+$toast=20=3D=20get_toast_for('test');=0A= +corrupt_first_page($toast);=0A+detects_corruption(=0A+=09= "verify_heapam('test',=20check_toast=20:=3D=20true)",=0A+=09"all-frozen=20= table=20with=20corrupted=20toast=20checking=20toast");=0A= +detects_no_corruption(=0A+=09"verify_heapam('test',=20check_toast=20:=3D=20= false)",=0A+=09"all-frozen=20table=20with=20corrupted=20toast=20skipping=20= toast");=0A+detects_corruption(=0A+=09"verify_heapam('$toast')",=0A+=09= "corrupted=20toast=20table=20of=20all-frozen=20table");=0A+=0A+#=20= Returns=20the=20filesystem=20path=20for=20the=20named=20relation.=0A+sub=20= relation_filepath=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A+=0A+=09my=20= $pgdata=20=3D=20$node->data_dir;=0A+=09my=20$rel=20=3D=20= $node->safe_psql('postgres',=0A+=09=09=09=09=09=09=09=20=20=20qq(SELECT=20= pg_relation_filepath('$relname')));=0A+=09die=20"path=20not=20found=20= for=20relation=20$relname"=20unless=20defined=20$rel;=0A+=09return=20= "$pgdata/$rel";=0A+}=0A+=0A+#=20Returns=20the=20fully=20qualified=20name=20= of=20the=20toast=20table=20for=20the=20named=20relation=0A+sub=20= get_toast_for=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A+=09= $node->safe_psql('postgres',=20qq(=0A+=09=09SELECT=20'pg_toast.'=20||=20= t.relname=0A+=09=09=09FROM=20pg_catalog.pg_class=20c,=20= pg_catalog.pg_class=20t=0A+=09=09=09WHERE=20c.relname=20=3D=20'$relname'=0A= +=09=09=09=20=20AND=20c.reltoastrelid=20=3D=20t.oid));=0A+}=0A+=0A+#=20= (Re)create=20and=20populate=20a=20test=20table=20of=20the=20given=20= name.=0A+sub=20fresh_test_table=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A= +=09$node->safe_psql('postgres',=20qq(=0A+=09=09DROP=20TABLE=20IF=20= EXISTS=20$relname=20CASCADE;=0A+=09=09CREATE=20TABLE=20$relname=20(a=20= integer,=20b=20text);=0A+=09=09ALTER=20TABLE=20$relname=20SET=20= (autovacuum_enabled=3Dfalse);=0A+=09=09ALTER=20TABLE=20$relname=20ALTER=20= b=20SET=20STORAGE=20external;=0A+=09=09INSERT=20INTO=20$relname=20(a,=20= b)=0A+=09=09=09(SELECT=20gs,=20repeat('b',gs*10)=20FROM=20= generate_series(1,1000)=20gs);=0A+=09));=0A+}=0A+=0A+#=20Stops=20the=20= test=20node,=20corrupts=20the=20first=20page=20of=20the=20named=20= relation,=20and=0A+#=20restarts=20the=20node.=0A+sub=20= corrupt_first_page_internal=0A+{=0A+=09my=20($relname,=20= $corrupt_header)=20=3D=20@_;=0A+=09my=20$relpath=20=3D=20= relation_filepath($relname);=0A+=0A+=09$node->stop;=0A+=09my=20$fh;=0A+=09= open($fh,=20'+<',=20$relpath);=0A+=09binmode=20$fh;=0A+=0A+=09#=20If=20= we=20corrupt=20the=20header,=20postgres=20won't=20allow=20the=20page=20= into=20the=20buffer.=0A+=09syswrite($fh,=20'\xFF\xFF\xFF\xFF',=208)=20if=20= ($corrupt_header);=0A+=0A+=09#=20Corrupt=20at=20least=20the=20line=20= pointers.=20=20Exactly=20what=20this=20corrupts=20will=0A+=09#=20depend=20= on=20the=20page,=20as=20it=20may=20run=20past=20the=20line=20pointers=20= into=20the=20user=0A+=09#=20data.=20=20We=20stop=20short=20of=20writing=20= 2048=20bytes=20(2k),=20the=20smallest=20supported=0A+=09#=20page=20size,=20= as=20we=20don't=20want=20to=20corrupt=20the=20next=20page.=0A+=09= seek($fh,=2032,=200);=0A+=09syswrite($fh,=20'\x77\x77\x77\x77',=20500);=0A= +=09close($fh);=0A+=09$node->start;=0A+}=0A+=0A+sub=20corrupt_first_page=0A= +{=0A+=09corrupt_first_page_internal($_[0],=20undef);=0A+}=0A+=0A+sub=20= corrupt_first_page_and_header=0A+{=0A+=09= corrupt_first_page_internal($_[0],=201);=0A+}=0A+=0A+sub=20= detects_corruption=0A+{=0A+=09my=20($function,=20$testname)=20=3D=20@_;=0A= +=0A+=09my=20$result=20=3D=20$node->safe_psql('postgres',=0A+=09=09= qq(SELECT=20COUNT(*)=20>=200=20FROM=20$function));=0A+=09is($result,=20= 't',=20$testname);=0A+}=0A+=0A+sub=20detects_no_corruption=0A+{=0A+=09my=20= ($function,=20$testname)=20=3D=20@_;=0A+=0A+=09my=20$result=20=3D=20= $node->safe_psql('postgres',=0A+=09=09qq(SELECT=20COUNT(*)=20=3D=200=20= FROM=20$function));=0A+=09is($result,=20't',=20$testname);=0A+}=0A+=0A+#=20= Check=20various=20options=20are=20stable=20(don't=20abort)=20and=20do=20= not=20report=20corruption=0A+#=20when=20running=20verify_heapam=20on=20= an=20uncorrupted=20test=20table.=0A+#=0A+#=20The=20relname=20*must*=20be=20= an=20uncorrupted=20table,=20or=20this=20will=20fail.=0A+#=0A+#=20The=20= prefix=20is=20used=20to=20identify=20the=20test,=20along=20with=20the=20= options,=0A+#=20and=20should=20be=20unique.=0A+sub=20= check_all_options_uncorrupted=0A+{=0A+=09my=20($relname,=20$prefix)=20=3D=20= @_;=0A+=09for=20my=20$stop=20(qw(true=20false))=0A+=09{=0A+=09=09for=20= my=20$check_toast=20(qw(true=20false))=0A+=09=09{=0A+=09=09=09for=20my=20= $skip=20("'none'",=20"'all-frozen'",=20"'all-visible'")=0A+=09=09=09{=0A= +=09=09=09=09for=20my=20$startblock=20(qw(NULL=200))=0A+=09=09=09=09{=0A= +=09=09=09=09=09for=20my=20$endblock=20(qw(NULL=200))=0A+=09=09=09=09=09= {=0A+=09=09=09=09=09=09my=20$opts=20=3D=20"on_error_stop=20:=3D=20$stop,=20= "=20.=0A+=09=09=09=09=09=09=09=09=20=20=20"check_toast=20:=3D=20= $check_toast,=20"=20.=0A+=09=09=09=09=09=09=09=09=20=20=20"skip=20:=3D=20= $skip,=20"=20.=0A+=09=09=09=09=09=09=09=09=20=20=20"startblock=20:=3D=20= $startblock,=20"=20.=0A+=09=09=09=09=09=09=09=09=20=20=20"endblock=20:=3D=20= $endblock";=0A+=0A+=09=09=09=09=09=09detects_no_corruption(=0A+=09=09=09=09= =09=09=09"verify_heapam('$relname',=20$opts)",=0A+=09=09=09=09=09=09=09= "$prefix:=20$opts");=0A+=09=09=09=09=09}=0A+=09=09=09=09}=0A+=09=09=09}=0A= +=09=09}=0A+=09}=0A+}=0Adiff=20--git=20a/contrib/amcheck/verify_heapam.c=20= b/contrib/amcheck/verify_heapam.c=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..b46cce798e=0A---=20/dev/null=0A+++=20= b/contrib/amcheck/verify_heapam.c=0A@@=20-0,0=20+1,1537=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20verify_heapam.c=0A+=20*=09=20=20Functions=20to=20= check=20postgresql=20heap=20relations=20for=20corruption=0A+=20*=0A+=20*=20= Copyright=20(c)=202016-2020,=20PostgreSQL=20Global=20Development=20Group=0A= +=20*=0A+=20*=09=20=20contrib/amcheck/verify_heapam.c=0A+=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres.h"=0A+=0A+#include=20= "access/detoast.h"=0A+#include=20"access/genam.h"=0A+#include=20= "access/heapam.h"=0A+#include=20"access/heaptoast.h"=0A+#include=20= "access/multixact.h"=0A+#include=20"access/toast_internals.h"=0A= +#include=20"access/visibilitymap.h"=0A+#include=20"catalog/pg_am.h"=0A= +#include=20"funcapi.h"=0A+#include=20"miscadmin.h"=0A+#include=20= "storage/bufmgr.h"=0A+#include=20"storage/procarray.h"=0A+#include=20= "utils/acl.h"=0A+#include=20"utils/builtins.h"=0A+#include=20= "utils/fmgroids.h"=0A+=0A+PG_FUNCTION_INFO_V1(verify_heapam);=0A+=0A+/*=20= The=20number=20of=20columns=20in=20tuples=20returned=20by=20= verify_heapam=20*/=0A+#define=20HEAPCHECK_RELATION_COLS=204=0A+=0A= +typedef=20enum=20XidBoundsViolation=0A+{=0A+=09XID_INVALID,=0A+=09= XID_IN_FUTURE,=0A+=09XID_PRECEDES_DATMIN,=0A+=09XID_PRECEDES_RELMIN,=0A+=09= XID_BOUNDS_OK=0A+}=20XidBoundsViolation;=0A+=0A+typedef=20enum=20= XidCommitStatus=0A+{=0A+=09XID_COMMITTED,=0A+=09XID_IN_PROGRESS,=0A+=09= XID_ABORTED=0A+}=20XidCommitStatus;=0A+=0A+typedef=20enum=20SkipPages=0A= +{=0A+=09SKIP_PAGES_ALL_FROZEN,=0A+=09SKIP_PAGES_ALL_VISIBLE,=0A+=09= SKIP_PAGES_NONE=0A+}=20SkipPages;=0A+=0A+/*=0A+=20*=20Struct=20holding=20= the=20running=20context=20information=20during=0A+=20*=20a=20lifetime=20= of=20a=20verify_heapam=20execution.=0A+=20*/=0A+typedef=20struct=20= HeapCheckContext=0A+{=0A+=09/*=0A+=09=20*=20Cached=20copies=20of=20= values=20from=20ShmemVariableCache=20and=20computed=20values=0A+=09=20*=20= from=20them.=0A+=09=20*/=0A+=09FullTransactionId=20next_fxid;=09/*=20= ShmemVariableCache->nextXid=20*/=0A+=09TransactionId=20next_xid;=09=09/*=20= 32-bit=20version=20of=20next_fxid=20*/=0A+=09TransactionId=20oldest_xid;=09= /*=20ShmemVariableCache->oldestXid=20*/=0A+=09FullTransactionId=20= oldest_fxid;=09/*=2064-bit=20version=20of=20oldest_xid,=20computed=0A+=09= =09=09=09=09=09=09=09=09=20*=20relative=20to=20next_fxid=20*/=0A+=0A+=09= /*=0A+=09=20*=20Cached=20copy=20of=20value=20from=20MultiXactState=0A+=09= =20*/=0A+=09MultiXactId=20next_mxact;=09=09/*=20= MultiXactState->nextMXact=20*/=0A+=09MultiXactId=20oldest_mxact;=09/*=20= MultiXactState->oldestMultiXactId=20*/=0A+=0A+=09/*=0A+=09=20*=20Cached=20= copies=20of=20the=20most=20recently=20checked=20xid=20and=20its=20= status.=0A+=09=20*/=0A+=09TransactionId=20cached_xid;=0A+=09= XidCommitStatus=20cached_status;=0A+=0A+=09/*=20Values=20concerning=20= the=20heap=20relation=20being=20checked=20*/=0A+=09Relation=09rel;=0A+=09= TransactionId=20relfrozenxid;=0A+=09FullTransactionId=20relfrozenfxid;=0A= +=09TransactionId=20relminmxid;=0A+=09Relation=09toast_rel;=0A+=09= Relation=20=20=20*toast_indexes;=0A+=09Relation=09valid_toast_index;=0A+=09= int=09=09=09num_toast_indexes;=0A+=0A+=09/*=20Values=20for=20iterating=20= over=20pages=20in=20the=20relation=20*/=0A+=09BlockNumber=20blkno;=0A+=09= BufferAccessStrategy=20bstrategy;=0A+=09Buffer=09=09buffer;=0A+=09Page=09= =09page;=0A+=0A+=09/*=20Values=20for=20iterating=20over=20tuples=20= within=20a=20page=20*/=0A+=09OffsetNumber=20offnum;=0A+=09ItemId=09=09= itemid;=0A+=09uint16=09=09lp_len;=0A+=09HeapTupleHeader=20tuphdr;=0A+=09= int=09=09=09natts;=0A+=0A+=09/*=20Values=20for=20iterating=20over=20= attributes=20within=20the=20tuple=20*/=0A+=09uint32=09=09offset;=09=09=09= /*=20offset=20in=20tuple=20data=20*/=0A+=09AttrNumber=09attnum;=0A+=0A+=09= /*=20Values=20for=20iterating=20over=20toast=20for=20the=20attribute=20= */=0A+=09int32=09=09chunkno;=0A+=09int32=09=09attrsize;=0A+=09int32=09=09= endchunk;=0A+=09int32=09=09totalchunks;=0A+=0A+=09/*=20Whether=20= verify_heapam=20has=20yet=20encountered=20any=20corrupt=20tuples=20*/=0A= +=09bool=09=09is_corrupt;=0A+=0A+=09/*=20The=20descriptor=20and=20= tuplestore=20for=20verify_heapam's=20result=20tuples=20*/=0A+=09= TupleDesc=09tupdesc;=0A+=09Tuplestorestate=20*tupstore;=0A+}=20= HeapCheckContext;=0A+=0A+/*=20Internal=20implementation=20*/=0A+static=20= void=20sanity_check_relation(Relation=20rel);=0A+static=20void=20= check_tuple(HeapCheckContext=20*ctx);=0A+static=20void=20= check_toast_tuple(HeapTuple=20toasttup,=20HeapCheckContext=20*ctx);=0A+=0A= +static=20bool=20check_tuple_attribute(HeapCheckContext=20*ctx);=0A= +static=20bool=20check_tuple_header_and_visibilty(HeapTupleHeader=20= tuphdr,=20HeapCheckContext=20*ctx);=0A+=0A+static=20void=20= report_corruption(HeapCheckContext=20*ctx,=20char=20*msg);=0A+static=20= TupleDesc=20verify_heapam_tupdesc(void);=0A+static=20FullTransactionId=20= FullTransactionIdFromXidAndCtx(TransactionId=20xid,=20const=20= HeapCheckContext=20*ctx);=0A+static=20void=20= update_cached_xid_range(HeapCheckContext=20*ctx);=0A+static=20void=20= update_cached_mxid_range(HeapCheckContext=20*ctx);=0A+static=20= XidBoundsViolation=20check_mxid_in_range(MultiXactId=20mxid,=20= HeapCheckContext=20*ctx);=0A+static=20XidBoundsViolation=20= check_mxid_valid_in_rel(MultiXactId=20mxid,=20HeapCheckContext=20*ctx);=0A= +static=20XidBoundsViolation=20get_xid_status(TransactionId=20xid,=20= HeapCheckContext=20*ctx,=20XidCommitStatus=20*status);=0A+=0A+/*=0A+=20*=20= Scan=20and=20report=20corruption=20in=20heap=20pages,=20optionally=20= reconciling=20toasted=0A+=20*=20attributes=20with=20entries=20in=20the=20= associated=20toast=20table.=20=20Intended=20to=20be=0A+=20*=20called=20= from=20SQL=20with=20the=20following=20parameters:=0A+=20*=0A+=20*=20=20=20= relation=0A+=20*=20=20=20=20=20The=20Oid=20of=20the=20heap=20relation=20= to=20be=20checked.=0A+=20*=0A+=20*=20=20=20on_error_stop:=0A+=20*=20=20=20= =20=20Whether=20to=20stop=20at=20the=20end=20of=20the=20first=20page=20= for=20which=20errors=20are=0A+=20*=20=20=20=20=20detected.=20=20Note=20= that=20multiple=20rows=20may=20be=20returned.=0A+=20*=0A+=20*=20=20=20= check_toast:=0A+=20*=20=20=20=20=20Whether=20to=20check=20each=20toasted=20= attribute=20against=20the=20toast=20table=20to=0A+=20*=20=20=20=20=20= verify=20that=20it=20can=20be=20found=20there.=0A+=20*=0A+=20*=20=20=20= skip:=0A+=20*=20=20=20=20=20What=20kinds=20of=20pages=20in=20the=20heap=20= relation=20should=20be=20skipped.=20=20Valid=0A+=20*=20=20=20=20=20= options=20are=20"all-visible",=20"all-frozen",=20and=20"none".=0A+=20*=0A= +=20*=20Returns=20to=20the=20SQL=20caller=20a=20set=20of=20tuples,=20= each=20containing=20the=20location=0A+=20*=20and=20a=20description=20of=20= a=20corruption=20found=20in=20the=20heap.=0A+=20*=0A+=20*=20Note=20that=20= if=20check_toast=20is=20true,=20it=20is=20the=20caller's=20= responsibility=20to=0A+=20*=20provide=20that=20the=20toast=20table=20and=20= index=20are=20not=20corrupt,=20and=20that=20they=0A+=20*=20do=20not=20= become=20corrupt=20while=20this=20function=20is=20running.=0A+=20*/=0A= +Datum=0A+verify_heapam(PG_FUNCTION_ARGS)=0A+{=0A+=09ReturnSetInfo=20= *rsinfo=20=3D=20(ReturnSetInfo=20*)=20fcinfo->resultinfo;=0A+=09= MemoryContext=20old_context;=0A+=09bool=09=09random_access;=0A+=09= HeapCheckContext=20ctx;=0A+=09Buffer=09=09vmbuffer=20=3D=20= InvalidBuffer;=0A+=09Oid=09=09=09relid;=0A+=09bool=09=09on_error_stop;=0A= +=09bool=09=09check_toast;=0A+=09SkipPages=09skip_option=20=3D=20= SKIP_PAGES_NONE;=0A+=09BlockNumber=20first_block;=0A+=09BlockNumber=20= last_block;=0A+=09BlockNumber=20nblocks;=0A+=09const=20char=20*skip;=0A+=0A= +=09/*=20Check=20to=20see=20if=20caller=20supports=20us=20returning=20a=20= tuplestore=20*/=0A+=09if=20(rsinfo=20=3D=3D=20NULL=20||=20!IsA(rsinfo,=20= ReturnSetInfo))=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),=0A+=09=09=09=09=20= errmsg("set-valued=20function=20called=20in=20context=20that=20cannot=20= accept=20a=20set")));=0A+=09if=20(!(rsinfo->allowedModes=20&=20= SFRM_Materialize))=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_SYNTAX_ERROR),=0A+=09=09=09=09=20errmsg("materialize=20= mode=20required,=20but=20it=20is=20not=20allowed=20in=20this=20= context")));=0A+=0A+=09/*=20Check=20supplied=20arguments=20*/=0A+=09if=20= (PG_ARGISNULL(0))=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09=20= errmsg("relation=20cannot=20be=20null")));=0A+=09relid=20=3D=20= PG_GETARG_OID(0);=0A+=0A+=09if=20(PG_ARGISNULL(1))=0A+=09=09= ereport(ERROR,=0A+=09=09=09=09(errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A= +=09=09=09=09=20errmsg("on_error_stop=20cannot=20be=20null")));=0A+=09= on_error_stop=20=3D=20PG_GETARG_BOOL(1);=0A+=0A+=09if=20= (PG_ARGISNULL(2))=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09=20= errmsg("check_toast=20cannot=20be=20null")));=0A+=09check_toast=20=3D=20= PG_GETARG_BOOL(2);=0A+=0A+=09if=20(PG_ARGISNULL(3))=0A+=09=09= ereport(ERROR,=0A+=09=09=09=09(errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A= +=09=09=09=09=20errmsg("skip=20cannot=20be=20null")));=0A+=09skip=20=3D=20= text_to_cstring(PG_GETARG_TEXT_PP(3));=0A+=09if=20(pg_strcasecmp(skip,=20= "all-visible")=20=3D=3D=200)=0A+=09=09skip_option=20=3D=20= SKIP_PAGES_ALL_VISIBLE;=0A+=09else=20if=20(pg_strcasecmp(skip,=20= "all-frozen")=20=3D=3D=200)=0A+=09=09skip_option=20=3D=20= SKIP_PAGES_ALL_FROZEN;=0A+=09else=20if=20(pg_strcasecmp(skip,=20"none")=20= =3D=3D=200)=0A+=09=09skip_option=20=3D=20SKIP_PAGES_NONE;=0A+=09else=0A+=09= =09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09=20= errmsg("invalid=20skip=20option"),=0A+=09=09=09=09=20errhint("Valid=20= skip=20options=20are=20\"all-visible\",=20\"all-frozen\",=20and=20= \"none\".")));=0A+=0A+=09memset(&ctx,=200,=20sizeof(HeapCheckContext));=0A= +=09ctx.cached_xid=20=3D=20InvalidTransactionId;=0A+=0A+=09/*=20The=20= tupdesc=20and=20tuplestore=20must=20be=20created=20in=20= ecxt_per_query_memory=20*/=0A+=09old_context=20=3D=20= MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);=0A+=09= random_access=20=3D=20(rsinfo->allowedModes=20&=20= SFRM_Materialize_Random)=20!=3D=200;=0A+=09ctx.tupdesc=20=3D=20= verify_heapam_tupdesc();=0A+=09ctx.tupstore=20=3D=20= tuplestore_begin_heap(random_access,=20false,=20work_mem);=0A+=09= rsinfo->returnMode=20=3D=20SFRM_Materialize;=0A+=09rsinfo->setResult=20=3D= =20ctx.tupstore;=0A+=09rsinfo->setDesc=20=3D=20ctx.tupdesc;=0A+=09= MemoryContextSwitchTo(old_context);=0A+=0A+=09/*=20Open=20relation,=20= check=20relkind=20and=20access=20method,=20and=20check=20privileges=20*/=0A= +=09ctx.rel=20=3D=20relation_open(relid,=20AccessShareLock);=0A+=09= sanity_check_relation(ctx.rel);=0A+=0A+=09/*=20Early=20exit=20if=20the=20= relation=20is=20empty=20*/=0A+=09nblocks=20=3D=20= RelationGetNumberOfBlocks(ctx.rel);=0A+=09if=20(!nblocks)=0A+=09{=0A+=09=09= relation_close(ctx.rel,=20AccessShareLock);=0A+=09=09PG_RETURN_NULL();=0A= +=09}=0A+=0A+=09ctx.bstrategy=20=3D=20GetAccessStrategy(BAS_BULKREAD);=0A= +=09ctx.buffer=20=3D=20InvalidBuffer;=0A+=09ctx.page=20=3D=20NULL;=0A+=0A= +=09/*=20Validate=20block=20numbers,=20or=20handle=20nulls.=20*/=0A+=09= if=20(PG_ARGISNULL(4))=0A+=09=09first_block=20=3D=200;=0A+=09else=0A+=09= {=0A+=09=09int64=09=09fb=20=3D=20PG_GETARG_INT64(4);=0A+=0A+=09=09if=20= (fb=20<=200=20||=20fb=20>=3D=20nblocks)=0A+=09=09=09ereport(ERROR,=0A+=09= =09=09=09=09(errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09=09= =20errmsg("starting=20block=20number=20must=20be=20between=200=20and=20= %u",=0A+=09=09=09=09=09=09=09nblocks=20-=201)));=0A+=09=09first_block=20= =3D=20(BlockNumber)=20fb;=0A+=09}=0A+=09if=20(PG_ARGISNULL(5))=0A+=09=09= last_block=20=3D=20nblocks=20-=201;=0A+=09else=0A+=09{=0A+=09=09int64=09=09= lb=20=3D=20PG_GETARG_INT64(5);=0A+=0A+=09=09if=20(lb=20<=200=20||=20lb=20= >=3D=20nblocks)=0A+=09=09=09ereport(ERROR,=0A+=09=09=09=09=09= (errcode(ERRCODE_INVALID_PARAMETER_VALUE),=0A+=09=09=09=09=09=20= errmsg("ending=20block=20number=20must=20be=20between=200=20and=20%u",=0A= +=09=09=09=09=09=09=09nblocks=20-=201)));=0A+=09=09last_block=20=3D=20= (BlockNumber)=20lb;=0A+=09}=0A+=0A+=09/*=20Optionally=20open=20the=20= toast=20relation,=20if=20any.=20*/=0A+=09if=20= (ctx.rel->rd_rel->reltoastrelid=20&&=20check_toast)=0A+=09{=0A+=09=09int=09= =09=09offset;=0A+=0A+=09=09/*=20Main=20relation=20has=20associated=20= toast=20relation=20*/=0A+=09=09ctx.toast_rel=20=3D=20= table_open(ctx.rel->rd_rel->reltoastrelid,=0A+=09=09=09=09=09=09=09=09=20= =20=20AccessShareLock);=0A+=09=09offset=20=3D=20= toast_open_indexes(ctx.toast_rel,=0A+=09=09=09=09=09=09=09=09=09= AccessShareLock,=0A+=09=09=09=09=09=09=09=09=09&(ctx.toast_indexes),=0A+=09= =09=09=09=09=09=09=09=09&(ctx.num_toast_indexes));=0A+=09=09= ctx.valid_toast_index=20=3D=20ctx.toast_indexes[offset];=0A+=09}=0A+=09= else=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20Main=20relation=20has=20no=20= associated=20toast=20relation,=20or=20we're=0A+=09=09=20*=20= intentionally=20skipping=20it.=0A+=09=09=20*/=0A+=09=09ctx.toast_rel=20=3D= =20NULL;=0A+=09=09ctx.toast_indexes=20=3D=20NULL;=0A+=09=09= ctx.num_toast_indexes=20=3D=200;=0A+=09}=0A+=0A+=09= update_cached_xid_range(&ctx);=0A+=09update_cached_mxid_range(&ctx);=0A+=09= ctx.relfrozenxid=20=3D=20ctx.rel->rd_rel->relfrozenxid;=0A+=09= ctx.relfrozenfxid=20=3D=20= FullTransactionIdFromXidAndCtx(ctx.relfrozenxid,=20&ctx);=0A+=09= ctx.relminmxid=20=3D=20ctx.rel->rd_rel->relminmxid;=0A+=0A+=09if=20= (TransactionIdIsNormal(ctx.relfrozenxid))=0A+=09=09ctx.oldest_xid=20=3D=20= ctx.relfrozenxid;=0A+=0A+=09for=20(ctx.blkno=20=3D=20first_block;=20= ctx.blkno=20<=3D=20last_block;=20ctx.blkno++)=0A+=09{=0A+=09=09= OffsetNumber=20maxoff;=0A+=0A+=09=09/*=20Optionally=20skip=20over=20= all-frozen=20or=20all-visible=20blocks=20*/=0A+=09=09if=20(skip_option=20= !=3D=20SKIP_PAGES_NONE)=0A+=09=09{=0A+=09=09=09int32=09=09mapbits;=0A+=0A= +=09=09=09mapbits=20=3D=20(int32)=20visibilitymap_get_status(ctx.rel,=20= ctx.blkno,=0A+=09=09=09=09=09=09=09=09=09=09=09=09=09=20=20=20= &vmbuffer);=0A+=09=09=09if=20(skip_option=20=3D=3D=20= SKIP_PAGES_ALL_FROZEN)=0A+=09=09=09{=0A+=09=09=09=09if=20((mapbits=20&=20= VISIBILITYMAP_ALL_FROZEN)=20!=3D=200)=0A+=09=09=09=09=09continue;=0A+=09=09= =09}=0A+=0A+=09=09=09if=20(skip_option=20=3D=3D=20= SKIP_PAGES_ALL_VISIBLE)=0A+=09=09=09{=0A+=09=09=09=09if=20((mapbits=20&=20= VISIBILITYMAP_ALL_VISIBLE)=20!=3D=200)=0A+=09=09=09=09=09continue;=0A+=09= =09=09}=0A+=09=09}=0A+=0A+=09=09/*=20Read=20and=20lock=20the=20next=20= page.=20*/=0A+=09=09ctx.buffer=20=3D=20ReadBufferExtended(ctx.rel,=20= MAIN_FORKNUM,=20ctx.blkno,=0A+=09=09=09=09=09=09=09=09=09=09RBM_NORMAL,=20= ctx.bstrategy);=0A+=09=09LockBuffer(ctx.buffer,=20BUFFER_LOCK_SHARE);=0A= +=09=09ctx.page=20=3D=20BufferGetPage(ctx.buffer);=0A+=0A+=09=09/*=20= Perform=20tuple=20checks=20*/=0A+=09=09maxoff=20=3D=20= PageGetMaxOffsetNumber(ctx.page);=0A+=09=09for=20(ctx.offnum=20=3D=20= FirstOffsetNumber;=20ctx.offnum=20<=3D=20maxoff;=0A+=09=09=09=20= ctx.offnum=20=3D=20OffsetNumberNext(ctx.offnum))=0A+=09=09{=0A+=09=09=09= ctx.itemid=20=3D=20PageGetItemId(ctx.page,=20ctx.offnum);=0A+=0A+=09=09=09= /*=20Skip=20over=20unused/dead=20line=20pointers=20*/=0A+=09=09=09if=20= (!ItemIdIsUsed(ctx.itemid)=20||=20ItemIdIsDead(ctx.itemid))=0A+=09=09=09=09= continue;=0A+=0A+=09=09=09/*=0A+=09=09=09=20*=20If=20this=20line=20= pointer=20has=20been=20redirected,=20check=20that=20it=0A+=09=09=09=20*=20= redirects=20to=20a=20valid=20offset=20within=20the=20line=20pointer=20= array.=0A+=09=09=09=20*/=0A+=09=09=09if=20= (ItemIdIsRedirected(ctx.itemid))=0A+=09=09=09{=0A+=09=09=09=09= OffsetNumber=20rdoffnum=20=3D=20ItemIdGetRedirect(ctx.itemid);=0A+=09=09=09= =09ItemId=09=09rditem;=0A+=0A+=09=09=09=09if=20(rdoffnum=20<=20= FirstOffsetNumber=20||=20rdoffnum=20>=20maxoff)=0A+=09=09=09=09{=0A+=09=09= =09=09=09report_corruption(&ctx,=0A+=09=09=09=09=09/*------=0A+=09=09=09=09= =09translator:=20the=20%u=20is=20an=20offset=20*/=0A+=09=09=09=09=09=09=09= =09=09=20=20psprintf(_("line=20pointer=20redirection=20to=20item=20at=20= offset=20%u=20exceeds=20maximum=20offset=20%u"),=0A+=09=09=09=09=09=09=09= =09=09=09=09=20=20=20(unsigned)=20rdoffnum,=0A+=09=09=09=09=09=09=09=09=09= =09=09=20=20=20(unsigned)=20maxoff));=0A+=09=09=09=09=09continue;=0A+=09=09= =09=09}=0A+=09=09=09=09rditem=20=3D=20PageGetItemId(ctx.page,=20= rdoffnum);=0A+=09=09=09=09if=20(!ItemIdIsUsed(rditem))=0A+=09=09=09=09=09= report_corruption(&ctx,=0A+=09=09=09=09=09/*------=0A+=09=09=09=09=09= translator:=20the=20%u=20is=20an=20offset=20*/=0A+=09=09=09=09=09=09=09=09= =09=20=20psprintf(_("line=20pointer=20redirection=20to=20unused=20item=20= at=20offset=20%u"),=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20=20= (unsigned)=20rdoffnum));=0A+=09=09=09=09continue;=0A+=09=09=09}=0A+=0A+=09= =09=09/*=20Set=20up=20context=20information=20about=20this=20next=20= tuple=20*/=0A+=09=09=09ctx.lp_len=20=3D=20ItemIdGetLength(ctx.itemid);=0A= +=09=09=09ctx.tuphdr=20=3D=20(HeapTupleHeader)=20PageGetItem(ctx.page,=20= ctx.itemid);=0A+=09=09=09ctx.natts=20=3D=20= HeapTupleHeaderGetNatts(ctx.tuphdr);=0A+=0A+=09=09=09/*=20Ok,=20ready=20= to=20check=20this=20next=20tuple=20*/=0A+=09=09=09check_tuple(&ctx);=0A+=09= =09}=0A+=0A+=09=09/*=20clean=20up=20*/=0A+=09=09= UnlockReleaseBuffer(ctx.buffer);=0A+=0A+=09=09if=20(on_error_stop=20&&=20= ctx.is_corrupt)=0A+=09=09=09break;=0A+=09}=0A+=0A+=09if=20(vmbuffer=20!=3D= =20InvalidBuffer)=0A+=09=09ReleaseBuffer(vmbuffer);=0A+=0A+=09/*=20Close=20= the=20associated=20toast=20table=20and=20indexes,=20if=20any.=20*/=0A+=09= if=20(ctx.toast_indexes)=0A+=09=09toast_close_indexes(ctx.toast_indexes,=20= ctx.num_toast_indexes,=0A+=09=09=09=09=09=09=09AccessShareLock);=0A+=09= if=20(ctx.toast_rel)=0A+=09=09table_close(ctx.toast_rel,=20= AccessShareLock);=0A+=0A+=09/*=20Close=20the=20main=20relation=20*/=0A+=09= relation_close(ctx.rel,=20AccessShareLock);=0A+=0A+=09PG_RETURN_NULL();=0A= +}=0A+=0A+/*=0A+=20*=20Check=20that=20a=20relation's=20relkind=20and=20= access=20method=20are=20both=20supported,=0A+=20*=20and=20that=20the=20= caller=20has=20select=20privilege=20on=20the=20relation.=0A+=20*/=0A= +static=20void=0A+sanity_check_relation(Relation=20rel)=0A+{=0A+=09= AclResult=09aclresult;=0A+=0A+=09if=20(rel->rd_rel->relkind=20!=3D=20= RELKIND_RELATION=20&&=0A+=09=09rel->rd_rel->relkind=20!=3D=20= RELKIND_MATVIEW=20&&=0A+=09=09rel->rd_rel->relkind=20!=3D=20= RELKIND_TOASTVALUE)=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_WRONG_OBJECT_TYPE),=0A+=09=09/*------=0A+=09=09= translator:=20%s=20is=20a=20user=20supplied=20object=20name=20*/=0A+=09=09= =09=09=20errmsg("\"%s\"=20is=20not=20a=20table,=20materialized=20view,=20= or=20TOAST=20table",=0A+=09=09=09=09=09=09= RelationGetRelationName(rel))));=0A+=09if=20(rel->rd_rel->relam=20!=3D=20= HEAP_TABLE_AM_OID)=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),=0A+=09=09=09=09=20errmsg("only=20= heap=20AM=20is=20supported")));=0A+=09aclresult=20=3D=20= pg_class_aclcheck(rel->rd_id,=20GetUserId(),=20ACL_SELECT);=0A+=09if=20= (aclresult=20!=3D=20ACLCHECK_OK)=0A+=09=09aclcheck_error(aclresult,=0A+=09= =09=09=09=09=20=20=20get_relkind_objtype(rel->rd_rel->relkind),=0A+=09=09= =09=09=09=20=20=20RelationGetRelationName(rel));=0A+}=0A+=0A+/*=0A+=20*=20= Record=20a=20single=20corruption=20found=20in=20the=20table.=20=20The=20= values=20in=20ctx=20should=0A+=20*=20reflect=20the=20location=20of=20the=20= corruption,=20and=20the=20msg=20argument=20should=20contain=0A+=20*=20a=20= human=20readable=20description=20of=20the=20corruption.=0A+=20*=0A+=20*=20= The=20msg=20argument=20is=20pfree'd=20by=20this=20function.=0A+=20*/=0A= +static=20void=0A+report_corruption(HeapCheckContext=20*ctx,=20char=20= *msg)=0A+{=0A+=09Datum=09=09values[HEAPCHECK_RELATION_COLS];=0A+=09bool=09= =09nulls[HEAPCHECK_RELATION_COLS];=0A+=09HeapTuple=09tuple;=0A+=0A+=09= MemSet(values,=200,=20sizeof(values));=0A+=09MemSet(nulls,=200,=20= sizeof(nulls));=0A+=09values[0]=20=3D=20Int64GetDatum(ctx->blkno);=0A+=09= values[1]=20=3D=20Int32GetDatum(ctx->offnum);=0A+=09values[2]=20=3D=20= Int32GetDatum(ctx->attnum);=0A+=09nulls[2]=20=3D=20(ctx->attnum=20<=20= 0);=0A+=09values[3]=20=3D=20CStringGetTextDatum(msg);=0A+=0A+=09/*=0A+=09= =20*=20In=20principle,=20there=20is=20nothing=20to=20prevent=20a=20scan=20= over=20a=20large,=20highly=0A+=09=20*=20corrupted=20table=20from=20using=20= work_mem=20worth=20of=20memory=20building=20up=20the=0A+=09=20*=20= tuplestore.=20=20That's=20ok,=20but=20if=20we=20also=20leak=20the=20msg=20= argument=20memory=0A+=09=20*=20until=20the=20end=20of=20the=20query,=20= we=20could=20exceed=20work_mem=20by=20more=20than=20a=0A+=09=20*=20= trivial=20amount.=20=20Therefore,=20free=20the=20msg=20argument=20each=20= time=20we=20are=0A+=09=20*=20called=20rather=20than=20waiting=20for=20= our=20current=20memory=20context=20to=20be=20freed.=0A+=09=20*/=0A+=09= pfree(msg);=0A+=0A+=09tuple=20=3D=20heap_form_tuple(ctx->tupdesc,=20= values,=20nulls);=0A+=09tuplestore_puttuple(ctx->tupstore,=20tuple);=0A+=09= ctx->is_corrupt=20=3D=20true;=0A+}=0A+=0A+/*=0A+=20*=20Construct=20the=20= TupleDesc=20used=20to=20report=20messages=20about=20corruptions=20found=0A= +=20*=20while=20scanning=20the=20heap.=0A+=20*/=0A+static=20TupleDesc=0A= +verify_heapam_tupdesc(void)=0A+{=0A+=09TupleDesc=09tupdesc;=0A+=09= AttrNumber=09a=20=3D=200;=0A+=0A+=09tupdesc=20=3D=20= CreateTemplateTupleDesc(HEAPCHECK_RELATION_COLS);=0A+=09= TupleDescInitEntry(tupdesc,=20++a,=20"blkno",=20INT8OID,=20-1,=200);=0A+=09= TupleDescInitEntry(tupdesc,=20++a,=20"offnum",=20INT4OID,=20-1,=200);=0A= +=09TupleDescInitEntry(tupdesc,=20++a,=20"attnum",=20INT4OID,=20-1,=20= 0);=0A+=09TupleDescInitEntry(tupdesc,=20++a,=20"msg",=20TEXTOID,=20-1,=20= 0);=0A+=09Assert(a=20=3D=3D=20HEAPCHECK_RELATION_COLS);=0A+=0A+=09return=20= BlessTupleDesc(tupdesc);=0A+}=0A+=0A+/*=0A+=20*=20Check=20for=20tuple=20= header=20corruption=20and=20tuple=20visibility.=0A+=20*=0A+=20*=20Since=20= we=20do=20not=20hold=20a=20snapshot,=20tuple=20visibility=20is=20not=20a=20= question=20of=0A+=20*=20whether=20we=20should=20be=20able=20to=20see=20= the=20tuple=20relative=20to=20any=20particular=0A+=20*=20snapshot,=20but=20= rather=20a=20question=20of=20whether=20it=20is=20safe=20and=20reasonable=20= to=0A+=20*=20to=20check=20the=20tuple=20attributes.=0A+=20*=0A+=20*=20= Some=20kinds=20of=20corruption=20make=20it=20unsafe=20to=20check=20the=20= tuple=20attributes,=20for=0A+=20*=20example=20when=20the=20line=20= pointer=20refers=20to=20a=20range=20of=20bytes=20outside=20the=20page.=0A= +=20*=20In=20such=20cases,=20we=20return=20false=20(not=20visible)=20= after=20recording=20appropriate=0A+=20*=20corruption=20messages.=0A+=20*=0A= +=20*=20Some=20other=20kinds=20of=20tuple=20header=20corruption=20= confuse=20the=20question=20of=20where=0A+=20*=20the=20tuple=20attributes=20= begin,=20or=20how=20long=20the=20nulls=20bitmap=20is,=20etc.,=20making=20= it=0A+=20*=20unreasonable=20to=20attempt=20to=20check=20attributes,=20= even=20if=20all=20candidate=20answers=0A+=20*=20to=20those=20questions=20= would=20not=20result=20in=20reading=20past=20the=20end=20of=20the=20line=0A= +=20*=20pointer=20or=20page.=20=20In=20such=20cases,=20like=20above,=20= we=20record=20corruption=20messages=0A+=20*=20about=20the=20header=20and=20= then=20return=20false.=0A+=20*=0A+=20*=20Other=20kinds=20of=20tuple=20= header=20corruption=20do=20not=20bear=20on=20the=20question=20of=0A+=20*=20= whether=20the=20tuple=20attributes=20can=20be=20checked,=20so=20we=20= record=20corruption=0A+=20*=20messages=20for=20them=20but=20do=20not=20= base=20our=20visibility=20determination=20on=20them.=20=20(In=0A+=20*=20= other=20words,=20we=20do=20not=20return=20false=20merely=20because=20we=20= detected=20them.)=0A+=20*=0A+=20*=20For=20visibility=20determination=20= not=20specifically=20related=20to=20corruption,=20what=20we=0A+=20*=20= want=20to=20know=20is=20if=20a=20tuple=20is=20potentially=20visible=20to=20= any=20running=0A+=20*=20transaction.=20=20If=20you=20are=20tempted=20to=20= replace=20this=20function's=20visibility=20logic=0A+=20*=20with=20a=20= call=20to=20another=20visibility=20checking=20function,=20keep=20in=20= mind=20that=20this=0A+=20*=20function=20does=20not=20update=20hint=20= bits,=20as=20it=20seems=20imprudent=20to=20write=20hint=20bits=0A+=20*=20= (or=20anything=20at=20all)=20to=20a=20table=20during=20a=20corruption=20= check.=20=20Nor=20does=20this=0A+=20*=20function=20bother=20classifying=20= tuple=20visibility=20beyond=20a=20boolean=20visible=20vs.=0A+=20*=20not=20= visible.=0A+=20*=0A+=20*=20The=20caller=20should=20already=20have=20= checked=20that=20xmin=20and=20xmax=20are=20not=20out=20of=0A+=20*=20= bounds=20for=20the=20relation.=0A+=20*=0A+=20*=20Returns=20whether=20the=20= tuple=20is=20both=20visible=20and=20sufficiently=20sensible=20to=0A+=20*=20= undergo=20attribute=20checks.=0A+=20*/=0A+static=20bool=0A= +check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A+{=0A+=09uint16=09=09infomask=20=3D=20= tuphdr->t_infomask;=0A+=09bool=09=09header_garbled=20=3D=20false;=0A+=09= unsigned=09expected_hoff;;=0A+=0A+=09if=20(ctx->tuphdr->t_hoff=20>=20= ctx->lp_len)=0A+=09{=0A+=09=09report_corruption(ctx,=0A+=09=09/*------=0A= +=09=09translator:=20first=20%u=20is=20an=20offset,=20second=20%u=20is=20= a=20length=20*/=0A+=09=09=09=09=09=09=20=20psprintf(_("data=20begins=20= at=20offset=20%u=20beyond=20the=20tuple=20length=20%u"),=0A+=09=09=09=09=09= =09=09=09=20=20=20ctx->tuphdr->t_hoff,=20ctx->lp_len));=0A+=09=09= header_garbled=20=3D=20true;=0A+=09}=0A+=09if=20= ((ctx->tuphdr->t_infomask=20&=20HEAP_XMAX_LOCK_ONLY)=20&&=0A+=09=09= (ctx->tuphdr->t_infomask2=20&=20HEAP_KEYS_UPDATED))=0A+=09{=0A+=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=20=20pstrdup(_("tuple=20is=20= marked=20as=20only=20locked,=20but=20also=20claims=20key=20columns=20= were=20updated")));=0A+=09=09header_garbled=20=3D=20true;=0A+=09}=0A+=0A= +=09if=20((ctx->tuphdr->t_infomask=20&=20HEAP_XMAX_COMMITTED)=20&&=0A+=09= =09(ctx->tuphdr->t_infomask=20&=20HEAP_XMAX_IS_MULTI))=0A+=09{=0A+=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=20=20pstrdup(_("multixact=20= should=20not=20be=20marked=20committed")));=0A+=0A+=09=09/*=0A+=09=09=20= *=20This=20condition=20is=20clearly=20wrong,=20but=20we=20do=20not=20= consider=20the=20header=0A+=09=09=20*=20garbled,=20because=20we=20don't=20= rely=20on=20this=20property=20for=20determining=20if=0A+=09=09=20*=20the=20= tuple=20is=20visible=20or=20for=20interpreting=20other=20relevant=20= header=0A+=09=09=20*=20fields.=0A+=09=09=20*/=0A+=09}=0A+=0A+=09if=20= (infomask=20&=20HEAP_HASNULL)=0A+=09=09expected_hoff=20=3D=20= MAXALIGN(SizeofHeapTupleHeader=20+=20BITMAPLEN(ctx->natts));=0A+=09else=0A= +=09=09expected_hoff=20=3D=20MAXALIGN(SizeofHeapTupleHeader);=0A+=09if=20= (ctx->tuphdr->t_hoff=20!=3D=20expected_hoff)=0A+=09{=0A+=09=09if=20= ((infomask=20&=20HEAP_HASNULL)=20&&=20ctx->natts=20=3D=3D=201)=0A+=09=09=09= report_corruption(ctx,=0A+=09=09=09/*------=0A+=09=09=09translator:=20= both=20%u=20represent=20an=20offset=20*/=0A+=09=09=09=09=09=09=09=20=20= psprintf(_("tuple=20data=20should=20begin=20at=20byte=20%u,=20but=20= actually=20begins=20at=20byte=20%u=20(1=20attribute,=20has=20nulls)"),=0A= +=09=09=09=09=09=09=09=09=09=20=20=20expected_hoff,=20= ctx->tuphdr->t_hoff));=0A+=09=09else=20if=20((infomask=20&=20= HEAP_HASNULL))=0A+=09=09=09report_corruption(ctx,=0A+=09=09=09/*------=0A= +=09=09=09translator:=20first=20and=20second=20%u=20represent=20an=20= offset,=20third=20%u=0A+=09=09=09represents=20the=20number=20of=20= attributes=20*/=0A+=09=09=09=09=09=09=09=20=20psprintf(_("tuple=20data=20= should=20begin=20at=20byte=20%u,=20but=20actually=20begins=20at=20byte=20= %u=20(%u=20attributes,=20has=20nulls)"),=0A+=09=09=09=09=09=09=09=09=09=20= =20=20expected_hoff,=20ctx->tuphdr->t_hoff,=20ctx->natts));=0A+=09=09= else=20if=20(ctx->natts=20=3D=3D=201)=0A+=09=09=09report_corruption(ctx,=0A= +=09=09=09/*------=0A+=09=09=09translator:=20both=20%u=20represent=20an=20= offset=20*/=0A+=09=09=09=09=09=09=09=20=20psprintf(_("tuple=20data=20= should=20begin=20at=20byte=20%u,=20but=20actually=20begins=20at=20byte=20= %u=20(1=20attribute,=20no=20nulls)"),=0A+=09=09=09=09=09=09=09=09=09=20=20= =20expected_hoff,=20ctx->tuphdr->t_hoff));=0A+=09=09else=0A+=09=09=09= report_corruption(ctx,=0A+=09=09=09/*------=0A+=09=09=09translator:=20= first=20and=20second=20%u=20represent=20an=20offset,=20third=20%u=0A+=09=09= =09represents=20the=20number=20of=20attributes=20*/=0A+=09=09=09=09=09=09= =09=20=20psprintf(_("tuple=20data=20should=20begin=20at=20byte=20%u,=20= but=20actually=20begins=20at=20byte=20%u=20(%u=20attributes,=20no=20= nulls)"),=0A+=09=09=09=09=09=09=09=09=09=20=20=20expected_hoff,=20= ctx->tuphdr->t_hoff,=20ctx->natts));=0A+=09=09header_garbled=20=3D=20= true;=0A+=09}=0A+=0A+=09if=20(header_garbled)=0A+=09=09return=20false;=09= =09=09/*=20checking=20of=20this=20tuple=20should=20not=20continue=20*/=0A= +=0A+=09/*=0A+=09=20*=20Ok,=20we=20can=20examine=20the=20header=20for=20= tuple=20visibility=20purposes,=20though=20we=0A+=09=20*=20still=20need=20= to=20be=20careful=20about=20a=20few=20remaining=20types=20of=20header=0A= +=09=20*=20corruption.=20=20This=20logic=20roughly=20follows=20that=20of=0A= +=09=20*=20HeapTupleSatisfiesVacuum.=20=20Where=20possible=20the=20= comments=20indicate=20which=0A+=09=20*=20HTSV_Result=20we=20think=20that=20= function=20might=20return=20for=20this=20tuple.=0A+=09=20*/=0A+=09if=20= (!HeapTupleHeaderXminCommitted(tuphdr))=0A+=09{=0A+=09=09TransactionId=20= raw_xmin=20=3D=20HeapTupleHeaderGetRawXmin(tuphdr);=0A+=0A+=09=09if=20= (HeapTupleHeaderXminInvalid(tuphdr))=0A+=09=09=09return=20false;=09=09/*=20= HEAPTUPLE_DEAD=20*/=0A+=09=09/*=20Used=20by=20pre-9.0=20binary=20= upgrades=20*/=0A+=09=09else=20if=20(infomask=20&=20HEAP_MOVED_OFF=20||=0A= +=09=09=09=09=20infomask=20&=20HEAP_MOVED_IN)=0A+=09=09{=0A+=09=09=09= XidCommitStatus=20status;=0A+=09=09=09TransactionId=20xvac=20=3D=20= HeapTupleHeaderGetXvac(tuphdr);=0A+=0A+=09=09=09switch=20= (get_xid_status(xvac,=20ctx,=20&status))=0A+=09=09=09{=0A+=09=09=09=09= case=20XID_INVALID:=0A+=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09= =09=09=09=09=09=09=20=20pstrdup(_("old-style=20VACUUM=20FULL=20= transaction=20ID=20is=20invalid")));=0A+=09=09=09=09=09return=20false;=09= /*=20corrupt=20*/=0A+=09=09=09=09case=20XID_IN_FUTURE:=0A+=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09/*------=0A+=09=09=09=09=09= translator:=20first=20and=20third=20%u=20are=20a=20transaction=0A+=09=09=09= =09=09identifier,=20second=20%u=20is=20an=20epoch=20*/=0A+=09=09=09=09=09= =09=09=09=09=20=20psprintf(_("old-style=20VACUUM=20FULL=20transaction=20= ID=20%u=20equals=20or=20exceeds=20next=20valid=20transaction=20ID=20= %u:%u"),=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20=20xvac,=0A+=09=09=09=09= =09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->next_fxid),=0A+=09=09=09=09=09=09=09=09=09= =09=09=20=20=20XidFromFullTransactionId(ctx->next_fxid)));=0A+=09=09=09=09= =09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20= XID_PRECEDES_RELMIN:=0A+=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09= =09=09/*------=0A+=09=09=09=09=09translator:=20first=20and=20third=20%u=20= are=20a=20transaction=0A+=09=09=09=09=09identifier,=20second=20%u=20is=20= an=20epoch=20*/=0A+=09=09=09=09=09=09=09=09=09=20=20= psprintf(_("old-style=20VACUUM=20FULL=20transaction=20ID=20%u=20precedes=20= relation=20freeze=20threshold=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09=09= =09=20=20=20xvac,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->relfrozenfxid),=0A+=09=09=09=09=09=09=09=09= =09=09=09=20=20=20XidFromFullTransactionId(ctx->relfrozenfxid)));=0A+=09=09= =09=09=09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09=09break;=0A= +=09=09=09=09case=20XID_PRECEDES_DATMIN:=0A+=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09/*------=0A+=09=09=09=09=09= translator:=20first=20and=20third=20%u=20are=20a=20transaction=0A+=09=09=09= =09=09identifier,=20second=20%u=20is=20an=20epoch=20*/=0A+=09=09=09=09=09= =09=09=09=09=20=20psprintf(_("old-style=20VACUUM=20FULL=20transaction=20= ID=20%u=20precedes=20oldest=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09= =09=09=09=09=09=09=09=09=09=20=20=20xvac,=0A+=09=09=09=09=09=09=09=09=09=09= =09=20=20=20EpochFromFullTransactionId(ctx->oldest_fxid),=0A+=09=09=09=09= =09=09=09=09=09=09=09=20=20=20= XidFromFullTransactionId(ctx->oldest_fxid)));=0A+=09=09=09=09=09return=20= false;=09/*=20corrupt=20*/=0A+=09=09=09=09=09break;=0A+=09=09=09=09case=20= XID_BOUNDS_OK:=0A+=09=09=09=09=09switch=20(status)=0A+=09=09=09=09=09{=0A= +=09=09=09=09=09=09case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09=09= return=20true;=09/*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A+=09=09=09=09=09= =09case=20XID_COMMITTED:=0A+=09=09=09=09=09=09case=20XID_ABORTED:=0A+=09=09= =09=09=09=09=09return=20false;=09/*=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09= =09}=0A+=09=09=09}=0A+=09=09}=0A+=09=09else=0A+=09=09{=0A+=09=09=09= XidCommitStatus=20status;=0A+=0A+=09=09=09switch=20= (get_xid_status(raw_xmin,=20ctx,=20&status))=0A+=09=09=09{=0A+=09=09=09=09= case=20XID_INVALID:=0A+=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09= =09=09=09=09=09=09=20=20pstrdup(_("raw=20xmin=20is=20invalid")));=0A+=09=09= =09=09=09return=20false;=0A+=09=09=09=09case=20XID_IN_FUTURE:=0A+=09=09=09= =09=09report_corruption(ctx,=0A+=09=09=09=09=09/*------=0A+=09=09=09=09=09= translator:=20first=20and=20third=20%u=20are=20a=20transaction=0A+=09=09=09= =09=09identifier,=20second=20%u=20is=20an=20epoch=20*/=0A+=09=09=09=09=09= =09=09=09=09=20=20psprintf(_("raw=20xmin=20%u=20equals=20or=20exceeds=20= next=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09= =09=09=20=20=20raw_xmin,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->next_fxid),=0A+=09=09=09=09=09=09=09=09=09= =09=09=20=20=20XidFromFullTransactionId(ctx->next_fxid)));=0A+=09=09=09=09= =09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20= XID_PRECEDES_RELMIN:=0A+=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09= =09=09/*------=0A+=09=09=09=09=09translator:=20first=20and=20third=20%u=20= are=20a=20transaction=0A+=09=09=09=09=09identifier,=20second=20%u=20is=20= an=20epoch=20*/=0A+=09=09=09=09=09=09=09=09=09=20=20psprintf(_("raw=20= xmin=20%u=20precedes=20relation=20freeze=20threshold=20%u:%u"),=0A+=09=09= =09=09=09=09=09=09=09=09=09=20=20=20raw_xmin,=0A+=09=09=09=09=09=09=09=09= =09=09=09=20=20=20EpochFromFullTransactionId(ctx->relfrozenfxid),=0A+=09=09= =09=09=09=09=09=09=09=09=09=20=20=20= XidFromFullTransactionId(ctx->relfrozenfxid)));=0A+=09=09=09=09=09return=20= false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20XID_PRECEDES_DATMIN:=0A= +=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09=09/*------=0A+=09=09= =09=09=09translator:=20first=20and=20third=20%u=20are=20a=20transaction=0A= +=09=09=09=09=09identifier,=20second=20%u=20is=20an=20epoch=20*/=0A+=09=09= =09=09=09=09=09=09=09=20=20psprintf(_("raw=20xmin=20%u=20precedes=20= oldest=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09= =09=09=20=20=20raw_xmin,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->oldest_fxid),=0A+=09=09=09=09=09=09=09=09= =09=09=09=20=20=20XidFromFullTransactionId(ctx->oldest_fxid)));=0A+=09=09= =09=09=09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20= XID_BOUNDS_OK:=0A+=09=09=09=09=09switch=20(status)=0A+=09=09=09=09=09{=0A= +=09=09=09=09=09=09case=20XID_COMMITTED:=0A+=09=09=09=09=09=09=09break;=0A= +=09=09=09=09=09=09case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09=09= return=20true;=09/*=20insert=20or=20delete=20in=20progress=20*/=0A+=09=09= =09=09=09=09case=20XID_ABORTED:=0A+=09=09=09=09=09=09=09return=20false;=09= /*=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09=09}=0A+=09=09=09}=0A+=09=09}=0A= +=09}=0A+=0A+=09if=20(!(infomask=20&=20HEAP_XMAX_INVALID)=20&&=20= !HEAP_XMAX_IS_LOCKED_ONLY(infomask))=0A+=09{=0A+=09=09if=20(infomask=20&=20= HEAP_XMAX_IS_MULTI)=0A+=09=09{=0A+=09=09=09XidCommitStatus=20status;=0A+=09= =09=09TransactionId=20xmax=20=3D=20HeapTupleGetUpdateXid(tuphdr);=0A+=0A= +=09=09=09switch=20(get_xid_status(xmax,=20ctx,=20&status))=0A+=09=09=09= {=0A+=09=09=09=09=09/*=20not=20LOCKED_ONLY,=20so=20it=20has=20to=20have=20= an=20xmax=20*/=0A+=09=09=09=09case=20XID_INVALID:=0A+=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09=09=09=20=20= pstrdup(_("xmax=20is=20invalid")));=0A+=09=09=09=09=09return=20false;=09= /*=20corrupt=20*/=0A+=09=09=09=09case=20XID_IN_FUTURE:=0A+=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09/*------=0A+=09=09=09=09=09= translator:=20first=20and=20third=20%u=20are=20a=20transaction=0A+=09=09=09= =09=09identifier,=20second=20%u=20is=20an=20epoch=20*/=0A+=09=09=09=09=09= =09=09=09=09=20=20psprintf(_("xmax=20%u=20equals=20or=20exceeds=20next=20= valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09=09=09=20= =20=20xmax,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->next_fxid),=0A+=09=09=09=09=09=09=09=09=09= =09=09=20=20=20XidFromFullTransactionId(ctx->next_fxid)));=0A+=09=09=09=09= =09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20= XID_PRECEDES_RELMIN:=0A+=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09= =09=09/*------=0A+=09=09=09=09=09translator:=20first=20and=20third=20%u=20= are=20a=20transaction=0A+=09=09=09=09=09identifier,=20second=20%u=20is=20= an=20epoch=20*/=0A+=09=09=09=09=09=09=09=09=09=20=20psprintf(_("xmax=20= %u=20precedes=20relation=20freeze=20threshold=20%u:%u"),=0A+=09=09=09=09=09= =09=09=09=09=09=09=20=20=20xmax,=0A+=09=09=09=09=09=09=09=09=09=09=09=20=20= =20EpochFromFullTransactionId(ctx->relfrozenfxid),=0A+=09=09=09=09=09=09=09= =09=09=09=09=20=20=20XidFromFullTransactionId(ctx->relfrozenfxid)));=0A+=09= =09=09=09=09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20= XID_PRECEDES_DATMIN:=0A+=09=09=09=09=09report_corruption(ctx,=0A+=09=09=09= =09=09/*------=0A+=09=09=09=09=09translator:=20first=20and=20third=20%u=20= are=20a=20transaction=0A+=09=09=09=09=09identifier,=20second=20%u=20is=20= an=20epoch=20*/=0A+=09=09=09=09=09=09=09=09=09=20=20psprintf(_("xmax=20= %u=20precedes=20oldest=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09= =09=09=09=09=09=09=09=09=20=20=20xmax,=0A+=09=09=09=09=09=09=09=09=09=09=09= =20=20=20EpochFromFullTransactionId(ctx->oldest_fxid),=0A+=09=09=09=09=09= =09=09=09=09=09=09=20=20=20XidFromFullTransactionId(ctx->oldest_fxid)));=0A= +=09=09=09=09=09return=20false;=09/*=20corrupt=20*/=0A+=09=09=09=09case=20= XID_BOUNDS_OK:=0A+=09=09=09=09=09switch=20(status)=0A+=09=09=09=09=09{=0A= +=09=09=09=09=09=09case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09=09= return=20true;=09/*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A+=09=09=09=09=09= =09case=20XID_COMMITTED:=0A+=09=09=09=09=09=09case=20XID_ABORTED:=0A+=09=09= =09=09=09=09=09return=20false;=09/*=20HEAPTUPLE_RECENTLY_DEAD=20or=0A+=09= =09=09=09=09=09=09=09=09=09=09=20*=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09=09= }=0A+=09=09=09}=0A+=0A+=09=09=09/*=20Ok,=20the=20tuple=20is=20live=20*/=0A= +=09=09}=0A+=09=09else=20if=20(!(infomask=20&=20HEAP_XMAX_COMMITTED))=0A= +=09=09=09return=20true;=09=09/*=20HEAPTUPLE_DELETE_IN_PROGRESS=20or=0A+=09= =09=09=09=09=09=09=09=20*=20HEAPTUPLE_LIVE=20*/=0A+=09=09else=0A+=09=09=09= return=20false;=09=09/*=20HEAPTUPLE_RECENTLY_DEAD=20or=20HEAPTUPLE_DEAD=20= */=0A+=09}=0A+=09return=20true;=09=09=09=09/*=20not=20dead=20*/=0A+}=0A+=0A= +/*=0A+=20*=20Check=20the=20current=20toast=20tuple=20against=20the=20= state=20tracked=20in=20ctx,=20recording=0A+=20*=20any=20corruption=20= found=20in=20ctx->tupstore.=0A+=20*=0A+=20*=20This=20is=20not=20= equivalent=20to=20running=20verify_heapam=20on=20the=20toast=20table=20= itself,=0A+=20*=20and=20is=20not=20hardened=20against=20corruption=20of=20= the=20toast=20table.=20=20Rather,=20when=0A+=20*=20validating=20a=20= toasted=20attribute=20in=20the=20main=20table,=20the=20sequence=20of=20= toast=0A+=20*=20tuples=20that=20store=20the=20toasted=20value=20are=20= retrieved=20and=20checked=20in=20order,=20with=0A+=20*=20each=20toast=20= tuple=20being=20checked=20against=20where=20we=20are=20in=20the=20= sequence,=20as=20well=0A+=20*=20as=20each=20toast=20tuple=20having=20its=20= varlena=20structure=20sanity=20checked.=0A+=20*/=0A+static=20void=0A= +check_toast_tuple(HeapTuple=20toasttup,=20HeapCheckContext=20*ctx)=0A+{=0A= +=09int32=09=09curchunk;=0A+=09Pointer=09=09chunk;=0A+=09bool=09=09= isnull;=0A+=09int32=09=09chunksize;=0A+=09int32=09=09expected_size;=0A+=0A= +=09/*=0A+=09=20*=20Have=20a=20chunk,=20extract=20the=20sequence=20= number=20and=20the=20data=0A+=09=20*/=0A+=09curchunk=20=3D=20= DatumGetInt32(fastgetattr(toasttup,=202,=0A+=09=09=09=09=09=09=09=09=09=09= =20ctx->toast_rel->rd_att,=20&isnull));=0A+=09if=20(isnull)=0A+=09{=0A+=09= =09report_corruption(ctx,=0A+=09=09=09=09=09=09=20=20pstrdup(_("toast=20= chunk=20sequence=20number=20is=20null")));=0A+=09=09return;=0A+=09}=0A+=09= chunk=20=3D=20DatumGetPointer(fastgetattr(toasttup,=203,=0A+=09=09=09=09=09= =09=09=09=09=09ctx->toast_rel->rd_att,=20&isnull));=0A+=09if=20(isnull)=0A= +=09{=0A+=09=09report_corruption(ctx,=0A+=09=09=09=09=09=09=20=20= pstrdup(_("toast=20chunk=20data=20is=20null")));=0A+=09=09return;=0A+=09= }=0A+=09if=20(!VARATT_IS_EXTENDED(chunk))=0A+=09=09chunksize=20=3D=20= VARSIZE(chunk)=20-=20VARHDRSZ;=0A+=09else=20if=20= (VARATT_IS_SHORT(chunk))=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20could=20= happen=20due=20to=20heap_form_tuple=20doing=20its=20thing=0A+=09=09=20*/=0A= +=09=09chunksize=20=3D=20VARSIZE_SHORT(chunk)=20-=20VARHDRSZ_SHORT;=0A+=09= }=0A+=09else=0A+=09{=0A+=09=09/*=20should=20never=20happen=20*/=0A+=09=09= uint32=09=09header=20=3D=20((varattrib_4b=20*)=20= chunk)->va_4byte.va_header;=0A+=0A+=09=09report_corruption(ctx,=0A+=09=09= /*------=0A+=09=09translator:=20%0x=20represents=20a=20bit=20pattern=20= in=20hexadecimal,=20%d=20represents=0A+=09=09the=20sequence=20number=20= */=0A+=09=09=09=09=09=09=20=20psprintf(_("corrupt=20extended=20toast=20= chunk=20has=20invalid=20varlena=20header:=20%0x=20(sequence=20number=20= %d)"),=0A+=09=09=09=09=09=09=09=09=20=20=20header,=20curchunk));=0A+=09=09= return;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20Some=20checks=20on=20the=20= data=20we've=20found=0A+=09=20*/=0A+=09if=20(curchunk=20!=3D=20= ctx->chunkno)=0A+=09{=0A+=09=09report_corruption(ctx,=0A+=09=09/*------=0A= +=09=09translator:=20both=20%u=20represent=20a=20sequence=20number=20*/=0A= +=09=09=09=09=09=09=20=20psprintf(_("toast=20chunk=20sequence=20number=20= %u=20does=20not=20match=20the=20expected=20sequence=20number=20%u"),=0A+=09= =09=09=09=09=09=09=09=20=20=20curchunk,=20ctx->chunkno));=0A+=09=09= return;=0A+=09}=0A+=09if=20(curchunk=20>=20ctx->endchunk)=0A+=09{=0A+=09=09= report_corruption(ctx,=0A+=09=09/*------=0A+=09=09translator:=20both=20= %u=20represent=20a=20sequence=20number=20*/=0A+=09=09=09=09=09=09=20=20= psprintf(_("toast=20chunk=20sequence=20number=20%u=20exceeds=20the=20end=20= chunk=20sequence=20number=20%u"),=0A+=09=09=09=09=09=09=09=09=20=20=20= curchunk,=20ctx->endchunk));=0A+=09=09return;=0A+=09}=0A+=0A+=09= expected_size=20=3D=20curchunk=20<=20ctx->totalchunks=20-=201=20?=20= TOAST_MAX_CHUNK_SIZE=0A+=09=09:=20ctx->attrsize=20-=20((ctx->totalchunks=20= -=201)=20*=20TOAST_MAX_CHUNK_SIZE);=0A+=09if=20(chunksize=20!=3D=20= expected_size)=0A+=09{=0A+=09=09report_corruption(ctx,=0A+=09=09/*------=0A= +=09=09translator:=20both=20%u=20represent=20a=20chunk=20size=20*/=0A+=09= =09=09=09=09=09=20=20psprintf(_("toast=20chunk=20size=20%u=20differs=20= from=20the=20expected=20size=20%u"),=0A+=09=09=09=09=09=09=09=09=20=20=20= chunksize,=20expected_size));=0A+=09=09return;=0A+=09}=0A+}=0A+=0A+/*=0A= +=20*=20Check=20the=20current=20attribute=20as=20tracked=20in=20ctx,=20= recording=20any=20corruption=0A+=20*=20found=20in=20ctx->tupstore.=0A+=20= *=0A+=20*=20This=20function=20follows=20the=20logic=20performed=20by=20= heap_deform_tuple(),=20and=20in=20the=0A+=20*=20case=20of=20a=20toasted=20= value,=20optionally=20continues=20along=20the=20logic=20of=0A+=20*=20= detoast_external_attr(),=20checking=20for=20any=20conditions=20that=20= would=20result=20in=0A+=20*=20either=20of=20those=20functions=20= Asserting=20or=20crashing=20the=20backend.=20=20The=20checks=0A+=20*=20= performed=20by=20Asserts=20present=20in=20those=20two=20functions=20are=20= also=20performed=20here.=0A+=20*=20In=20cases=20where=20those=20two=20= functions=20are=20a=20bit=20cavalier=20in=20their=20assumptions=0A+=20*=20= about=20data=20being=20correct,=20we=20perform=20additional=20checks=20= not=20present=20in=20either=0A+=20*=20of=20those=20two=20functions.=20=20= Where=20some=20condition=20is=20checked=20in=20both=20of=20those=0A+=20*=20= functions,=20we=20perform=20it=20here=20twice,=20as=20we=20parallel=20= the=20logical=20flow=20of=0A+=20*=20those=20two=20functions.=20=20The=20= presence=20of=20duplicate=20checks=20seems=20a=20reasonable=0A+=20*=20= price=20to=20pay=20for=20keeping=20this=20code=20tightly=20coupled=20= with=20the=20code=20it=0A+=20*=20protects.=0A+=20*=0A+=20*=20Returns=20= true=20if=20the=20tuple=20attribute=20is=20sane=20enough=20for=20= processing=20to=0A+=20*=20continue=20on=20to=20the=20next=20attribute,=20= false=20otherwise.=0A+=20*/=0A+static=20bool=0A= +check_tuple_attribute(HeapCheckContext=20*ctx)=0A+{=0A+=09struct=20= varatt_external=20toast_pointer;=0A+=09ScanKeyData=20toastkey;=0A+=09= SysScanDesc=20toastscan;=0A+=09SnapshotData=20SnapshotToast;=0A+=09= HeapTuple=09toasttup;=0A+=09bool=09=09found_toasttup;=0A+=09Datum=09=09= attdatum;=0A+=09struct=20varlena=20*attr;=0A+=09char=09=20=20=20*tp;=09=09= =09=09/*=20pointer=20to=20the=20tuple=20data=20*/=0A+=09uint16=09=09= infomask;=0A+=09Form_pg_attribute=20thisatt;=0A+=0A+=09infomask=20=3D=20= ctx->tuphdr->t_infomask;=0A+=09thisatt=20=3D=20= TupleDescAttr(RelationGetDescr(ctx->rel),=20ctx->attnum);=0A+=0A+=09tp=20= =3D=20(char=20*)=20ctx->tuphdr=20+=20ctx->tuphdr->t_hoff;=0A+=0A+=09if=20= (ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20ctx->lp_len)=0A+=09{=0A+=09=09= report_corruption(ctx,=0A+=09=09/*------=0A+=09=09translator:=20first=20= %u=20represents=20an=20attribute=20number;=20second=20and=20fourth=0A+=09= =09%u=20represent=20a=20length;=20third=20%u=20represents=20an=20offset=20= */=0A+=09=09=09=09=09=09=20=20psprintf(_("attribute=20%u=20with=20length=20= %u=20starts=20at=20offset=20%u=20beyond=20total=20tuple=20length=20%u"),=0A= +=09=09=09=09=09=09=09=09=20=20=20ctx->attnum,=0A+=09=09=09=09=09=09=09=09= =20=20=20thisatt->attlen,=0A+=09=09=09=09=09=09=09=09=20=20=20= ctx->tuphdr->t_hoff=20+=20ctx->offset,=0A+=09=09=09=09=09=09=09=09=20=20=20= ctx->lp_len));=0A+=09=09return=20false;=0A+=09}=0A+=0A+=09/*=20Skip=20= null=20values=20*/=0A+=09if=20(infomask=20&=20HEAP_HASNULL=20&&=20= att_isnull(ctx->attnum,=20ctx->tuphdr->t_bits))=0A+=09=09return=20true;=0A= +=0A+=09/*=20Skip=20non-varlena=20values,=20but=20update=20offset=20= first=20*/=0A+=09if=20(thisatt->attlen=20!=3D=20-1)=0A+=09{=0A+=09=09= ctx->offset=20=3D=20att_align_nominal(ctx->offset,=20thisatt->attalign);=0A= +=09=09ctx->offset=20=3D=20att_addlength_pointer(ctx->offset,=20= thisatt->attlen,=0A+=09=09=09=09=09=09=09=09=09=09=09tp=20+=20= ctx->offset);=0A+=09=09if=20(ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20= ctx->lp_len)=0A+=09=09{=0A+=09=09=09report_corruption(ctx,=0A+=09=09=09= /*------=0A+=09=09=09translator:=20first=20%u=20represents=20an=20= attribute=20number;=20second=20and=0A+=09=09=09fourth=20%u=20represent=20= a=20length;=20third=20%u=20represents=20an=20offset=20*/=0A+=09=09=09=09=09= =09=09=20=20psprintf(_("attribute=20%u=20with=20length=20%u=20ends=20at=20= offset=20%u=20beyond=20total=20tuple=20length=20%u"),=0A+=09=09=09=09=09=09= =09=09=09=20=20=20ctx->attnum,=0A+=09=09=09=09=09=09=09=09=09=20=20=20= thisatt->attlen,=0A+=09=09=09=09=09=09=09=09=09=20=20=20= ctx->tuphdr->t_hoff=20+=20ctx->offset,=0A+=09=09=09=09=09=09=09=09=09=20=20= =20ctx->lp_len));=0A+=09=09=09return=20false;=0A+=09=09}=0A+=09=09return=20= true;=0A+=09}=0A+=0A+=09/*=20Ok,=20we're=20looking=20at=20a=20varlena=20= attribute.=20*/=0A+=09ctx->offset=20=3D=20att_align_pointer(ctx->offset,=20= thisatt->attalign,=20-1,=0A+=09=09=09=09=09=09=09=09=09tp=20+=20= ctx->offset);=0A+=0A+=09/*=20Get=20the=20(possibly=20corrupt)=20varlena=20= datum=20*/=0A+=09attdatum=20=3D=20fetchatt(thisatt,=20tp=20+=20= ctx->offset);=0A+=0A+=09/*=0A+=09=20*=20We=20have=20the=20datum,=20but=20= we=20cannot=20decode=20it=20carelessly,=20as=20it=20may=20still=0A+=09=20= *=20be=20corrupt.=0A+=09=20*/=0A+=0A+=09/*=0A+=09=20*=20Check=20that=20= VARTAG_SIZE=20won't=20hit=20a=20TrapMacro=20on=20a=20corrupt=20va_tag=20= before=0A+=09=20*=20risking=20a=20call=20into=20att_addlength_pointer=0A= +=09=20*/=0A+=09if=20(VARATT_IS_EXTERNAL(tp=20+=20ctx->offset))=0A+=09{=0A= +=09=09uint8=09=09va_tag=20=3D=20VARTAG_EXTERNAL(tp=20+=20ctx->offset);=0A= +=0A+=09=09if=20(va_tag=20!=3D=20VARTAG_ONDISK)=0A+=09=09{=0A+=09=09=09= report_corruption(ctx,=0A+=09=09=09/*------=0A+=09=09=09translator:=20= first=20%u=20represents=20an=20attribute=20number,=20second=20%u=0A+=09=09= =09represents=20an=20enumeration=20value=20*/=0A+=09=09=09=09=09=09=09=20= =20psprintf(_("toasted=20attribute=20%u=20has=20unexpected=20TOAST=20tag=20= %u"),=0A+=09=09=09=09=09=09=09=09=09=20=20=20ctx->attnum,=0A+=09=09=09=09= =09=09=09=09=09=20=20=20va_tag));=0A+=09=09=09/*=20We=20can't=20know=20= where=20the=20next=20attribute=20begins=20*/=0A+=09=09=09return=20false;=0A= +=09=09}=0A+=09}=0A+=0A+=09/*=20Ok,=20should=20be=20safe=20now=20*/=0A+=09= ctx->offset=20=3D=20att_addlength_pointer(ctx->offset,=20= thisatt->attlen,=0A+=09=09=09=09=09=09=09=09=09=09tp=20+=20ctx->offset);=0A= +=0A+=09if=20(ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20ctx->lp_len)=0A= +=09{=0A+=09=09report_corruption(ctx,=0A+=09=09/*------=0A+=09=09= translator:=20first=20%u=20represents=20an=20attribute=20number;=20= second=20and=20fourth=0A+=09=09%u=20represent=20a=20length;=20third=20%u=20= represents=20an=20offset=20*/=0A+=09=09=09=09=09=09=20=20= psprintf(_("attribute=20%u=20with=20length=20%u=20ends=20at=20offset=20= %u=20beyond=20total=20tuple=20length=20%u"),=0A+=09=09=09=09=09=09=09=09=20= =20=20ctx->attnum,=0A+=09=09=09=09=09=09=09=09=20=20=20thisatt->attlen,=0A= +=09=09=09=09=09=09=09=09=20=20=20ctx->tuphdr->t_hoff=20+=20ctx->offset,=0A= +=09=09=09=09=09=09=09=09=20=20=20ctx->lp_len));=0A+=0A+=09=09return=20= false;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20heap_deform_tuple=20would=20be=20= done=20with=20this=20attribute=20at=20this=20point,=0A+=09=20*=20having=20= stored=20it=20in=20values[],=20and=20would=20continue=20to=20the=20next=20= attribute.=0A+=09=20*=20We=20go=20further,=20because=20we=20need=20to=20= check=20if=20the=20toast=20datum=20is=20corrupt.=0A+=09=20*/=0A+=0A+=09= attr=20=3D=20(struct=20varlena=20*)=20DatumGetPointer(attdatum);=0A+=0A+=09= /*=0A+=09=20*=20Now=20we=20follow=20the=20logic=20of=20= detoast_external_attr(),=20with=20the=20same=0A+=09=20*=20caveats=20= about=20being=20paranoid=20about=20corruption.=0A+=09=20*/=0A+=0A+=09/*=20= Skip=20values=20that=20are=20not=20external=20*/=0A+=09if=20= (!VARATT_IS_EXTERNAL(attr))=0A+=09=09return=20true;=0A+=0A+=09/*=20It=20= is=20external,=20and=20we're=20looking=20at=20a=20page=20on=20disk=20*/=0A= +=0A+=09/*=20The=20tuple=20header=20better=20claim=20to=20contain=20= toasted=20values=20*/=0A+=09if=20(!(infomask=20&=20HEAP_HASEXTERNAL))=0A= +=09{=0A+=09=09report_corruption(ctx,=0A+=09=09/*------=0A+=09=09= translator:=20%u=20represents=20the=20attribute=20number=20*/=0A+=09=09=09= =09=09=09=20=20psprintf(_("attribute=20%u=20is=20external=20but=20tuple=20= header=20flag=20HEAP_HASEXTERNAL=20not=20set"),=0A+=09=09=09=09=09=09=09=09= =20=20=20ctx->attnum));=0A+=09=09return=20true;=0A+=09}=0A+=0A+=09/*=20= The=20relation=20better=20have=20a=20toast=20table=20*/=0A+=09if=20= (!ctx->rel->rd_rel->reltoastrelid)=0A+=09{=0A+=09=09= report_corruption(ctx,=0A+=09=09/*------=0A+=09=09translator:=20%u=20= represents=20the=20attribute=20number=20*/=0A+=09=09=09=09=09=09=20=20= psprintf(_("attribute=20%u=20is=20external=20but=20relation=20has=20no=20= toast=20relation"),=0A+=09=09=09=09=09=09=09=09=20=20=20ctx->attnum));=0A= +=09=09return=20true;=0A+=09}=0A+=0A+=09/*=20If=20we=20were=20told=20to=20= skip=20toast=20checking,=20then=20we're=20done.=20*/=0A+=09if=20= (ctx->toast_rel=20=3D=3D=20NULL)=0A+=09=09return=20true;=0A+=0A+=09/*=0A= +=09=20*=20Must=20copy=20attr=20into=20toast_pointer=20for=20alignment=20= considerations=0A+=09=20*/=0A+=09= VARATT_EXTERNAL_GET_POINTER(toast_pointer,=20attr);=0A+=0A+=09= ctx->attrsize=20=3D=20toast_pointer.va_extsize;=0A+=09ctx->endchunk=20=3D=20= (ctx->attrsize=20-=201)=20/=20TOAST_MAX_CHUNK_SIZE;=0A+=09= ctx->totalchunks=20=3D=20ctx->endchunk=20+=201;=0A+=0A+=09/*=0A+=09=20*=20= Setup=20a=20scan=20key=20to=20find=20chunks=20in=20toast=20table=20with=20= matching=20va_valueid=0A+=09=20*/=0A+=09ScanKeyInit(&toastkey,=0A+=09=09=09= =09(AttrNumber)=201,=0A+=09=09=09=09BTEqualStrategyNumber,=20F_OIDEQ,=0A= +=09=09=09=09ObjectIdGetDatum(toast_pointer.va_valueid));=0A+=0A+=09/*=0A= +=09=20*=20Check=20if=20any=20chunks=20for=20this=20toasted=20object=20= exist=20in=20the=20toast=20table,=0A+=09=20*=20accessible=20via=20the=20= index.=0A+=09=20*/=0A+=09init_toast_snapshot(&SnapshotToast);=0A+=09= toastscan=20=3D=20systable_beginscan_ordered(ctx->toast_rel,=0A+=09=09=09= =09=09=09=09=09=09=09=20=20=20ctx->valid_toast_index,=0A+=09=09=09=09=09=09= =09=09=09=09=20=20=20&SnapshotToast,=201,=0A+=09=09=09=09=09=09=09=09=09=09= =20=20=20&toastkey);=0A+=09ctx->chunkno=20=3D=200;=0A+=09found_toasttup=20= =3D=20false;=0A+=09while=20((toasttup=20=3D=0A+=09=09=09= systable_getnext_ordered(toastscan,=0A+=09=09=09=09=09=09=09=09=09=20= ForwardScanDirection))=20!=3D=20NULL)=0A+=09{=0A+=09=09found_toasttup=20= =3D=20true;=0A+=09=09check_toast_tuple(toasttup,=20ctx);=0A+=09=09= ctx->chunkno++;=0A+=09}=0A+=09if=20(ctx->chunkno=20!=3D=20(ctx->endchunk=20= +=201))=0A+=09=09report_corruption(ctx,=0A+=09=09/*------=0A+=09=09= translator:=20both=20%u=20represent=20a=20chunk=20number=20*/=0A+=09=09=09= =09=09=09=20=20psprintf(_("final=20toast=20chunk=20number=20%u=20differs=20= from=20expected=20value=20%u"),=0A+=09=09=09=09=09=09=09=09=20=20=20= ctx->chunkno,=20(ctx->endchunk=20+=201)));=0A+=09if=20(!found_toasttup)=0A= +=09=09report_corruption(ctx,=0A+=09=09/*------=0A+=09=09translator:=20= %u=20represents=20the=20attribute=20number=20*/=0A+=09=09=09=09=09=09=20=20= psprintf(_("toasted=20value=20for=20attribute=20%u=20missing=20from=20= toast=20table"),=0A+=09=09=09=09=09=09=09=09=20=20=20ctx->attnum));=0A+=09= systable_endscan_ordered(toastscan);=0A+=0A+=09return=20true;=0A+}=0A+=0A= +/*=0A+=20*=20Check=20the=20current=20tuple=20as=20tracked=20in=20ctx,=20= recording=20any=20corruption=20found=20in=0A+=20*=20ctx->tupstore.=0A+=20= */=0A+static=20void=0A+check_tuple(HeapCheckContext=20*ctx)=0A+{=0A+=09= TransactionId=20xmin;=0A+=09TransactionId=20xmax;=0A+=09bool=09=09fatal=20= =3D=20false;=0A+=09uint16=09=09infomask=20=3D=20ctx->tuphdr->t_infomask;=0A= +=0A+=09/*=0A+=09=20*=20If=20we=20report=20corruption=20before=20= iterating=20over=20individual=20attributes,=20we=0A+=09=20*=20need=20= attnum=20to=20be=20reported=20as=20NULL.=20=20Set=20that=20up=20before=20= any=20corruption=0A+=09=20*=20reporting=20might=20happen.=0A+=09=20*/=0A= +=09ctx->attnum=20=3D=20-1;=0A+=0A+=09/*=0A+=09=20*=20If=20the=20line=20= pointer=20for=20this=20tuple=20does=20not=20reserve=20enough=20space=20= for=20a=0A+=09=20*=20complete=20tuple=20header,=20we=20dare=20not=20read=20= the=20tuple=20header.=0A+=09=20*/=0A+=09if=20(ctx->lp_len=20<=20= MAXALIGN(SizeofHeapTupleHeader))=0A+=09{=0A+=09=09report_corruption(ctx,=0A= +=09=09/*------=0A+=09=09translator:=20first=20%u=20represents=20a=20= length,=20second=20%u=20represents=20a=20size=0A+=09=09*/=0A+=09=09=09=09= =09=09=20=20psprintf(_("line=20pointer=20length=20%u=20is=20less=20than=20= the=20minimum=20tuple=20header=20size=20%u"),=0A+=09=09=09=09=09=09=09=09= =20=20=20ctx->lp_len,=20(uint32)=20MAXALIGN(SizeofHeapTupleHeader)));=0A= +=09=09return;=0A+=09}=0A+=0A+=09/*=20If=20xmin=20is=20normal,=20it=20= should=20be=20within=20valid=20range=20*/=0A+=09xmin=20=3D=20= HeapTupleHeaderGetXmin(ctx->tuphdr);=0A+=09switch=20= (get_xid_status(xmin,=20ctx,=20NULL))=0A+=09{=0A+=09=09case=20= XID_INVALID:=0A+=09=09case=20XID_BOUNDS_OK:=0A+=09=09=09break;=0A+=09=09= case=20XID_IN_FUTURE:=0A+=09=09=09report_corruption(ctx,=0A+=09=09=09= /*------=0A+=09=09=09translator:=20first=20and=20third=20%u=20are=20a=20= transaction=20identifier,=20second=0A+=09=09=09%u=20is=20an=20epoch=20*/=0A= +=09=09=09=09=09=09=09=20=20psprintf(_("xmin=20%u=20equals=20or=20= exceeds=20next=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09= =09=09=09=20=20=20xmin,=0A+=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->next_fxid),=0A+=09=09=09=09=09=09=09=09=09= =20=20=20XidFromFullTransactionId(ctx->next_fxid)));=0A+=09=09=09fatal=20= =3D=20true;=0A+=09=09=09break;=0A+=09=09case=20XID_PRECEDES_DATMIN:=0A+=09= =09=09report_corruption(ctx,=0A+=09=09=09/*------=0A+=09=09=09= translator:=20first=20and=20third=20%u=20are=20a=20transaction=20= identifier,=20second=0A+=09=09=09%u=20is=20an=20epoch=20*/=0A+=09=09=09=09= =09=09=09=20=20psprintf(_("xmin=20%u=20precedes=20oldest=20valid=20= transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09=20=20=20xmin,=0A= +=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->oldest_fxid),=0A+=09=09=09=09=09=09=09=09= =09=20=20=20XidFromFullTransactionId(ctx->oldest_fxid)));=0A+=09=09=09= fatal=20=3D=20true;=0A+=09=09=09break;=0A+=09=09case=20= XID_PRECEDES_RELMIN:=0A+=09=09=09report_corruption(ctx,=0A+=09=09=09= /*------=0A+=09=09=09translator:=20first=20and=20third=20%u=20are=20a=20= transaction=20identifier,=20second=0A+=09=09=09%u=20is=20an=20epoch=20*/=0A= +=09=09=09=09=09=09=09=20=20psprintf(_("xmin=20%u=20precedes=20relation=20= freeze=20threshold=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09=20=20=20= xmin,=0A+=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->relfrozenfxid),=0A+=09=09=09=09=09=09=09=09= =09=20=20=20XidFromFullTransactionId(ctx->relfrozenfxid)));=0A+=09=09=09= fatal=20=3D=20true;=0A+=09=09=09break;=0A+=09}=0A+=0A+=09xmax=20=3D=20= HeapTupleHeaderGetRawXmax(ctx->tuphdr);=0A+=0A+=09/*=20If=20xmax=20is=20= a=20multixact,=20it=20should=20be=20within=20valid=20range=20*/=0A+=09if=20= (infomask=20&=20HEAP_XMAX_IS_MULTI)=0A+=09{=0A+=09=09switch=20= (check_mxid_valid_in_rel(xmax,=20ctx))=0A+=09=09{=0A+=09=09=09case=20= XID_INVALID:=0A+=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09=09=09=09= =09=20=20pstrdup(_("multitransaction=20ID=20is=20invalid")));=0A+=09=09=09= =09fatal=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20= XID_PRECEDES_RELMIN:=0A+=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09= /*------=0A+=09=09=09=09translator:=20both=20%u=20are=20multitransaction=20= IDs=20*/=0A+=09=09=09=09=09=09=09=09=20=20psprintf(_("multitransaction=20= ID=20%u=20precedes=20relation=20minimum=20multitransaction=20ID=20= threshold=20%u"),=0A+=09=09=09=09=09=09=09=09=09=09=20=20=20xmax,=20= ctx->relminmxid));=0A+=09=09=09=09fatal=20=3D=20true;=0A+=09=09=09=09= break;=0A+=09=09=09case=20XID_PRECEDES_DATMIN:=0A+=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09/*------=0A+=09=09=09=09= translator:=20Both=20%u=20are=20multitransaction=20IDs=20*/=0A+=09=09=09=09= =09=09=09=09=20=20psprintf(_("multitransaction=20ID=20%u=20precedes=20= oldest=20valid=20multitransaction=20ID=20threshold=20%u"),=0A+=09=09=09=09= =09=09=09=09=09=09=20=20=20xmax,=20ctx->oldest_mxact));=0A+=09=09=09=09= fatal=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20= XID_IN_FUTURE:=0A+=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09= /*------=0A+=09=09=09=09translator:=20%u=20is=20a=20multitransaction=20= ID=20*/=0A+=09=09=09=09=09=09=09=09=20=20psprintf(_("multitransaction=20= ID=20%u=20equals=20or=20exceeds=20next=20valid=20multitransaction=20ID=20= %u"),=0A+=09=09=09=09=09=09=09=09=09=09=20=20=20xmax,=0A+=09=09=09=09=09=09= =09=09=09=09=20=20=20ctx->next_mxact));=0A+=09=09=09=09fatal=20=3D=20= true;=0A+=09=09=09=09break;=0A+=09=09=09case=20XID_BOUNDS_OK:=0A+=09=09=09= =09break;=0A+=09=09}=0A+=09}=0A+=09/*=20If=20xmax=20is=20not=20a=20= multixact=20and=20is=20normal,=20it=20should=20be=20within=20valid=20= range=20*/=0A+=09else=0A+=09{=0A+=09=09switch=20(get_xid_status(xmax,=20= ctx,=20NULL))=0A+=09=09{=0A+=09=09=09case=20XID_INVALID:=0A+=09=09=09= case=20XID_BOUNDS_OK:=0A+=09=09=09=09break;=0A+=09=09=09case=20= XID_IN_FUTURE:=0A+=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09= /*------=0A+=09=09=09=09translator:=20first=20and=20third=20%u=20are=20a=20= transaction=20identifier,=0A+=09=09=09=09second=20%u=20is=20an=20epoch=20= */=0A+=09=09=09=09=09=09=09=09=20=20psprintf(_("xmax=20%u=20equals=20or=20= exceeds=20next=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09= =09=09=09=09=20=20=20xmax,=0A+=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->next_fxid),=0A+=09=09=09=09=09=09=09=09=09= =09=20=20=20XidFromFullTransactionId(ctx->next_fxid)));=0A+=09=09=09=09= fatal=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20= XID_PRECEDES_DATMIN:=0A+=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09= /*------=0A+=09=09=09=09translator:=20first=20and=20third=20%u=20are=20a=20= transaction=20identifier,=0A+=09=09=09=09second=20%u=20is=20an=20epoch=20= */=0A+=09=09=09=09=09=09=09=09=20=20psprintf(_("xmax=20%u=20precedes=20= oldest=20valid=20transaction=20ID=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09= =09=20=20=20xmax,=0A+=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->oldest_fxid),=0A+=09=09=09=09=09=09=09=09= =09=09=20=20=20XidFromFullTransactionId(ctx->oldest_fxid)));=0A+=09=09=09= =09fatal=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20= XID_PRECEDES_RELMIN:=0A+=09=09=09=09report_corruption(ctx,=0A+=09=09=09=09= /*------=0A+=09=09=09=09translator:=20first=20and=20third=20%u=20are=20a=20= transaction=20identifier,=0A+=09=09=09=09second=20%u=20is=20an=20epoch=20= */=0A+=09=09=09=09=09=09=09=09=20=20psprintf(_("xmax=20%u=20precedes=20= relation=20freeze=20threshold=20%u:%u"),=0A+=09=09=09=09=09=09=09=09=09=09= =20=20=20xmax,=0A+=09=09=09=09=09=09=09=09=09=09=20=20=20= EpochFromFullTransactionId(ctx->relfrozenfxid),=0A+=09=09=09=09=09=09=09=09= =09=09=20=20=20XidFromFullTransactionId(ctx->relfrozenfxid)));=0A+=09=09=09= =09fatal=20=3D=20true;=0A+=09=09}=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20= Cannot=20process=20tuple=20data=20if=20tuple=20header=20was=20corrupt,=20= as=20the=20offsets=0A+=09=20*=20within=20the=20page=20cannot=20be=20= trusted,=20leaving=20too=20much=20risk=20of=20reading=0A+=09=20*=20= garbage=20if=20we=20continue.=0A+=09=20*=0A+=09=20*=20We=20also=20cannot=20= process=20the=20tuple=20if=20the=20xmin=20or=20xmax=20were=20invalid=0A+=09= =20*=20relative=20to=20relfrozenxid=20or=20relminmxid,=20as=20clog=20= entries=20for=20the=20xids=0A+=09=20*=20may=20already=20be=20gone.=0A+=09= =20*/=0A+=09if=20(fatal)=0A+=09=09return;=0A+=0A+=09/*=0A+=09=20*=20= Check=20various=20forms=20of=20tuple=20header=20corruption.=20=20If=20= the=20header=20is=20too=0A+=09=20*=20corrupt=20to=20continue=20checking,=20= or=20if=20the=20tuple=20is=20not=20visible=20to=20anyone,=0A+=09=20*=20= we=20cannot=20continue=20with=20other=20checks.=0A+=09=20*/=0A+=09if=20= (!check_tuple_header_and_visibilty(ctx->tuphdr,=20ctx))=0A+=09=09return;=0A= +=0A+=09/*=0A+=09=20*=20The=20tuple=20is=20visible,=20so=20it=20must=20= be=20compatible=20with=20the=20current=20version=0A+=09=20*=20of=20the=20= relation=20descriptor.=20It=20might=20have=20fewer=20columns=20than=20= are=0A+=09=20*=20present=20in=20the=20relation=20descriptor,=20but=20it=20= cannot=20have=20more.=0A+=09=20*/=0A+=09if=20= (RelationGetDescr(ctx->rel)->natts=20<=20ctx->natts)=0A+=09{=0A+=09=09= report_corruption(ctx,=0A+=09=09/*------=0A+=09=09translator:=20both=20= %u=20are=20a=20number=20*/=0A+=09=09=09=09=09=09=20=20psprintf(_("number=20= of=20attributes=20%u=20exceeds=20maximum=20expected=20for=20table=20= %u"),=0A+=09=09=09=09=09=09=09=09=20=20=20ctx->natts,=0A+=09=09=09=09=09=09= =09=09=20=20=20RelationGetDescr(ctx->rel)->natts));=0A+=09=09return;=0A+=09= }=0A+=0A+=09/*=0A+=09=20*=20Check=20each=20attribute=20unless=20we=20hit=20= corruption=20that=20confuses=20what=20to=20do=0A+=09=20*=20next,=20at=20= which=20point=20we=20abort=20further=20attribute=20checks=20for=20this=20= tuple.=0A+=09=20*=20Note=20that=20we=20don't=20abort=20for=20all=20types=20= of=20corruption,=20only=20for=20those=0A+=09=20*=20types=20where=20we=20= don't=20know=20how=20to=20continue.=0A+=09=20*/=0A+=09ctx->offset=20=3D=20= 0;=0A+=09for=20(ctx->attnum=20=3D=200;=20ctx->attnum=20<=20ctx->natts;=20= ctx->attnum++)=0A+=09=09if=20(!check_tuple_attribute(ctx))=0A+=09=09=09= break;=09=09=09=09/*=20cannot=20continue=20*/=0A+}=0A+=0A+/*=0A+=20*=20= Convert=20a=20TransactionId=20into=20a=20FullTransactionId=20using=20our=20= cached=20values=20of=0A+=20*=20the=20valid=20transaction=20ID=20range.=20= =20It=20is=20the=20caller's=20responsibility=20to=20have=0A+=20*=20= already=20updated=20the=20cached=20values,=20if=20necessary.=0A+=20*/=0A= +static=20FullTransactionId=0A= +FullTransactionIdFromXidAndCtx(TransactionId=20xid,=20const=20= HeapCheckContext=20*ctx)=0A+{=0A+=09uint32=09=09epoch;=0A+=0A+=09if=20= (!TransactionIdIsNormal(xid))=0A+=09=09return=20= FullTransactionIdFromEpochAndXid(0,=20xid);=0A+=09epoch=20=3D=20= EpochFromFullTransactionId(ctx->next_fxid);=0A+=09if=20(xid=20>=20= ctx->next_xid)=0A+=09=09epoch--;=0A+=09return=20= FullTransactionIdFromEpochAndXid(epoch,=20xid);=0A+}=0A+=0A+/*=0A+=20*=20= Update=20our=20cached=20range=20of=20valid=20transaction=20IDs.=0A+=20*/=0A= +static=20void=0A+update_cached_xid_range(HeapCheckContext=20*ctx)=0A+{=0A= +=09/*=20Make=20cached=20copies=20*/=0A+=09LWLockAcquire(XidGenLock,=20= LW_SHARED);=0A+=09ctx->next_fxid=20=3D=20ShmemVariableCache->nextXid;=0A= +=09ctx->oldest_xid=20=3D=20ShmemVariableCache->oldestXid;=0A+=09= LWLockRelease(XidGenLock);=0A+=0A+=09/*=20And=20compute=20alternate=20= versions=20of=20the=20same=20*/=0A+=09ctx->oldest_fxid=20=3D=20= FullTransactionIdFromXidAndCtx(ctx->oldest_xid,=20ctx);=0A+=09= ctx->next_xid=20=3D=20XidFromFullTransactionId(ctx->next_fxid);=0A+}=0A+=0A= +/*=0A+=20*=20Update=20our=20cached=20range=20of=20valid=20= multitransaction=20IDs.=0A+=20*/=0A+static=20void=0A= +update_cached_mxid_range(HeapCheckContext=20*ctx)=0A+{=0A+=09= ReadMultiXactIdRange(&ctx->oldest_mxact,=20&ctx->next_mxact);=0A+}=0A+=0A= +/*=0A+=20*=20Return=20whether=20the=20given=20FullTransactionId=20is=20= within=20our=20cached=20valid=0A+=20*=20transaction=20ID=20range.=0A+=20= */=0A+static=20inline=20bool=0A+fxid_in_cached_range(FullTransactionId=20= fxid,=20const=20HeapCheckContext=20*ctx)=0A+{=0A+=09return=20= (FullTransactionIdPrecedesOrEquals(ctx->oldest_fxid,=20fxid)=20&&=0A+=09=09= =09FullTransactionIdPrecedes(fxid,=20ctx->next_fxid));=0A+}=0A+=0A+/*=0A= +=20*=20Checks=20wheter=20a=20multitransaction=20ID=20is=20in=20the=20= cached=20valid=20range,=20returning=0A+=20*=20the=20nature=20of=20the=20= range=20violation,=20if=20any.=0A+=20*/=0A+static=20XidBoundsViolation=0A= +check_mxid_in_range(MultiXactId=20mxid,=20HeapCheckContext=20*ctx)=0A+{=0A= +=09if=20(!TransactionIdIsValid(mxid))=0A+=09=09return=20XID_INVALID;=0A= +=09if=20(MultiXactIdPrecedes(mxid,=20ctx->relminmxid))=0A+=09=09return=20= XID_PRECEDES_RELMIN;=0A+=09if=20(MultiXactIdPrecedes(mxid,=20= ctx->oldest_mxact))=0A+=09=09return=20XID_PRECEDES_DATMIN;=0A+=09if=20= (MultiXactIdPrecedesOrEquals(ctx->next_mxact,=20mxid))=0A+=09=09return=20= XID_IN_FUTURE;=0A+=09return=20XID_BOUNDS_OK;=0A+}=0A+=0A+/*=0A+=20*=20= Checks=20whether=20the=20given=20mxid=20is=20valid=20to=20appear=20in=20= the=20heap=20being=20checked,=0A+=20*=20returning=20the=20nature=20of=20= the=20range=20violation,=20if=20any.=0A+=20*=0A+=20*=20This=20function=20= attempts=20to=20return=20quickly=20by=20caching=20the=20known=20valid=20= mxid=0A+=20*=20range=20in=20ctx.=20=20Callers=20should=20already=20have=20= performed=20the=20initial=20setup=20of=0A+=20*=20the=20cache=20prior=20= to=20the=20first=20call=20to=20this=20function.=0A+=20*/=0A+static=20= XidBoundsViolation=0A+check_mxid_valid_in_rel(MultiXactId=20mxid,=20= HeapCheckContext=20*ctx)=0A+{=0A+=09XidBoundsViolation=20result;=0A+=0A+=09= result=20=3D=20check_mxid_in_range(mxid,=20ctx);=0A+=09if=20(result=20=3D=3D= =20XID_BOUNDS_OK)=0A+=09=09return=20XID_BOUNDS_OK;=0A+=0A+=09/*=20The=20= range=20may=20have=20advanced.=20=20Recheck.=20*/=0A+=09= update_cached_mxid_range(ctx);=0A+=09return=20check_mxid_in_range(mxid,=20= ctx);=0A+}=0A+=0A+/*=0A+=20*=20Checks=20whether=20the=20given=20= transaction=20ID=20is=20(or=20was=20recently)=20valid=20to=20appear=0A+=20= *=20in=20the=20heap=20being=20checked,=20or=20whether=20it=20is=20too=20= old=20or=20too=20new=20to=20appear=20in=0A+=20*=20the=20relation,=20= returning=20information=20about=20the=20nature=20of=20the=20bounds=20= violation.=0A+=20*=0A+=20*=20We=20cache=20the=20range=20of=20valid=20= transaction=20IDs.=20=20If=20xid=20is=20in=20that=20range,=20we=0A+=20*=20= conclude=20that=20it=20is=20valid,=20even=20though=20concurrent=20= changes=20to=20the=20table=20might=0A+=20*=20invalidate=20it=20under=20= certain=20corrupt=20conditions.=20=20(For=20example,=20if=20the=20table=0A= +=20*=20contains=20corrupt=20all-frozen=20bits,=20a=20concurrent=20= vacuum=20might=20skip=20the=20page(s)=0A+=20*=20containing=20the=20xid=20= and=20then=20truncate=20clog=20and=20advance=20the=20relfrozenxid=0A+=20= *=20beyond=20xid.)=20Reporting=20the=20xid=20as=20valid=20under=20such=20= conditions=20seems=0A+=20*=20acceptable,=20since=20if=20we=20had=20= checked=20it=20earlier=20in=20our=20scan=20it=20would=20have=0A+=20*=20= truly=20been=20valid=20at=20that=20time.=0A+=20*=0A+=20*=20If=20the=20= status=20argument=20is=20not=20NULL,=20and=20if=20and=20only=20if=20the=20= transaction=20ID=0A+=20*=20appears=20to=20be=20valid=20in=20this=20= relation,=20clog=20will=20be=20consulted=20and=20the=20commit=0A+=20*=20= status=20argument=20will=20be=20set=20with=20the=20status=20of=20the=20= transaction=20ID.=0A+=20*/=0A+static=20XidBoundsViolation=0A= +get_xid_status(TransactionId=20xid,=20HeapCheckContext=20*ctx,=20= XidCommitStatus=20*status)=0A+{=0A+=09XidBoundsViolation=20result;=0A+=09= FullTransactionId=20fxid;=0A+=09FullTransactionId=20clog_horizon;=0A+=0A= +=09/*=20Quick=20check=20for=20special=20xids=20*/=0A+=09if=20= (!TransactionIdIsValid(xid))=0A+=09=09result=20=3D=20XID_INVALID;=0A+=09= else=20if=20(xid=20=3D=3D=20BootstrapTransactionId=20||=20xid=20=3D=3D=20= FrozenTransactionId)=0A+=09=09result=20=3D=20XID_BOUNDS_OK;=0A+=09else=0A= +=09{=0A+=09=09/*=20Check=20if=20the=20xid=20is=20within=20bounds=20*/=0A= +=09=09fxid=20=3D=20FullTransactionIdFromXidAndCtx(xid,=20ctx);=0A+=09=09= if=20(!fxid_in_cached_range(fxid,=20ctx))=0A+=09=09{=0A+=09=09=09/*=0A+=09= =09=09=20*=20We=20may=20have=20been=20checking=20against=20stale=20= values.=20=20Update=20the=0A+=09=09=09=20*=20cached=20range=20to=20be=20= sure,=20and=20since=20we=20relied=20on=20the=20cached=0A+=09=09=09=20*=20= range=20when=20we=20performed=20the=20full=20xid=20conversion,=20= reconvert.=0A+=09=09=09=20*/=0A+=09=09=09update_cached_xid_range(ctx);=0A= +=09=09=09fxid=20=3D=20FullTransactionIdFromXidAndCtx(xid,=20ctx);=0A+=09= =09}=0A+=0A+=09=09if=20= (FullTransactionIdPrecedesOrEquals(ctx->next_fxid,=20fxid))=0A+=09=09=09= result=20=3D=20XID_IN_FUTURE;=0A+=09=09else=20if=20= (FullTransactionIdPrecedes(fxid,=20ctx->oldest_fxid))=0A+=09=09=09result=20= =3D=20XID_PRECEDES_DATMIN;=0A+=09=09else=20if=20= (FullTransactionIdPrecedes(fxid,=20ctx->relfrozenfxid))=0A+=09=09=09= result=20=3D=20XID_PRECEDES_RELMIN;=0A+=09=09else=0A+=09=09=09result=20=3D= =20XID_BOUNDS_OK;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20Early=20return=20if=20= the=20caller=20does=20not=20request=20clog=20checking,=20or=20if=20the=0A= +=09=20*=20xid=20is=20already=20known=20to=20be=20out=20of=20bounds.=20=20= We=20dare=20not=20check=20clog=20for=0A+=09=20*=20out=20of=20bounds=20= transaction=20IDs.=0A+=09=20*/=0A+=09if=20(status=20=3D=3D=20NULL=20||=20= result=20!=3D=20XID_BOUNDS_OK)=0A+=09=09return=20result;=0A+=0A+=09/*=20= Early=20return=20if=20we=20just=20checked=20this=20xid=20in=20a=20prior=20= call=20*/=0A+=09if=20(xid=20=3D=3D=20ctx->cached_xid)=0A+=09{=0A+=09=09= *status=20=3D=20ctx->cached_status;=0A+=09=09return=20result;=0A+=09}=0A= +=0A+=09*status=20=3D=20XID_COMMITTED;=0A+=09= LWLockAcquire(XactTruncationLock,=20LW_SHARED);=0A+=09clog_horizon=20=3D=20= FullTransactionIdFromXidAndCtx(ShmemVariableCache->oldestClogXid,=20= ctx);=0A+=09if=20(FullTransactionIdPrecedesOrEquals(clog_horizon,=20= fxid))=0A+=09{=0A+=09=09if=20(TransactionIdIsCurrentTransactionId(xid))=0A= +=09=09=09*status=20=3D=20XID_IN_PROGRESS;=0A+=09=09else=20if=20= (TransactionIdDidCommit(xid))=0A+=09=09=09*status=20=3D=20XID_COMMITTED;=0A= +=09=09else=20if=20(TransactionIdDidAbort(xid))=0A+=09=09=09*status=20=3D=20= XID_ABORTED;=0A+=09=09else=0A+=09=09=09*status=20=3D=20XID_IN_PROGRESS;=0A= +=09}=0A+=09LWLockRelease(XactTruncationLock);=0A+=09ctx->cached_xid=20=3D= =20xid;=0A+=09ctx->cached_status=20=3D=20*status;=0A+=09return=20result;=0A= +}=0Adiff=20--git=20a/doc/src/sgml/amcheck.sgml=20= b/doc/src/sgml/amcheck.sgml=0Aindex=20a9df2c1a9d..a57781992a=20100644=0A= ---=20a/doc/src/sgml/amcheck.sgml=0A+++=20b/doc/src/sgml/amcheck.sgml=0A= @@=20-9,12=20+9,11=20@@=0A=20=0A=20=20=0A=20=20=20The=20= amcheck=20module=20provides=20functions=20that=20= allow=20you=20to=0A-=20=20verify=20the=20logical=20consistency=20of=20= the=20structure=20of=20relations.=20=20If=20the=0A-=20=20structure=20= appears=20to=20be=20valid,=20no=20error=20is=20raised.=0A+=20=20verify=20= the=20logical=20consistency=20of=20the=20structure=20of=20relations.=0A=20= =20=0A=20=0A=20=20=0A-=20=20The=20functions=20verify=20= various=20invariants=20in=20the=0A+=20=20The=20= B-Tree=20checking=20functions=20verify=20various=20= invariants=20in=20the=0A=20=20=20structure=20of=20= the=20representation=20of=20particular=20relations.=20=20The=0A=20=20=20= correctness=20of=20the=20access=20method=20functions=20behind=20index=20= scans=20and=0A=20=20=20other=20important=20operations=20relies=20on=20= these=20invariants=20always=0A@@=20-24,7=20+23,7=20@@=0A=20=20=20= collated=20lexical=20order).=20=20If=20that=20particular=20invariant=20= somehow=20fails=0A=20=20=20to=20hold,=20we=20can=20expect=20binary=20= searches=20on=20the=20affected=20page=20to=0A=20=20=20incorrectly=20= guide=20index=20scans,=20resulting=20in=20wrong=20answers=20to=20SQL=0A-=20= =20queries.=0A+=20=20queries.=20=20If=20the=20structure=20appears=20to=20= be=20valid,=20no=20error=20is=20raised.=0A=20=20=0A=20=20=0A= =20=20=20Verification=20is=20performed=20using=20the=20same=20procedures=20= as=20those=20used=20by=0A@@=20-35,7=20+34,22=20@@=0A=20=20=20functions.=0A= =20=20=0A=20=20=0A-=20=20amcheck=20= functions=20may=20only=20be=20used=20by=20superusers.=0A+=20=20Unlike=20= the=20B-Tree=20checking=20functions=20which=20report=20corruption=20by=20= raising=0A+=20=20errors,=20the=20heap=20checking=20function=20= verify_heapam=20checks=0A+=20=20a=20table=20and=20= attempts=20to=20return=20a=20set=20of=20rows,=20one=20row=20per=20= corruption=0A+=20=20detected.=20=20Despite=20this,=20if=20facilities=20= that=0A+=20=20verify_heapam=20relies=20upon=20are=20= themselves=20corrupted,=20the=0A+=20=20function=20may=20be=20unable=20to=20= continue=20and=20may=20instead=20raise=20an=20error.=0A+=20=0A+=20= =0A+=20=20Permission=20to=20execute=20amcheck=20= functions=20may=20be=20granted=0A+=20=20to=20non-superusers,=20but=20= before=20granting=20such=20permissions=20careful=20consideration=0A+=20=20= should=20be=20given=20to=20data=20security=20and=20privacy=20concerns.=20= =20Although=20the=0A+=20=20corruption=20reports=20generated=20by=20these=20= functions=20do=20not=20focus=20on=20the=20contents=0A+=20=20of=20the=20= corrupted=20data=20so=20much=20as=20on=20the=20structure=20of=20that=20= data=20and=20the=20nature=0A+=20=20of=20the=20corruptions=20found,=20an=20= attacker=20who=20gains=20permission=20to=20execute=20these=0A+=20=20= functions,=20particularly=20if=20the=20attacker=20can=20also=20induce=20= corruption,=20might=20be=0A+=20=20able=20to=20infer=20something=20of=20= the=20data=20itself=20from=20such=20messages.=0A=20=20=0A=20=0A=20= =20=0A@@=20-187,12=20+201,223=20@@=20SET=20client_min_messages=20= =3D=20DEBUG1;=0A=20=20=20=20=0A=20=20=20=0A=20=0A+=20=20= =0A+=20=20=20=0A+=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20=20verify_heapam(relation=20= regclass,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= on_error_stop=20boolean,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20check_toast=20boolean,=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20skip=20cstring,=0A+=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20startblock=20bigint,=0A+=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20endblock=20bigint,=0A+=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20blkno=20OUT=20bigint,=0A= +=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20offnum=20= OUT=20integer,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20attnum=20OUT=20integer,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20msg=20OUT=20text)=0A+=20=20=20=20=20=20returns=20record=0A= +=20=20=20=20=20=0A+=20=20=20=20=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=20Checks=20a=20= table=20for=20structural=20corruption,=20where=20pages=20in=20the=20= relation=0A+=20=20=20=20=20=20contain=20data=20that=20is=20invalidly=20= formatted,=20and=20for=20logical=20corruption,=0A+=20=20=20=20=20=20= where=20pages=20are=20structurally=20valid=20but=20inconsistent=20with=20= the=20rest=20of=20the=0A+=20=20=20=20=20=20database=20cluster.=20=20= Example=20usage:=0A+=0A+test=3D#=20select=20*=20from=20= verify_heapam('mytable',=20check_toast=20:=3D=20true);=0A+=20blkno=20|=20= offnum=20|=20attnum=20|=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20msg=0A= +-------+--------+--------+-----------------------------------------------= ---------------------------------------------------=0A+=20=20=20=2017=20= |=20=20=20=20=2012=20|=20=20=20=20=20=20=20=20|=20xmin=204294967295=20= precedes=20relation=20freeze=20threshold=2017:1134217582=0A+=20=20=20960=20= |=20=20=20=20=20=204=20|=20=20=20=20=20=20=20=20|=20data=20begins=20at=20= offset=20152=20beyond=20the=20tuple=20length=2058=0A+=20=20=20960=20|=20=20= =20=20=20=204=20|=20=20=20=20=20=20=20=20|=20tuple=20data=20should=20= begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20152=20(3=20= attributes,=20no=20nulls)=0A+=20=20=20960=20|=20=20=20=20=20=205=20|=20=20= =20=20=20=20=20=20|=20tuple=20data=20should=20begin=20at=20byte=2024,=20= but=20actually=20begins=20at=20byte=2027=20(3=20attributes,=20no=20= nulls)=0A+=20=20=20960=20|=20=20=20=20=20=206=20|=20=20=20=20=20=20=20=20= |=20tuple=20data=20should=20begin=20at=20byte=2024,=20but=20actually=20= begins=20at=20byte=2016=20(3=20attributes,=20no=20nulls)=0A+=20=20=20960=20= |=20=20=20=20=20=207=20|=20=20=20=20=20=20=20=20|=20tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 21=20(3=20attributes,=20no=20nulls)=0A+=20=201147=20|=20=20=20=20=20=202=20= |=20=20=20=20=20=20=20=20|=20number=20of=20attributes=202047=20exceeds=20= maximum=20expected=20for=20table=203=0A+=20=201147=20|=20=20=20=20=2010=20= |=20=20=20=20=20=20=20=20|=20tuple=20data=20should=20begin=20at=20byte=20= 280,=20but=20actually=20begins=20at=20byte=2024=20(2047=20attributes,=20= has=20nulls)=0A+=20=201147=20|=20=20=20=20=2015=20|=20=20=20=20=20=20=20=20= |=20number=20of=20attributes=2067=20exceeds=20maximum=20expected=20for=20= table=203=0A+=20=201147=20|=20=20=20=20=2016=20|=20=20=20=20=20=201=20|=20= attribute=201=20with=20length=204294967295=20ends=20at=20offset=20= 416848000=20beyond=20total=20tuple=20length=2058=0A+=20=201147=20|=20=20=20= =20=2018=20|=20=20=20=20=20=202=20|=20final=20toast=20chunk=20number=200=20= differs=20from=20expected=20value=206=0A+=20=201147=20|=20=20=20=20=2019=20= |=20=20=20=20=20=202=20|=20toasted=20value=20for=20attribute=202=20= missing=20from=20toast=20table=0A+=20=201147=20|=20=20=20=20=2021=20|=20=20= =20=20=20=20=20=20|=20tuple=20is=20marked=20as=20only=20locked,=20but=20= also=20claims=20key=20columns=20were=20updated=0A+=20=201147=20|=20=20=20= =20=2022=20|=20=20=20=20=20=20=20=20|=20multitransaction=20ID=201775655=20= is=20from=20before=20relation=20cutoff=202355572=0A+(14=20rows)=0A= +=0A+=20=20=20=20=20=20As=20this=20example=20shows,=20the=20= Tuple=20ID=20(TID)=20of=20the=20corrupt=20tuple=20is=20given=0A+=20=20=20= =20=20=20in=20the=20(blkno,=20= offnum)=20columns,=20and=0A+=20=20=20=20=20=20for=20= corruptions=20specific=20to=20a=20particular=20attribute=20in=20the=20= tuple,=20the=0A+=20=20=20=20=20=20attnum=20field=20= shows=20which=20one.=0A+=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20Structural=20corruption=20can=20happen=20due=20to=20= faulty=20storage=20hardware,=20or=0A+=20=20=20=20=20=20relation=20files=20= being=20overwritten=20or=20modified=20by=20unrelated=20software.=0A+=20=20= =20=20=20=20This=20kind=20of=20corruption=20can=20also=20be=20detected=20= with=0A+=20=20=20=20=20=20data=20page=0A+=20=20=20= =20=20=20checksums.=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=20Relation=20pages=20which=20are=20= correctly=20formatted,=20internally=20consistent,=20and=0A+=20=20=20=20=20= =20correct=20relative=20to=20their=20own=20internal=20checksums=20may=20= still=20contain=0A+=20=20=20=20=20=20logical=20corruption.=20=20As=20= such,=20this=20kind=20of=20corruption=20cannot=20be=20detected=0A+=20=20=20= =20=20=20with=20checksums.=20=20Examples=20= include=20toasted=0A+=20=20=20=20=20=20values=20in=20the=20main=20table=20= which=20lack=20a=20corresponding=20entry=20in=20the=20toast=0A+=20=20=20=20= =20=20table,=20and=20tuples=20in=20the=20main=20table=20with=20a=20= Transaction=20ID=20that=20is=20older=0A+=20=20=20=20=20=20than=20the=20= oldest=20valid=20Transaction=20ID=20in=20the=20database=20or=20cluster.=0A= +=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= Multiple=20causes=20of=20logical=20corruption=20have=20been=20observed=20= in=20production=0A+=20=20=20=20=20=20systems,=20including=20bugs=20in=20= the=20PostgreSQL=0A+=20=20=20=20=20=20server=20= software,=20faulty=20and=20ill-conceived=20backup=20and=20restore=20= tools,=20and=0A+=20=20=20=20=20=20user=20error.=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=20=20Corrupt=20= relations=20are=20most=20concerning=20in=20live=20production=20= environments,=0A+=20=20=20=20=20=20precisely=20the=20same=20environments=20= where=20high=20risk=20activities=20are=20least=0A+=20=20=20=20=20=20= welcome.=20=20For=20this=20reason,=20verify_heapam=20= has=20been=0A+=20=20=20=20=20=20designed=20to=20diagnose=20corruption=20= without=20undue=20risk.=20=20It=20cannot=20guard=0A+=20=20=20=20=20=20= against=20all=20causes=20of=20backend=20crashes,=20as=20even=20executing=20= the=20calling=0A+=20=20=20=20=20=20query=20could=20be=20unsafe=20on=20a=20= badly=20corrupted=20system.=20=20=20Access=20to=20catalog=20tables=20are=20= performed=20and=20could=0A+=20=20=20=20=20=20be=20problematic=20if=20the=20= catalogs=20themselves=20are=20corrupted.=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=20The=20design=20principle=20= adhered=20to=20in=20verify_heapam=20is=0A+=20=20=20=20= =20=20that,=20if=20the=20rest=20of=20the=20system=20and=20server=20= hardware=20are=20correct,=20under=0A+=20=20=20=20=20=20default=20= options,=20verify_heapam=20will=20not=20crash=20the=0A= +=20=20=20=20=20=20server=20due=20merely=20to=20structural=20or=20= logical=20corruption=20in=20the=20target=0A+=20=20=20=20=20=20table.=0A+=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20An=20= experimental=20option,=20check_toast,=20exists=20to=0A= +=20=20=20=20=20=20reconcile=20the=20target=20table=20against=20entries=20= in=20its=20corresponding=20toast=0A+=20=20=20=20=20=20table.=20=20This=20= option=20may=20change=20in=20future,=20is=20disabled=20by=20default,=20= and=20is=0A+=20=20=20=20=20=20known=20to=20be=20slow.=20=20It=20is=20= also=20unsafe=20under=20some=20conditions.=20=20If=20the=0A+=20=20=20=20=20= =20target=20relation's=20corresponding=20toast=20table=20or=20toast=20= index=20are=20corrupt,=0A+=20=20=20=20=20=20reconciling=20the=20target=20= table=20against=20toast=20values=20may=20be=20unsafe.=20=20If=20the=0A+=20= =20=20=20=20=20catalogs,=20toast=20table=20and=20toast=20index=20are=20= uncorrupted,=20and=20remain=20so=0A+=20=20=20=20=20=20during=20the=20= check=20of=20the=20target=20table,=20reconciling=20the=20target=20table=0A= +=20=20=20=20=20=20against=20its=20toast=20table=20should=20be=20safe.=0A= +=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= The=20following=20optional=20arguments=20are=20recognized:=0A+=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20on_error_stop=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=20If=20true,=20corruption=20checking=20stops=20at=20the=20= end=20of=20the=20first=20block=20on=0A+=20=20=20=20=20=20=20=20=20which=20= any=20corruptions=20are=20found.=0A+=20=20=20=20=20=20=20=20=0A+=20= =20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=20Defaults=20to=20= false.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20check_toast=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=20If=20this=20experimental=20option=20is=20true,=20toasted=20= values=20are=20checked=20gainst=0A+=20=20=20=20=20=20=20=20=20the=20= corresponding=20TOAST=20table.=0A+=20=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=20Defaults=20to=20= false.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20skip=0A+=20=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20=20If=20not=20none,=20corruption=20checking=20= skips=20blocks=20that=0A+=20=20=20=20=20=20=20=20=20are=20marked=20as=20= all-visible=20or=20all-frozen,=20as=20given.=0A+=20=20=20=20=20=20=20=20=20= Valid=20options=20are=20all-visible,=0A+=20=20=20=20=20= =20=20=20=20all-frozen=20and=20= none.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20Defaults=20to=20= none.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20startblock=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=20=20=20If=20specified,=20corruption=20checking=20begins=20at=20= the=20specified=20block,=0A+=20=20=20=20=20=20=20=20=20skipping=20all=20= previous=20blocks.=20=20It=20is=20an=20error=20to=20specify=20a=0A+=20=20= =20=20=20=20=20=20=20startblock=20outside=20the=20= range=20of=20blocks=20in=20the=0A+=20=20=20=20=20=20=20=20=20target=20= table.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20=20By=20default,=20does=20not=20skip=20= any=20blocks.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20endblock=0A+=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=20=20If=20specified,=20corruption=20checking=20ends=20at=20the=20= specified=20block,=0A+=20=20=20=20=20=20=20=20=20skipping=20all=20= remaining=20blocks.=20=20It=20is=20an=20error=20to=20specify=20an=0A+=20=20= =20=20=20=20=20=20=20endblock=20outside=20the=20range=20= of=20blocks=20in=20the=20target=0A+=20=20=20=20=20=20=20=20=20table.=0A+=20= =20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=20=20=20By=20default,=20does=20not=20skip=20any=20blocks.=0A+=20= =20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=20=20For=20each=20corruption=20detected,=20= verify_heapam=20returns=0A+=20=20=20=20=20=20a=20= row=20with=20the=20following=20columns:=0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20blkno=0A+=20=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20number=20of=20= the=20block=20containing=20the=20corrupt=20page.=0A+=20=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20offnum=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20OffsetNumber=20of=20= the=20corrupt=20tuple.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20attnum=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=20The=20attribute=20number=20of=20the=20corrupt=20column=20= in=20the=20tuple,=20if=20the=0A+=20=20=20=20=20=20=20=20=20corruption=20= is=20specific=20to=20a=20column=20and=20not=20the=20tuple=20as=20a=20= whole.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20msg=0A+=20=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= =20=20A=20human=20readable=20message=20describing=20the=20corruption=20= in=20the=20page.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=0A= +=20=20=0A=20=20=0A=20=0A=20=20=0A=20=20=20= Optional=20<parameter>heapallindexed</parameter>=20= Verification=0A=20=20=0A-=20=20When=20the=20= heapallindexed=20argument=20to=0A+=20=20When=20= the=20heapallindexed=20argument=20to=20B-Tree=0A=20= =20=20verification=20functions=20is=20true,=20an=20= additional=0A=20=20=20phase=20of=20verification=20is=20performed=20= against=20the=20table=20associated=20with=0A=20=20=20the=20target=20= index=20relation.=20=20This=20consists=20of=20a=20dummy=0A= diff=20--git=20a/src/backend/access/heap/hio.c=20= b/src/backend/access/heap/hio.c=0Aindex=20aa3f14c019..ca357410a2=20= 100644=0A---=20a/src/backend/access/heap/hio.c=0A+++=20= b/src/backend/access/heap/hio.c=0A@@=20-47,6=20+47,17=20@@=20= RelationPutHeapTuple(Relation=20relation,=0A=20=09=20*/=0A=20=09= Assert(!token=20||=20HeapTupleHeaderIsSpeculative(tuple->t_data));=0A=20=0A= +=09/*=0A+=09=20*=20Do=20not=20allow=20tuples=20with=20invalid=20= combinations=20of=20hint=20bits=20to=20be=20placed=0A+=09=20*=20on=20a=20= page.=20=20These=20combinations=20are=20detected=20as=20corruption=20by=20= the=0A+=09=20*=20contrib/amcheck=20logic,=20so=20if=20you=20disable=20= one=20or=20both=20of=20these=0A+=09=20*=20assertions,=20make=20= corresponding=20changes=20there.=0A+=09=20*/=0A+=09= Assert(!((tuple->t_data->t_infomask=20&=20HEAP_XMAX_LOCK_ONLY)=20&&=0A+=09= =09=09=20(tuple->t_data->t_infomask2=20&=20HEAP_KEYS_UPDATED)));=0A+=09= Assert(!((tuple->t_data->t_infomask=20&=20HEAP_XMAX_COMMITTED)=20&&=0A+=09= =09=09=20(tuple->t_data->t_infomask=20&=20HEAP_XMAX_IS_MULTI)));=0A+=0A=20= =09/*=20Add=20the=20tuple=20to=20the=20page=20*/=0A=20=09pageHeader=20=3D=20= BufferGetPage(buffer);=0A=20=0Adiff=20--git=20= a/src/backend/access/transam/multixact.c=20= b/src/backend/access/transam/multixact.c=0Aindex=20= a2ce617c8c..81752b68eb=20100644=0A---=20= a/src/backend/access/transam/multixact.c=0A+++=20= b/src/backend/access/transam/multixact.c=0A@@=20-735,6=20+735,25=20@@=20= ReadNextMultiXactId(void)=0A=20=09return=20mxid;=0A=20}=0A=20=0A+/*=0A+=20= *=20ReadMultiXactIdRange=0A+=20*=09=09Get=20the=20range=20of=20IDs=20= that=20may=20still=20be=20referenced=20by=20a=20relation.=0A+=20*/=0A= +void=0A+ReadMultiXactIdRange(MultiXactId=20*oldest,=20MultiXactId=20= *next)=0A+{=0A+=09LWLockAcquire(MultiXactGenLock,=20LW_SHARED);=0A+=09= *oldest=20=3D=20MultiXactState->oldestMultiXactId;=0A+=09*next=20=3D=20= MultiXactState->nextMXact;=0A+=09LWLockRelease(MultiXactGenLock);=0A+=0A= +=09if=20(*oldest=20<=20FirstMultiXactId)=0A+=09=09*oldest=20=3D=20= FirstMultiXactId;=0A+=09if=20(*next=20<=20FirstMultiXactId)=0A+=09=09= *next=20=3D=20FirstMultiXactId;=0A+}=0A+=0A+=0A=20/*=0A=20=20*=20= MultiXactIdCreateFromMembers=0A=20=20*=09=09Make=20a=20new=20MultiXactId=20= from=20the=20specified=20set=20of=20members=0Adiff=20--git=20= a/src/include/access/multixact.h=20b/src/include/access/multixact.h=0A= index=2058c42ffe1f..9a30380901=20100644=0A---=20= a/src/include/access/multixact.h=0A+++=20= b/src/include/access/multixact.h=0A@@=20-109,6=20+109,7=20@@=20extern=20= MultiXactId=20MultiXactIdCreateFromMembers(int=20nmembers,=0A=20=09=09=09= =09=09=09=09=09=09=09=09=09MultiXactMember=20*members);=0A=20=0A=20= extern=20MultiXactId=20ReadNextMultiXactId(void);=0A+extern=20void=20= ReadMultiXactIdRange(MultiXactId=20*oldest,=20MultiXactId=20*next);=0A=20= extern=20bool=20MultiXactIdIsRunning(MultiXactId=20multi,=20bool=20= isLockOnly);=0A=20extern=20void=20MultiXactIdSetOldestMember(void);=0A=20= extern=20int=09GetMultiXactIdMembers(MultiXactId=20multi,=20= MultiXactMember=20**xids,=0Adiff=20--git=20= a/src/tools/pgindent/typedefs.list=20b/src/tools/pgindent/typedefs.list=0A= index=204191f94869..bca30f3dde=20100644=0A---=20= a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-1020,6=20+1020,7=20@@=20= HbaToken=0A=20HeadlineJsonState=0A=20HeadlineParsedText=0A=20= HeadlineWordEntry=0A+HeapCheckContext=0A=20HeapScanDesc=0A=20HeapTuple=0A= =20HeapTupleData=0A@@=20-2287,6=20+2288,7=20@@=20SimpleStringList=0A=20= SimpleStringListCell=0A=20SingleBoundSortItem=0A=20Size=0A+SkipPages=0A=20= SlabBlock=0A=20SlabChunk=0A=20SlabContext=0A@@=20-2788,6=20+2790,8=20@@=20= XactCallback=0A=20XactCallbackItem=0A=20XactEvent=0A=20= XactLockTableWaitInfo=0A+XidBoundsViolation=0A+XidCommitStatus=0A=20= XidHorizonPrefetchState=0A=20XidStatus=0A=20XmlExpr=0A--=20=0A2.21.1=20= (Apple=20Git-122.3)=0A=0A= --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479 Content-Disposition: attachment; filename=v17-0002-Adding-contrib-module-pg_amcheck.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v17-0002-Adding-contrib-module-pg_amcheck.patch" Content-Transfer-Encoding: quoted-printable =46rom=202f4b08a77308c003db5a49020649d6fc61c2352e=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Mon,=205=20Oct=202020=2015:43:00=20-0700=0ASubject:=20[PATCH=20= v17=202/4]=20Adding=20contrib=20module=20pg_amcheck=0A=0AAdding=20new=20= contrib=20module=20pg_amcheck,=20which=20is=20a=20command=20line=0A= interface=20for=20running=20amcheck's=20verifications=20against=20tables=20= and=0Aindexes.=0A---=0A=20contrib/Makefile=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= contrib/pg_amcheck/.gitignore=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=20=202=20+=0A=20contrib/pg_amcheck/Makefile=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20|=20=20=2028=20+=0A=20= contrib/pg_amcheck/pg_amcheck.c=20=20=20=20=20=20=20=20=20=20=20=20|=20= 1281=20++++++++++++++++++++=0A=20contrib/pg_amcheck/pg_amcheck.control=20= =20=20=20=20=20|=20=20=20=205=20+=0A=20contrib/pg_amcheck/t/001_basic.pl=20= =20=20=20=20=20=20=20=20=20|=20=20=20=209=20+=0A=20= contrib/pg_amcheck/t/002_nonesuch.pl=20=20=20=20=20=20=20|=20=20=2060=20= +=0A=20contrib/pg_amcheck/t/003_check.pl=20=20=20=20=20=20=20=20=20=20|=20= =20231=20++++=0A=20contrib/pg_amcheck/t/004_verify_heapam.pl=20=20|=20=20= 489=20++++++++=0A=20contrib/pg_amcheck/t/005_opclass_damage.pl=20|=20=20=20= 52=20+=0A=20doc/src/sgml/contrib.sgml=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=20=20=201=20+=0A=20doc/src/sgml/filelist.sgml=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= doc/src/sgml/pgamcheck.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20228=20++++=0A=20src/tools/msvc/Mkvcbuild.pm=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20|=20=20=20=206=20+-=0A=20= src/tools/pgindent/typedefs.list=20=20=20=20=20=20=20=20=20=20=20|=20=20=20= =202=20+=0A=2015=20files=20changed,=202393=20insertions(+),=203=20= deletions(-)=0A=20create=20mode=20100644=20contrib/pg_amcheck/.gitignore=0A= =20create=20mode=20100644=20contrib/pg_amcheck/Makefile=0A=20create=20= mode=20100644=20contrib/pg_amcheck/pg_amcheck.c=0A=20create=20mode=20= 100644=20contrib/pg_amcheck/pg_amcheck.control=0A=20create=20mode=20= 100644=20contrib/pg_amcheck/t/001_basic.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/002_nonesuch.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/003_check.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/004_verify_heapam.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/005_opclass_damage.pl=0A=20create=20mode=20100644=20= doc/src/sgml/pgamcheck.sgml=0A=0Adiff=20--git=20a/contrib/Makefile=20= b/contrib/Makefile=0Aindex=207a4866e338..0fd4125902=20100644=0A---=20= a/contrib/Makefile=0A+++=20b/contrib/Makefile=0A@@=20-30,6=20+30,7=20@@=20= SUBDIRS=20=3D=20\=0A=20=09=09old_snapshot=09\=0A=20=09=09pageinspect=09\=0A= =20=09=09passwordcheck=09\=0A+=09=09pg_amcheck=09\=0A=20=09=09= pg_buffercache=09\=0A=20=09=09pg_freespacemap=20\=0A=20=09=09pg_prewarm=09= \=0Adiff=20--git=20a/contrib/pg_amcheck/.gitignore=20= b/contrib/pg_amcheck/.gitignore=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..f8eecf70bf=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/.gitignore=0A@@=20-0,0=20+1,2=20@@=0A+/pg_amcheck=0A= +/tmp_check/=0Adiff=20--git=20a/contrib/pg_amcheck/Makefile=20= b/contrib/pg_amcheck/Makefile=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..74554b9e8d=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/Makefile=0A@@=20-0,0=20+1,28=20@@=0A+#=20= contrib/pg_amcheck/Makefile=0A+=0A+PGFILEDESC=20=3D=20"pg_amcheck=20-=20= detects=20corruption=20within=20database=20relations"=0A+PGAPPICON=20=3D=20= win32=0A+=0A+PROGRAM=20=3D=20pg_amcheck=0A+OBJS=20=3D=20\=0A+=09= $(WIN32RES)=20\=0A+=09pg_amcheck.o=0A+=0A+REGRESS_OPTS=20+=3D=20= --load-extension=3Damcheck=20--load-extension=3Dpageinspect=0A= +EXTRA_INSTALL=20+=3D=20contrib/amcheck=20contrib/pageinspect=0A+=0A= +TAP_TESTS=20=3D=201=0A+=0A+PG_CPPFLAGS=20=3D=20-I$(libpq_srcdir)=0A= +PG_LIBS_INTERNAL=20=3D=20-L$(top_builddir)/src/fe_utils=20-lpgfeutils=20= $(libpq_pgport)=0A+=0A+ifdef=20USE_PGXS=0A+PG_CONFIG=20=3D=20pg_config=0A= +PGXS=20:=3D=20$(shell=20$(PG_CONFIG)=20--pgxs)=0A+include=20$(PGXS)=0A= +else=0A+subdir=20=3D=20contrib/pg_amcheck=0A+top_builddir=20=3D=20../..=0A= +include=20$(top_builddir)/src/Makefile.global=0A+include=20= $(top_srcdir)/contrib/contrib-global.mk=0A+endif=0Adiff=20--git=20= a/contrib/pg_amcheck/pg_amcheck.c=20b/contrib/pg_amcheck/pg_amcheck.c=0A= new=20file=20mode=20100644=0Aindex=200000000000..324cf1cfc8=0A---=20= /dev/null=0A+++=20b/contrib/pg_amcheck/pg_amcheck.c=0A@@=20-0,0=20= +1,1281=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20pg_amcheck.c=0A+=20*=09=09Detects=20corruption=20= within=20database=20relations.=0A+=20*=0A+=20*=20Copyright=20(c)=20= 2017-2020,=20PostgreSQL=20Global=20Development=20Group=0A+=20*=0A+=20*=20= IDENTIFICATION=0A+=20*=09=20=20contrib/pg_amcheck/pg_amcheck.c=0A+=20*=0A= +=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres_fe.h"=0A+=0A+#include=20= "catalog/pg_am.h"=0A+#include=20"catalog/pg_class.h"=0A+#include=20= "common/logging.h"=0A+#include=20"common/username.h"=0A+#include=20= "common/connect.h"=0A+#include=20"common/string.h"=0A+#include=20= "fe_utils/print.h"=0A+#include=20"fe_utils/simple_list.h"=0A+#include=20= "fe_utils/string_utils.h"=0A+#include=20"pg_getopt.h"=0A+=0A+const=20= char=20*usage_text[]=20=3D=20{=0A+=09"pg_amcheck=20is=20the=20PostgreSQL=20= command=20line=20frontend=20for=20the=20amcheck=20database=20corruption=20= checker.",=0A+=09"",=0A+=09"Usage:",=0A+=09"=20=20pg_amcheck=20= [OPTION]...=20[DBNAME=20[USERNAME]]",=0A+=09"",=0A+=09"General=20= options:",=0A+=09"=20=20-V,=20--version=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20output=20version=20information,=20then=20exit",=0A+=09"=20=20= -?,=20--help=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= show=20this=20help,=20then=20exit",=0A+=09"=20=20-s,=20--strict-names=20=20= =20=20=20=20=20=20=20=20=20require=20include=20patterns=20to=20match=20= at=20least=20one=20entity=20each",=0A+=09"=20=20-o,=20--on-error-stop=20=20= =20=20=20=20=20=20=20=20stop=20checking=20at=20end=20of=20first=20= corrupt=20page",=0A+=09"",=0A+=09"Schema=20checking=20options:",=0A+=09"=20= =20-n,=20--schema=3DPATTERN=20=20=20=20=20=20=20=20=20check=20relations=20= in=20the=20specified=20schema(s)=20only",=0A+=09"=20=20-N,=20= --exclude-schema=3DPATTERN=20do=20NOT=20check=20relations=20in=20the=20= specified=20schema(s)",=0A+=09"",=0A+=09"Table=20checking=20options:",=0A= +=09"=20=20-t,=20--table=3DPATTERN=20=20=20=20=20=20=20=20=20=20check=20= the=20specified=20table(s)=20only",=0A+=09"=20=20-T,=20= --exclude-table=3DPATTERN=20=20do=20NOT=20check=20the=20specified=20= table(s)",=0A+=09"=20=20-b,=20--startblock=20=20=20=20=20=20=20=20=20=20=20= =20=20begin=20checking=20table(s)=20at=20the=20given=20starting=20block=20= number",=0A+=09"=20=20-e,=20--endblock=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20check=20table(s)=20only=20up=20to=20the=20given=20ending=20= block=20number",=0A+=09"=20=20-f,=20--skip-all-frozen=20=20=20=20=20=20=20= =20do=20NOT=20check=20blocks=20marked=20as=20all=20frozen",=0A+=09"=20=20= -v,=20--skip-all-visible=20=20=20=20=20=20=20do=20NOT=20check=20blocks=20= marked=20as=20all=20visible",=0A+=09"",=0A+=09"TOAST=20table=20checking=20= options:",=0A+=09"=20=20-z,=20--check-toast=20=20=20=20=20=20=20=20=20=20= =20=20check=20associated=20toast=20tables=20and=20toast=20indexes",=0A+=09= "=20=20-Z,=20--skip-toast=20=20=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20= check=20associated=20toast=20tables=20and=20toast=20indexes",=0A+=09"=20=20= -B,=20--toast-startblock=20=20=20=20=20=20=20begin=20checking=20toast=20= table(s)=20at=20the=20given=20starting=20block",=0A+=09"=20=20-E,=20= --toast-endblock=20=20=20=20=20=20=20=20=20check=20toast=20table(s)=20= only=20up=20to=20the=20given=20ending=20block",=0A+=09"",=0A+=09"Index=20= checking=20options:",=0A+=09"=20=20-x,=20--check-indexes=20=20=20=20=20=20= =20=20=20=20check=20btree=20indexes=20associated=20with=20tables=20being=20= checked",=0A+=09"=20=20-X,=20--skip-indexes=20=20=20=20=20=20=20=20=20=20= =20do=20NOT=20check=20any=20btree=20indexes",=0A+=09"=20=20-i,=20= --index=3DPATTERN=20=20=20=20=20=20=20=20=20=20check=20the=20specified=20= index(es)=20only",=0A+=09"=20=20-I,=20--exclude-index=3DPATTERN=20=20do=20= NOT=20check=20the=20specified=20index(es)",=0A+=09"=20=20-c,=20= --check-corrupt=20=20=20=20=20=20=20=20=20=20check=20indexes=20even=20if=20= their=20associated=20table=20is=20corrupt",=0A+=09"=20=20-C,=20= --skip-corrupt=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20check=20= indexes=20if=20their=20associated=20table=20is=20corrupt",=0A+=09"=20=20= -a,=20--heapallindexed=20=20=20=20=20=20=20=20=20check=20index=20tuples=20= against=20the=20table=20tuples",=0A+=09"=20=20-A,=20--no-heapallindexed=20= =20=20=20=20=20do=20NOT=20check=20index=20tuples=20against=20the=20table=20= tuples",=0A+=09"=20=20-r,=20--rootdescend=20=20=20=20=20=20=20=20=20=20=20= =20search=20from=20the=20root=20page=20for=20each=20index=20tuple",=0A+=09= "=20=20-R,=20--no-rootdescend=20=20=20=20=20=20=20=20=20do=20NOT=20= search=20from=20the=20root=20page=20for=20each=20index=20tuple",=0A+=09= "",=0A+=09"Connection=20options:",=0A+=09"=20=20-d,=20--dbname=3DDBNAME=20= =20=20=20=20=20=20=20=20=20database=20name=20to=20connect=20to",=0A+=09"=20= =20-h,=20--host=3DHOSTNAME=20=20=20=20=20=20=20=20=20=20database=20= server=20host=20or=20socket=20directory",=0A+=09"=20=20-p,=20--port=3DPORT= =20=20=20=20=20=20=20=20=20=20=20=20=20=20database=20server=20port",=0A+=09= "=20=20-U,=20--username=3DUSERNAME=20=20=20=20=20=20database=20user=20= name",=0A+=09"=20=20-w,=20--no-password=20=20=20=20=20=20=20=20=20=20=20=20= never=20prompt=20for=20password",=0A+=09"=20=20-W,=20--password=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20force=20password=20prompt=20(should=20= happen=20automatically)",=0A+=09"",=0A+=09NULL=09=09=09=09=09=09/*=20= sentinel=20*/=0A+};=0A+=0A+typedef=20struct=0A+AmCheckSettings=0A+{=0A+=09= char=09=20=20=20*dbname;=0A+=09char=09=20=20=20*host;=0A+=09char=09=20=20= =20*port;=0A+=09char=09=20=20=20*username;=0A+}=20ConnectOptions;=0A+=0A= +typedef=20enum=20trivalue=0A+{=0A+=09TRI_DEFAULT,=0A+=09TRI_NO,=0A+=09= TRI_YES=0A+}=20trivalue;=0A+=0A+typedef=20struct=0A+{=0A+=09PGconn=09=20=20= =20*db;=09=09=09=09/*=20connection=20to=20backend=20*/=0A+=09bool=09=09= notty;=09=09=09/*=20stdin=20or=20stdout=20is=20not=20a=20tty=20(as=20= determined=0A+=09=09=09=09=09=09=09=09=20*=20on=20startup)=20*/=0A+=09= trivalue=09getPassword;=09/*=20prompt=20for=20a=20username=20and=20= password=20*/=0A+=09const=20char=20*progname;=09=09/*=20in=20case=20you=20= renamed=20pg_amcheck=20*/=0A+=09bool=09=09strict_names;=09/*=20The=20= specified=20names/patterns=20should=20to=0A+=09=09=09=09=09=09=09=09=20*=20= match=20at=20least=20one=20entity=20*/=0A+=09bool=09=09on_error_stop;=09= /*=20The=20checking=20of=20each=20table=20should=20stop=0A+=09=09=09=09=09= =09=09=09=20*=20after=20the=20first=20corrupt=20page=20is=20found.=20*/=0A= +=09bool=09=09skip_frozen;=09/*=20Do=20not=20check=20pages=20marked=20= all=20frozen=20*/=0A+=09bool=09=09skip_visible;=09/*=20Do=20not=20check=20= pages=20marked=20all=20visible=20*/=0A+=09bool=09=09check_indexes;=09/*=20= Check=20btree=20indexes=20*/=0A+=09bool=09=09check_toast;=09/*=20Check=20= associated=20toast=20tables=20and=20indexes=20*/=0A+=09bool=09=09= check_corrupt;=09/*=20Check=20indexes=20even=20if=20table=20is=20corrupt=20= */=0A+=09bool=09=09heapallindexed;=20/*=20Perform=20index=20to=20table=20= reconciling=20checks=20*/=0A+=09bool=09=09rootdescend;=09/*=20Perform=20= index=20rootdescend=20checks=20*/=0A+=09char=09=20=20=20*startblock;=09=09= /*=20Block=20number=20where=20checking=20begins=20*/=0A+=09char=09=20=20=20= *endblock;=09=09/*=20Block=20number=20where=20checking=20ends,=20= inclusive=20*/=0A+=09char=09=20=20=20*toaststart;=09=09/*=20Block=20= number=20where=20toast=20checking=20begins=20*/=0A+=09char=09=20=20=20= *toastend;=09=09/*=20Block=20number=20where=20toast=20checking=20ends,=0A= +=09=09=09=09=09=09=09=09=20*=20inclusive=20*/=0A+}=20AmCheckSettings;=0A= +=0A+static=20AmCheckSettings=20settings;=0A+=0A+/*=0A+=20*=20Object=20= inclusion/exclusion=20lists=0A+=20*=0A+=20*=20The=20string=20lists=20= record=20the=20patterns=20given=20by=20command-line=20switches,=0A+=20*=20= which=20we=20then=20convert=20to=20lists=20of=20Oids=20of=20matching=20= objects.=0A+=20*/=0A+static=20SimpleStringList=20schema_include_patterns=20= =3D=20{NULL,=20NULL};=0A+static=20SimpleOidList=20schema_include_oids=20= =3D=20{NULL,=20NULL};=0A+static=20SimpleStringList=20= schema_exclude_patterns=20=3D=20{NULL,=20NULL};=0A+static=20= SimpleOidList=20schema_exclude_oids=20=3D=20{NULL,=20NULL};=0A+=0A= +static=20SimpleStringList=20table_include_patterns=20=3D=20{NULL,=20= NULL};=0A+static=20SimpleOidList=20table_include_oids=20=3D=20{NULL,=20= NULL};=0A+static=20SimpleStringList=20table_exclude_patterns=20=3D=20= {NULL,=20NULL};=0A+static=20SimpleOidList=20table_exclude_oids=20=3D=20= {NULL,=20NULL};=0A+=0A+static=20SimpleStringList=20= index_include_patterns=20=3D=20{NULL,=20NULL};=0A+static=20SimpleOidList=20= index_include_oids=20=3D=20{NULL,=20NULL};=0A+static=20SimpleStringList=20= index_exclude_patterns=20=3D=20{NULL,=20NULL};=0A+static=20SimpleOidList=20= index_exclude_oids=20=3D=20{NULL,=20NULL};=0A+=0A+/*=0A+=20*=20List=20of=20= tables=20to=20be=20checked,=20compiled=20from=20above=20lists.=0A+=20*/=0A= +static=20SimpleOidList=20checklist=20=3D=20{NULL,=20NULL};=0A+=0A+/*=0A= +=20*=20Strings=20to=20be=20constructed=20once=20upon=20first=20use.=20=20= These=20could=20be=20made=0A+=20*=20string=20constants=20instead,=20but=20= that=20would=20require=20embedding=20knowledge=0A+=20*=20of=20the=20= single=20character=20values=20for=20each=20relkind,=20such=20as=20'm'=20= for=0A+=20*=20materialized=20views,=20which=20we'd=20rather=20not=20= embed=20here.=0A+=20*/=0A+static=20char=20*table_relkind_quals=20=3D=20= NULL;=0A+static=20char=20*index_relkind_quals=20=3D=20NULL;=0A+=0A+/*=0A= +=20*=20Functions=20to=20get=20pointers=20to=20the=20two=20strings,=20= above,=20after=20initializing=0A+=20*=20them=20upon=20the=20first=20call=20= to=20the=20function.=0A+=20*/=0A+static=20const=20char=20= *get_table_relkind_quals(void);=0A+static=20const=20char=20= *get_index_relkind_quals(void);=0A+=0A+/*=0A+=20*=20Functions=20for=20= running=20the=20various=20corruption=20checks.=0A+=20*/=0A+static=20void=20= check_tables(SimpleOidList=20*checklist);=0A+static=20uint64=20= check_toast(Oid=20tbloid);=0A+static=20uint64=20check_table(Oid=20= tbloid,=20const=20char=20*startblock,=0A+=09=09=09=09=09=09=20=20const=20= char=20*endblock,=20bool=20on_error_stop,=0A+=09=09=09=09=09=09=20=20= bool=20check_toast);=0A+static=20uint64=20check_indexes(Oid=20tbloid,=20= const=20SimpleOidList=20*include_oids,=0A+=09=09=09=09=09=09=09const=20= SimpleOidList=20*exclude_oids);=0A+static=20uint64=20check_index(const=20= char=20*idxoid,=20const=20char=20*idxname,=0A+=09=09=09=09=09=09=20=20= const=20char=20*tblname);=0A+=0A+/*=0A+=20*=20Functions=20implementing=20= standard=20command=20line=20behaviors.=0A+=20*/=0A+static=20void=20= parse_cli_options(int=20argc,=20char=20*argv[],=0A+=09=09=09=09=09=09=09=20= =20ConnectOptions=20*connOpts);=0A+static=20void=20usage(void);=0A= +static=20void=20showVersion(void);=0A+static=20void=20= NoticeProcessor(void=20*arg,=20const=20char=20*message);=0A+=0A+/*=0A+=20= *=20Functions=20for=20converting=20command=20line=20options=20that=20= include=20or=20exclude=0A+=20*=20schemas,=20tables,=20and=20indexes=20by=20= pattern=20into=20internally=20useful=20lists=20of=0A+=20*=20Oids=20for=20= objects=20that=20match=20those=20patterns.=0A+=20*/=0A+static=20void=20= expand_schema_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09=09= =09=09=09=09=09=09=09=09const=20SimpleOidList=20*exclude_oids,=0A+=09=09=09= =09=09=09=09=09=09=09SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09=09=09= =09bool=20strict_names);=0A+static=20void=20= expand_relkind_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09= =09=09=09=09=09=09=09=09=09=20const=20SimpleOidList=20*exclude_nsp_oids,=0A= +=09=09=09=09=09=09=09=09=09=09=20const=20SimpleOidList=20*exclude_oids,=0A= +=09=09=09=09=09=09=09=09=09=09=20SimpleOidList=20*oids,=0A+=09=09=09=09=09= =09=09=09=09=09=20bool=20strict_names,=0A+=09=09=09=09=09=09=09=09=09=09=20= const=20char=20*missing_errtext,=0A+=09=09=09=09=09=09=09=09=09=09=20= const=20char=20*relkind_quals);=0A+static=20void=20= expand_table_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09=09= =09=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20*exclude_nsp_oids,=0A= +=09=09=09=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20= *exclude_oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20SimpleOidList=20= *oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20bool=20strict_names);=0A= +static=20void=20expand_index_name_patterns(const=20SimpleStringList=20= *patterns,=0A+=09=09=09=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20= *exclude_nsp_oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20const=20= SimpleOidList=20*exclude_oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20= SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09=09=09=20=20=20bool=20= strict_names);=0A+static=20void=20get_table_check_list(const=20= SimpleOidList=20*include_nsp,=0A+=09=09=09=09=09=09=09=09=20const=20= SimpleOidList=20*exclude_nsp,=0A+=09=09=09=09=09=09=09=09=20const=20= SimpleOidList=20*include_tbl,=0A+=09=09=09=09=09=09=09=09=20const=20= SimpleOidList=20*exclude_tbl,=0A+=09=09=09=09=09=09=09=09=20= SimpleOidList=20*checklist);=0A+static=20PGresult=20= *ExecuteSqlQuery(const=20char=20*query,=20char=20**error);=0A+static=20= PGresult=20*ExecuteSqlQueryOrDie(const=20char=20*query);=0A+=0A+static=20= void=20append_csv_oids(PQExpBuffer=20querybuf,=20const=20SimpleOidList=20= *oids);=0A+static=20void=20apply_filter(PQExpBuffer=20querybuf,=20const=20= char=20*lval,=0A+=09=09=09=09=09=09=20const=20SimpleOidList=20*oids,=20= bool=20include);=0A+=0A+#define=20fatal(...)=20do=20{=20= pg_log_error(__VA_ARGS__);=20exit(1);=20}=20while(0)=0A+=0A+/*=20Like=20= fatal(),=20but=20with=20a=20complaint=20about=20a=20particular=20query.=20= */=0A+static=20void=0A+die_on_query_failure(const=20char=20*query)=0A+{=0A= +=09pg_log_error("query=20failed:=20%s",=0A+=09=09=09=09=20= PQerrorMessage(settings.db));=0A+=09fatal("query=20was:=20%s",=20query);=0A= +}=0A+=0A+#define=20EXIT_BADCONN=202=0A+=0A+int=0A+main(int=20argc,=20= char=20**argv)=0A+{=0A+=09ConnectOptions=20connOpts;=0A+=09bool=09=09= have_password=20=3D=20false;=0A+=09char=09=20=20=20*password=20=3D=20= NULL;=0A+=09bool=09=09new_pass;=0A+=0A+=09pg_logging_init(argv[0]);=0A+=09= set_pglocale_pgservice(argv[0],=20PG_TEXTDOMAIN("pg_amcheck"));=0A+=0A+=09= if=20(argc=20>=201)=0A+=09{=0A+=09=09if=20((strcmp(argv[1],=20"-?")=20=3D=3D= =200)=20||=0A+=09=09=09(argc=20=3D=3D=202=20&&=20(strcmp(argv[1],=20= "--help")=20=3D=3D=200)))=0A+=09=09{=0A+=09=09=09usage();=0A+=09=09=09= exit(EXIT_SUCCESS);=0A+=09=09}=0A+=09=09if=20(strcmp(argv[1],=20= "--version")=20=3D=3D=200=20||=20strcmp(argv[1],=20"-V")=20=3D=3D=200)=0A= +=09=09{=0A+=09=09=09showVersion();=0A+=09=09=09exit(EXIT_SUCCESS);=0A+=09= =09}=0A+=09}=0A+=0A+=09memset(&settings,=200,=20sizeof(settings));=0A+=09= settings.progname=20=3D=20get_progname(argv[0]);=0A+=0A+=09settings.db=20= =3D=20NULL;=0A+=09setDecimalLocale();=0A+=0A+=09settings.notty=20=3D=20= (!isatty(fileno(stdin))=20||=20!isatty(fileno(stdout)));=0A+=0A+=09= settings.getPassword=20=3D=20TRI_DEFAULT;=0A+=0A+=09/*=0A+=09=20*=20= Default=20behaviors=20for=20user=20settable=20options.=20=20Note=20that=20= these=20default=0A+=09=20*=20to=20doing=20all=20the=20safe=20checks=20= and=20none=20of=20the=20unsafe=20ones,=20on=20the=20theory=0A+=09=20*=20= that=20if=20a=20user=20says=20"pg_amcheck=20mydb"=20without=20specifying=20= any=20additional=0A+=09=20*=20options,=20we=20should=20check=20= everything=20we=20know=20how=20to=20check=20without=0A+=09=20*=20risking=20= any=20backend=20aborts.=0A+=09=20*/=0A+=0A+=09settings.on_error_stop=20=3D= =20false;=0A+=09settings.skip_frozen=20=3D=20false;=0A+=09= settings.skip_visible=20=3D=20false;=0A+=0A+=09/*=20Index=20checking=20= options=20*/=0A+=09settings.check_indexes=20=3D=20false;=0A+=09= settings.check_corrupt=20=3D=20false;=0A+=09settings.heapallindexed=20=3D=20= false;=0A+=09settings.rootdescend=20=3D=20false;=0A+=0A+=09/*=0A+=09=20*=20= Reconciling=20toasted=20attributes=20from=20the=20main=20table=20against=20= the=20toast=0A+=09=20*=20table=20can=20crash=20the=20backend=20if=20the=20= toast=20table=20or=20index=20are=20corrupt.=0A+=09=20*=20We=20can=20= optionally=20check=20the=20toast=20table=20and=20then=20the=20toast=20= index=20prior=0A+=09=20*=20to=20checking=20the=20main=20table,=20but=20= if=20the=20toast=20table=20or=20index=20are=0A+=09=20*=20concurrently=20= corrupted=20after=20we=20conclude=20they=20are=20valid,=20the=20check=20= of=0A+=09=20*=20the=20main=20table=20can=20crash=20the=20backend.=20=20= The=20oneous=20is=20on=20any=20caller=20who=0A+=09=20*=20enables=20this=20= option=20to=20make=20certain=20the=20environment=20is=20sufficiently=0A+=09= =20*=20stable=20that=20concurrent=20corruptions=20of=20the=20toast=20is=20= not=20possible.=0A+=09=20*/=0A+=09settings.check_toast=20=3D=20false;=0A= +=0A+=09parse_cli_options(argc,=20argv,=20&connOpts);=0A+=0A+=09if=20= (settings.getPassword=20=3D=3D=20TRI_YES)=0A+=09{=0A+=09=09/*=0A+=09=09=20= *=20We=20can't=20be=20sure=20yet=20of=20the=20username=20that=20will=20= be=20used,=20so=20don't=0A+=09=09=20*=20offer=20a=20potentially=20wrong=20= one.=20=20Typical=20uses=20of=20this=20option=20are=0A+=09=09=20*=20= noninteractive=20anyway.=0A+=09=09=20*/=0A+=09=09password=20=3D=20= simple_prompt("Password:=20",=20false);=0A+=09=09have_password=20=3D=20= true;=0A+=09}=0A+=0A+=09/*=20loop=20until=20we=20have=20a=20password=20= if=20requested=20by=20backend=20*/=0A+=09do=0A+=09{=0A+#define=20= ARRAY_SIZE=098=0A+=09=09const=20char=20**keywords=20=3D=20= pg_malloc(ARRAY_SIZE=20*=20sizeof(*keywords));=0A+=09=09const=20char=20= **values=20=3D=20pg_malloc(ARRAY_SIZE=20*=20sizeof(*values));=0A+=0A+=09=09= keywords[0]=20=3D=20"host";=0A+=09=09values[0]=20=3D=20connOpts.host;=0A= +=09=09keywords[1]=20=3D=20"port";=0A+=09=09values[1]=20=3D=20= connOpts.port;=0A+=09=09keywords[2]=20=3D=20"user";=0A+=09=09values[2]=20= =3D=20connOpts.username;=0A+=09=09keywords[3]=20=3D=20"password";=0A+=09=09= values[3]=20=3D=20have_password=20?=20password=20:=20NULL;=0A+=09=09= keywords[4]=20=3D=20"dbname";=20/*=20see=20do_connect()=20*/=0A+=09=09if=20= (connOpts.dbname=20=3D=3D=20NULL)=0A+=09=09{=0A+=09=09=09if=20= (getenv("PGDATABASE"))=0A+=09=09=09=09values[4]=20=3D=20= getenv("PGDATABASE");=0A+=09=09=09else=20if=20(getenv("PGUSER"))=0A+=09=09= =09=09values[4]=20=3D=20getenv("PGUSER");=0A+=09=09=09else=0A+=09=09=09=09= values[4]=20=3D=20"postgres";=0A+=09=09}=0A+=09=09else=0A+=09=09=09= values[4]=20=3D=20connOpts.dbname;=0A+=09=09keywords[5]=20=3D=20= "fallback_application_name";=0A+=09=09values[5]=20=3D=20= settings.progname;=0A+=09=09keywords[6]=20=3D=20"client_encoding";=0A+=09= =09values[6]=20=3D=20(settings.notty=20||=0A+=09=09=09=09=09=20= getenv("PGCLIENTENCODING"))=20?=20NULL=20:=20"auto";=0A+=09=09= keywords[7]=20=3D=20NULL;=0A+=09=09values[7]=20=3D=20NULL;=0A+=0A+=09=09= new_pass=20=3D=20false;=0A+=09=09settings.db=20=3D=20= PQconnectdbParams(keywords,=20values,=20true);=0A+=09=09if=20= (settings.db=20=3D=3D=20NULL)=0A+=09=09{=0A+=09=09=09pg_log_error("no=20= connection=20to=20server=20after=20initial=20attempt");=0A+=09=09=09= exit(EXIT_BADCONN);=0A+=09=09}=0A+=0A+=09=09free(keywords);=0A+=09=09= free(values);=0A+=0A+=09=09if=20(PQstatus(settings.db)=20=3D=3D=20= CONNECTION_BAD=20&&=0A+=09=09=09PQconnectionNeedsPassword(settings.db)=20= &&=0A+=09=09=09!have_password=20&&=0A+=09=09=09settings.getPassword=20!=3D= =20TRI_NO)=0A+=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20Before=20closing=20= the=20old=20PGconn,=20extract=20the=20user=20name=20that=20was=0A+=09=09=09= =20*=20actually=20connected=20with.=0A+=09=09=09=20*/=0A+=09=09=09const=20= char=20*realusername=20=3D=20PQuser(settings.db);=0A+=09=09=09char=09=20=20= =20*password_prompt;=0A+=0A+=09=09=09if=20(realusername=20&&=20= realusername[0])=0A+=09=09=09=09password_prompt=20=3D=20= psprintf(_("Password=20for=20user=20%s:=20"),=0A+=09=09=09=09=09=09=09=09= =09=09=20=20=20realusername);=0A+=09=09=09else=0A+=09=09=09=09= password_prompt=20=3D=20pg_strdup(_("Password:=20"));=0A+=09=09=09= PQfinish(settings.db);=0A+=0A+=09=09=09password=20=3D=20= simple_prompt(password_prompt,=20false);=0A+=09=09=09= free(password_prompt);=0A+=09=09=09have_password=20=3D=20true;=0A+=09=09=09= new_pass=20=3D=20true;=0A+=09=09}=0A+=09}=20while=20(new_pass);=0A+=0A+=09= if=20(!settings.db)=0A+=09{=0A+=09=09pg_log_error("no=20connection=20to=20= server");=0A+=09=09exit(EXIT_BADCONN);=0A+=09}=0A+=0A+=09if=20= (PQstatus(settings.db)=20=3D=3D=20CONNECTION_BAD)=0A+=09{=0A+=09=09= pg_log_error("could=20not=20connect=20to=20server:=20%s",=0A+=09=09=09=09= =09=20PQerrorMessage(settings.db));=0A+=09=09PQfinish(settings.db);=0A+=09= =09exit(EXIT_BADCONN);=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20Expand=20= schema,=20table,=20and=20index=20exclusion=20patterns,=20if=20any.=20=20= Note=20that=0A+=09=20*=20non-matching=20exclusion=20patterns=20are=20not=20= an=20error,=20even=20when=0A+=09=20*=20--strict-names=20was=20specified.=0A= +=09=20*/=0A+=09expand_schema_name_patterns(&schema_exclude_patterns,=20= NULL,=0A+=09=09=09=09=09=09=09=09&schema_exclude_oids,=20false);=0A+=09= expand_table_name_patterns(&table_exclude_patterns,=20NULL,=20NULL,=0A+=09= =09=09=09=09=09=09=20=20=20&table_exclude_oids,=20false);=0A+=09= expand_index_name_patterns(&index_exclude_patterns,=20NULL,=20NULL,=0A+=09= =09=09=09=09=09=09=20=20=20&index_exclude_oids,=20false);=0A+=0A+=09/*=20= Expand=20schema=20selection=20patterns=20into=20Oid=20lists=20*/=0A+=09= if=20(schema_include_patterns.head=20!=3D=20NULL)=0A+=09{=0A+=09=09= expand_schema_name_patterns(&schema_include_patterns,=0A+=09=09=09=09=09=09= =09=09=09&schema_exclude_oids,=0A+=09=09=09=09=09=09=09=09=09= &schema_include_oids,=0A+=09=09=09=09=09=09=09=09=09= settings.strict_names);=0A+=09=09if=20(schema_include_oids.head=20=3D=3D=20= NULL)=0A+=09=09=09fatal("no=20matching=20schemas=20were=20found");=0A+=09= }=0A+=0A+=09/*=20Expand=20table=20selection=20patterns=20into=20Oid=20= lists=20*/=0A+=09if=20(table_include_patterns.head=20!=3D=20NULL)=0A+=09= {=0A+=09=09expand_table_name_patterns(&table_include_patterns,=0A+=09=09=09= =09=09=09=09=09=20=20=20&schema_exclude_oids,=0A+=09=09=09=09=09=09=09=09= =20=20=20&table_exclude_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= &table_include_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= settings.strict_names);=0A+=09=09if=20(table_include_oids.head=20=3D=3D=20= NULL)=0A+=09=09=09fatal("no=20matching=20tables=20were=20found");=0A+=09= }=0A+=0A+=09/*=20Expand=20index=20selection=20patterns=20into=20Oid=20= lists=20*/=0A+=09if=20(index_include_patterns.head=20!=3D=20NULL)=0A+=09= {=0A+=09=09expand_index_name_patterns(&index_include_patterns,=0A+=09=09=09= =09=09=09=09=09=20=20=20&schema_exclude_oids,=0A+=09=09=09=09=09=09=09=09= =20=20=20&index_exclude_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= &index_include_oids,=0A+=09=09=09=09=09=09=09=09=20=20=20= settings.strict_names);=0A+=09=09if=20(index_include_oids.head=20=3D=3D=20= NULL)=0A+=09=09=09fatal("no=20matching=20indexes=20were=20found");=0A+=09= }=0A+=0A+=09/*=0A+=09=20*=20Compile=20list=20of=20all=20tables=20to=20be=20= checked=20based=20on=20namespace=20and=20table=0A+=09=20*=20includes=20= and=20excludes.=0A+=09=20*/=0A+=09= get_table_check_list(&schema_include_oids,=20&schema_exclude_oids,=0A+=09= =09=09=09=09=09=20&table_include_oids,=20&table_exclude_oids,=20= &checklist);=0A+=0A+=09PQsetNoticeProcessor(settings.db,=20= NoticeProcessor,=20NULL);=0A+=0A+=09/*=0A+=09=20*=20All=20information=20= about=20corrupt=20indexes=20are=20returned=20via=20ereport,=20not=20as=0A= +=09=20*=20tuples.=20=20We=20want=20all=20the=20details=20to=20report=20= if=20corruption=20exists.=0A+=09=20*/=0A+=09= PQsetErrorVerbosity(settings.db,=20PQERRORS_VERBOSE);=0A+=0A+=09= check_tables(&checklist);=0A+=0A+=09return=200;=0A+}=0A+=0A+/*=0A+=20*=20= Conditionally=20add=20a=20restriction=20to=20a=20query=20such=20that=20= lval=20must=20be=20an=20Oid=20in=0A+=20*=20the=20given=20list=20of=20= Oids,=20except=20that=20for=20a=20null=20or=20empty=20oids=20list=20= argument,=0A+=20*=20no=20filtering=20is=20done=20and=20we=20return=20= without=20having=20modified=20the=20query=20buffer.=0A+=20*=0A+=20*=20= The=20query=20argument=20must=20already=20have=20begun=20the=20WHERE=20= clause=20and=20must=20be=20in=20a=0A+=20*=20state=20where=20we=20can=20= append=20an=20AND=20clause.=20=20No=20checking=20of=20this=20requirement=20= is=0A+=20*=20done=20here.=0A+=20*=0A+=20*=20On=20return,=20the=20query=20= buffer=20will=20be=20extended=20with=20an=20AND=20clause=20that=20= filters=0A+=20*=20only=20those=20rows=20where=20the=20lval=20is=20an=20= Oid=20present=20in=20the=20given=20list=20of=20oids.=0A+=20*/=0A+static=20= inline=20void=0A+include_filter(PQExpBuffer=20querybuf,=20const=20char=20= *lval,=20const=20SimpleOidList=20*oids)=0A+{=0A+=09= apply_filter(querybuf,=20lval,=20oids,=20true);=0A+}=0A+=0A+/*=0A+=20*=20= Same=20as=20include_filter,=20above,=20except=20that=20for=20a=20= non-null,=20non-empty=20oids=0A+=20*=20list,=20the=20lval=20is=20= restricted=20to=20not=20be=20any=20of=20the=20values=20in=20the=20list.=0A= +=20*/=0A+static=20inline=20void=0A+exclude_filter(PQExpBuffer=20= querybuf,=20const=20char=20*lval,=20const=20SimpleOidList=20*oids)=0A+{=0A= +=09apply_filter(querybuf,=20lval,=20oids,=20false);=0A+}=0A+=0A+/*=0A+=20= *=20Check=20each=20table=20from=20the=20given=20checklist=20per=20the=20= user=20specified=20options.=0A+=20*/=0A+static=20void=0A= +check_tables(SimpleOidList=20*checklist)=0A+{=0A+=09const=20= SimpleOidListCell=20*cell;=0A+=0A+=09for=20(cell=20=3D=20= checklist->head;=20cell;=20cell=20=3D=20cell->next)=0A+=09{=0A+=09=09= uint64=09=09corruptions=20=3D=200;=0A+=09=09bool=09=09reconcile_toast;=0A= +=0A+=09=09/*=0A+=09=09=20*=20If=20we=20skip=20checking=20the=20toast=20= table,=20or=20if=20during=20the=20check=20we=0A+=09=09=20*=20detect=20= any=20toast=20table=20corruption,=20the=20main=20table=20checks=20below=20= must=0A+=09=09=20*=20not=20reconcile=20toasted=20attributes=20against=20= the=20toast=20table,=20as=20such=0A+=09=09=20*=20accesses=20to=20the=20= toast=20table=20might=20crash=20the=20backend.=20=20Instead,=20skip=0A+=09= =09=20*=20such=20reconciliations=20for=20this=20table.=0A+=09=09=20*=0A+=09= =09=20*=20This=20protection=20contains=20a=20race=20condition;=20the=20= toast=20table=20or=20index=0A+=09=09=20*=20could=20become=20corrupted=20= concurrently=20with=20our=20checks,=20but=20prevention=0A+=09=09=20*=20= of=20such=20concurrent=20corruption=20is=20documented=20as=20the=20= caller's=0A+=09=09=20*=20reponsibility,=20so=20we=20don't=20worry=20= about=20it=20here.=0A+=09=09=20*/=0A+=09=09reconcile_toast=20=3D=20= false;=0A+=09=09if=20(settings.check_toast)=0A+=09=09{=0A+=09=09=09if=20= (check_toast(cell->val)=20=3D=3D=200)=0A+=09=09=09=09reconcile_toast=20=3D= =20true;=0A+=09=09}=0A+=0A+=09=09corruptions=20=3D=20= check_table(cell->val,=0A+=09=09=09=09=09=09=09=09=20=20= settings.startblock,=0A+=09=09=09=09=09=09=09=09=20=20settings.endblock,=0A= +=09=09=09=09=09=09=09=09=20=20settings.on_error_stop,=0A+=09=09=09=09=09= =09=09=09=20=20reconcile_toast);=0A+=0A+=09=09if=20= (settings.check_indexes)=0A+=09=09{=0A+=09=09=09bool=09=09= old_heapallindexed;=0A+=0A+=09=09=09/*=20Optionally=20skip=20the=20index=20= checks=20for=20a=20corrupt=20table.=20*/=0A+=09=09=09if=20(corruptions=20= &&=20!settings.check_corrupt)=0A+=09=09=09=09continue;=0A+=0A+=09=09=09= /*=0A+=09=09=09=20*=20The=20btree=20checking=20logic=20which=20= optionally=20checks=20the=20contents=0A+=09=09=09=20*=20of=20an=20index=20= against=20the=20corresponding=20table=20has=20not=20yet=20been=0A+=09=09=09= =20*=20sufficiently=20hardened=20against=20corrupt=20tables.=20=20In=20= particular,=0A+=09=09=09=20*=20when=20called=20with=20heapallindexed=20= true,=20it=20segfaults=20if=20the=20file=0A+=09=09=09=20*=20backing=20= the=20table=20relation=20has=20been=20erroneously=20unlinked.=20=20In=0A= +=09=09=09=20*=20any=20event,=20it=20seems=20unwise=20to=20reconcile=20= an=20index=20against=20its=0A+=09=09=09=20*=20table=20when=20we=20= already=20know=20the=20table=20is=20corrupt.=0A+=09=09=09=20*/=0A+=09=09=09= old_heapallindexed=20=3D=20settings.heapallindexed;=0A+=09=09=09if=20= (corruptions)=0A+=09=09=09=09settings.heapallindexed=20=3D=20false;=0A+=0A= +=09=09=09corruptions=20+=3D=20check_indexes(cell->val,=0A+=09=09=09=09=09= =09=09=09=09=09=20&index_include_oids,=0A+=09=09=09=09=09=09=09=09=09=09=20= &index_exclude_oids);=0A+=0A+=09=09=09settings.heapallindexed=20=3D=20= old_heapallindexed;=0A+=09=09}=0A+=09}=0A+}=0A+=0A+/*=0A+=20*=20For=20a=20= given=20main=20table=20relation,=20returns=20the=20associated=20toast=20= table,=0A+=20*=20or=20InvalidOid=20if=20none=20exists.=0A+=20*/=0A= +static=20Oid=0A+get_toast_oid(Oid=20tbloid)=0A+{=0A+=09PQExpBuffer=20= querybuf=20=3D=20createPQExpBuffer();=0A+=09PGresult=20=20=20*res;=0A+=09= char=09=20=20=20*error=20=3D=20NULL;=0A+=09Oid=09=09=09result=20=3D=20= InvalidOid;=0A+=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09=20=20= "SELECT=20c.reltoastrelid"=0A+=09=09=09=09=09=20=20"\nFROM=20= pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=20=20"\nWHERE=20c.oid=20=3D=20= %u",=0A+=09=09=09=09=09=20=20tbloid);=0A+=09res=20=3D=20= ExecuteSqlQuery(querybuf->data,=20&error);=0A+=09if=20= (PQresultStatus(res)=20=3D=3D=20PGRES_TUPLES_OK=20&&=20PQntuples(res)=20= >=200)=0A+=09=09result=20=3D=20atooid(PQgetvalue(res,=200,=200));=0A+=09= else=20if=20(error)=0A+=09=09die_on_query_failure(querybuf->data);=0A+=0A= +=09PQclear(res);=0A+=09destroyPQExpBuffer(querybuf);=0A+=0A+=09return=20= result;=0A+}=0A+=0A+/*=0A+=20*=20For=20the=20given=20main=20table=20= relation,=20checks=20the=20associated=20toast=20table=20and=0A+=20*=20= index,=20in=20any.=20=20This=20should=20be=20performed=20*before*=20= checking=20the=20main=20table=0A+=20*=20relation,=20as=20the=20checks=20= inside=20verify_heapam=20assume=20both=20the=20toast=20table=20and=0A+=20= *=20toast=20index=20are=20usable.=0A+=20*=0A+=20*=20Returns=20the=20= number=20of=20corruptions=20detected.=0A+=20*/=0A+static=20uint64=0A= +check_toast(Oid=20tbloid)=0A+{=0A+=09Oid=09=09=09toastoid;=0A+=09uint64=09= =09corruption_cnt=20=3D=200;=0A+=0A+=09if=20(settings.db=20=3D=3D=20= NULL)=0A+=09=09fatal("no=20connection=20on=20entry=20to=20check_toast");=0A= +=0A+=09toastoid=20=3D=20get_toast_oid(tbloid);=0A+=09if=20= (OidIsValid(toastoid))=0A+=09{=0A+=09=09corruption_cnt=20=3D=20= check_table(toastoid,=20settings.toaststart,=0A+=09=09=09=09=09=09=09=09=09= =20settings.toastend,=20settings.on_error_stop,=0A+=09=09=09=09=09=09=09=09= =09=20false);=0A+=0A+=09=09/*=0A+=09=09=20*=20If=20the=20toast=20table=20= is=20corrupt,=20checking=20the=20index=20is=20not=20safe.=0A+=09=09=20*=20= There=20is=20a=20race=20condition=20here,=20as=20the=20toast=20table=20= could=20be=0A+=09=09=20*=20concurrently=20corrupted,=20but=20preventing=20= concurrent=20corruption=20is=20the=0A+=09=09=20*=20caller's=20= responsibility,=20not=20ours.=0A+=09=09=20*/=0A+=09=09if=20= (corruption_cnt=20=3D=3D=200)=0A+=09=09=09corruption_cnt=20+=3D=20= check_indexes(toastoid,=20NULL,=20NULL);=0A+=09}=0A+=0A+=09return=20= corruption_cnt;=0A+}=0A+=0A+/*=0A+=20*=20Checks=20the=20given=20table=20= for=20corruption,=20returning=20the=20number=20of=20corruptions=0A+=20*=20= detected=20and=20printed=20to=20the=20user.=0A+=20*/=0A+static=20uint64=0A= +check_table(Oid=20tbloid,=20const=20char=20*startblock,=20const=20char=20= *endblock,=0A+=09=09=09bool=20on_error_stop,=20bool=20check_toast)=0A+{=0A= +=09PQExpBuffer=20querybuf;=0A+=09PGresult=20=20=20*res;=0A+=09int=09=09=09= i;=0A+=09char=09=20=20=20*skip;=0A+=09char=09=20=20=20*toast;=0A+=09= const=20char=20*stop;=0A+=09char=09=20=20=20*error=20=3D=20NULL;=0A+=09= uint64=09=09corruption_cnt=20=3D=200;=0A+=0A+=09if=20(settings.db=20=3D=3D= =20NULL)=0A+=09=09fatal("no=20connection=20on=20entry=20to=20= check_table");=0A+=0A+=09if=20(startblock=20=3D=3D=20NULL)=0A+=09=09= startblock=20=3D=20"NULL";=0A+=09if=20(endblock=20=3D=3D=20NULL)=0A+=09=09= endblock=20=3D=20"NULL";=0A+=09if=20(settings.skip_frozen)=0A+=09=09skip=20= =3D=20pg_strdup("'all=20frozen'");=0A+=09else=20if=20= (settings.skip_visible)=0A+=09=09skip=20=3D=20pg_strdup("'all=20= visible'");=0A+=09else=0A+=09=09skip=20=3D=20pg_strdup("'none'");=0A+=09= stop=20=3D=20(on_error_stop)=20?=20"true"=20:=20"false";=0A+=09toast=20=3D= =20(check_toast)=20?=20"true"=20:=20"false";=0A+=0A+=09querybuf=20=3D=20= createPQExpBuffer();=0A+=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09= =09=20=20"SELECT=20c.relname,=20v.blkno,=20v.offnum,=20v.attnum,=20v.msg=20= "=0A+=09=09=09=09=09=20=20"FROM=20verify_heapam("=0A+=09=09=09=09=09=20=20= "relation=20:=3D=20%u,=20"=0A+=09=09=09=09=09=20=20"on_error_stop=20:=3D=20= %s,=20"=0A+=09=09=09=09=09=20=20"skip=20:=3D=20%s,=20"=0A+=09=09=09=09=09= =20=20"check_toast=20:=3D=20%s,=20"=0A+=09=09=09=09=09=20=20"startblock=20= :=3D=20%s,=20"=0A+=09=09=09=09=09=20=20"endblock=20:=3D=20%s)=20v,=20"=0A= +=09=09=09=09=09=20=20"pg_catalog.pg_class=20c=20"=0A+=09=09=09=09=09=20=20= "WHERE=20c.oid=20=3D=20%u",=0A+=09=09=09=09=09=20=20tbloid,=20stop,=20= skip,=20toast,=20startblock,=20endblock,=20tbloid);=0A+=0A+=09res=20=3D=20= ExecuteSqlQuery(querybuf->data,=20&error);=0A+=09if=20= (PQresultStatus(res)=20=3D=3D=20PGRES_TUPLES_OK=20&&=20PQntuples(res)=20= >=200)=0A+=09{=0A+=09=09corruption_cnt=20+=3D=20PQntuples(res);=0A+=09=09= for=20(i=20=3D=200;=20i=20<=20PQntuples(res);=20i++)=0A+=09=09{=0A+=09=09= =09printf("(relname=3D%s,blkno=3D%s,offnum=3D%s,attnum=3D%s)\n%s\n",=0A+=09= =09=09=09=20=20=20PQgetvalue(res,=20i,=200),=09/*=20relname=20*/=0A+=09=09= =09=09=20=20=20PQgetvalue(res,=20i,=201),=09/*=20blkno=20*/=0A+=09=09=09=09= =20=20=20PQgetvalue(res,=20i,=202),=09/*=20offnum=20*/=0A+=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=203),=09/*=20attnum=20*/=0A+=09=09=09=09=20=20= =20PQgetvalue(res,=20i,=204));=09/*=20msg=20*/=0A+=09=09}=0A+=09}=0A+=09= else=20if=20(error)=0A+=09{=0A+=09=09corruption_cnt++;=0A+=09=09= printf("%s\n",=20error);=0A+=09=09pfree(error);=0A+=09}=0A+=0A+=09= PQclear(res);=0A+=09destroyPQExpBuffer(querybuf);=0A+=09return=20= corruption_cnt;=0A+}=0A+=0A+static=20uint64=0A+check_indexes(Oid=20= tbloid,=20const=20SimpleOidList=20*include_oids,=0A+=09=09=09=20=20const=20= SimpleOidList=20*exclude_oids)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09= PGresult=20=20=20*res;=0A+=09int=09=09=09i;=0A+=09char=09=20=20=20*error=20= =3D=20NULL;=0A+=09uint64=09=09corruption_cnt=20=3D=200;=0A+=0A+=09if=20= (settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20= entry=20to=20check_indexes");=0A+=0A+=09querybuf=20=3D=20= createPQExpBuffer();=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09= =20=20"SELECT=20i.indexrelid,=20ci.relname,=20ct.relname"=0A+=09=09=09=09= =09=20=20"\nFROM=20pg_catalog.pg_index=20i,=20pg_catalog.pg_class=20ci,=20= "=0A+=09=09=09=09=09=20=20"pg_catalog.pg_class=20ct"=0A+=09=09=09=09=09=20= =20"\nWHERE=20i.indexrelid=20=3D=20ci.oid"=0A+=09=09=09=09=09=20=20"\n=20= =20=20=20AND=20i.indrelid=20=3D=20ct.oid"=0A+=09=09=09=09=09=20=20"\n=20=20= =20=20AND=20ci.relam=20=3D=20%u"=0A+=09=09=09=09=09=20=20"\n=20=20=20=20= AND=20i.indrelid=20=3D=20%u",=0A+=09=09=09=09=09=20=20BTREE_AM_OID,=20= tbloid);=0A+=09include_filter(querybuf,=20"i.indexrelid",=20= include_oids);=0A+=09exclude_filter(querybuf,=20"i.indexrelid",=20= exclude_oids);=0A+=0A+=09res=20=3D=20ExecuteSqlQuery(querybuf->data,=20= &error);=0A+=09if=20(PQresultStatus(res)=20=3D=3D=20PGRES_TUPLES_OK)=0A+=09= {=0A+=09=09for=20(i=20=3D=200;=20i=20<=20PQntuples(res);=20i++)=0A+=09=09= =09corruption_cnt=20+=3D=20check_index(PQgetvalue(res,=20i,=200),=0A+=09=09= =09=09=09=09=09=09=09=09=20=20PQgetvalue(res,=20i,=201),=0A+=09=09=09=09=09= =09=09=09=09=09=20=20PQgetvalue(res,=20i,=202));=0A+=09}=0A+=09else=20if=20= (error)=0A+=09{=0A+=09=09corruption_cnt++;=0A+=09=09printf("%s\n",=20= error);=0A+=09=09pfree(error);=0A+=09}=0A+=0A+=09PQclear(res);=0A+=09= destroyPQExpBuffer(querybuf);=0A+=0A+=09return=20corruption_cnt;=0A+}=0A= +=0A+static=20uint64=0A+check_index(const=20char=20*idxoid,=20const=20= char=20*idxname,=20const=20char=20*tblname)=0A+{=0A+=09PQExpBuffer=20= querybuf;=0A+=09PGresult=20=20=20*res;=0A+=09uint64=09=09corruption_cnt=20= =3D=200;=0A+=0A+=09if=20(settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20= connection=20on=20entry=20to=20check_index");=0A+=09if=20(idxname=20=3D=3D= =20NULL)=0A+=09=09fatal("no=20index=20name=20on=20entry=20to=20= check_index");=0A+=09if=20(tblname=20=3D=3D=20NULL)=0A+=09=09fatal("no=20= table=20name=20on=20entry=20to=20check_index");=0A+=0A+=09querybuf=20=3D=20= createPQExpBuffer();=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09= =20=20"SELECT=20bt_index_parent_check('%s'::regclass,=20%s,=20%s)",=0A+=09= =09=09=09=09=20=20idxoid,=0A+=09=09=09=09=09=20=20= settings.heapallindexed=20?=20"true"=20:=20"false",=0A+=09=09=09=09=09=20= =20settings.rootdescend=20?=20"true"=20:=20"false");=0A+=09res=20=3D=20= PQexec(settings.db,=20querybuf->data);=0A+=09if=20(PQresultStatus(res)=20= !=3D=20PGRES_TUPLES_OK)=0A+=09{=0A+=09=09corruption_cnt++;=0A+=09=09= printf("index=20check=20failed=20for=20index=20%s=20of=20table=20%s:\n",=0A= +=09=09=09=20=20=20idxname,=20tblname);=0A+=09=09printf("%s",=20= PQerrorMessage(settings.db));=0A+=09}=0A+=0A+=09PQclear(res);=0A+=09= destroyPQExpBuffer(querybuf);=0A+=0A+=09return=20corruption_cnt;=0A+}=0A= +=0A+static=20void=0A+parse_cli_options(int=20argc,=20char=20*argv[],=20= ConnectOptions=20*connOpts)=0A+{=0A+=09static=20struct=20option=20= long_options[]=20=3D=0A+=09{=0A+=09=09{"check-corrupt",=20no_argument,=20= NULL,=20'c'},=0A+=09=09{"check-indexes",=20no_argument,=20NULL,=20'x'},=0A= +=09=09{"check-toast",=20no_argument,=20NULL,=20'z'},=0A+=09=09= {"dbname",=20required_argument,=20NULL,=20'd'},=0A+=09=09{"endblock",=20= required_argument,=20NULL,=20'e'},=0A+=09=09{"exclude-index",=20= required_argument,=20NULL,=20'I'},=0A+=09=09{"exclude-schema",=20= required_argument,=20NULL,=20'N'},=0A+=09=09{"exclude-table",=20= required_argument,=20NULL,=20'T'},=0A+=09=09{"heapallindexed",=20= no_argument,=20NULL,=20'a'},=0A+=09=09{"help",=20optional_argument,=20= NULL,=20'?'},=0A+=09=09{"host",=20required_argument,=20NULL,=20'h'},=0A+=09= =09{"index",=20required_argument,=20NULL,=20'i'},=0A+=09=09= {"no-heapallindexed",=20no_argument,=20NULL,=20'A'},=0A+=09=09= {"no-password",=20no_argument,=20NULL,=20'w'},=0A+=09=09= {"no-rootdescend",=20no_argument,=20NULL,=20'R'},=0A+=09=09= {"on-error-stop",=20no_argument,=20NULL,=20'o'},=0A+=09=09{"password",=20= no_argument,=20NULL,=20'W'},=0A+=09=09{"port",=20required_argument,=20= NULL,=20'p'},=0A+=09=09{"rootdescend",=20no_argument,=20NULL,=20'r'},=0A= +=09=09{"schema",=20required_argument,=20NULL,=20'n'},=0A+=09=09= {"skip-all-frozen",=20no_argument,=20NULL,=20'f'},=0A+=09=09= {"skip-all-visible",=20no_argument,=20NULL,=20'v'},=0A+=09=09= {"skip-corrupt",=20no_argument,=20NULL,=20'C'},=0A+=09=09= {"skip-indexes",=20no_argument,=20NULL,=20'X'},=0A+=09=09{"skip-toast",=20= no_argument,=20NULL,=20'Z'},=0A+=09=09{"startblock",=20= required_argument,=20NULL,=20'b'},=0A+=09=09{"strict-names",=20= no_argument,=20NULL,=20's'},=0A+=09=09{"table",=20required_argument,=20= NULL,=20't'},=0A+=09=09{"toast-endblock",=20required_argument,=20NULL,=20= 'E'},=0A+=09=09{"toast-startblock",=20required_argument,=20NULL,=20'B'},=0A= +=09=09{"username",=20required_argument,=20NULL,=20'U'},=0A+=09=09= {"version",=20no_argument,=20NULL,=20'V'},=0A+=09=09{NULL,=200,=20NULL,=20= 0}=0A+=09};=0A+=0A+=09int=09=09=09optindex;=0A+=09int=09=09=09c;=0A+=0A+=09= memset(connOpts,=200,=20sizeof=20*connOpts);=0A+=0A+=09while=20((c=20=3D=20= getopt_long(argc,=20argv,=20= "aAb:B:cCd:e:E:fh:i:I:n:N:op:rRst:T:U:vVwWxXzZ?1",=0A+=09=09=09=09=09=09=09= long_options,=20&optindex))=20!=3D=20-1)=0A+=09{=0A+=09=09switch=20(c)=0A= +=09=09{=0A+=09=09=09case=20'a':=0A+=09=09=09=09settings.heapallindexed=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'A':=0A+=09=09=09=09= settings.heapallindexed=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'b':=0A+=09=09=09=09settings.startblock=20=3D=20= pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'B':=0A+=09=09= =09=09settings.toaststart=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09= break;=0A+=09=09=09case=20'c':=0A+=09=09=09=09settings.check_corrupt=20=3D= =20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'C':=0A+=09=09=09=09= settings.check_corrupt=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'd':=0A+=09=09=09=09connOpts->dbname=20=3D=20pg_strdup(optarg);=0A= +=09=09=09=09break;=0A+=09=09=09case=20'e':=0A+=09=09=09=09= settings.endblock=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09= =09=09case=20'E':=0A+=09=09=09=09settings.toastend=20=3D=20= pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'f':=0A+=09=09= =09=09settings.skip_frozen=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'h':=0A+=09=09=09=09connOpts->host=20=3D=20pg_strdup(optarg);=0A+=09= =09=09=09break;=0A+=09=09=09case=20'i':=0A+=09=09=09=09= simple_string_list_append(&index_include_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'I':=0A+=09=09=09=09= simple_string_list_append(&index_exclude_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'n':=09=09=09/*=20include=20schema(s)=20*/=0A= +=09=09=09=09simple_string_list_append(&schema_include_patterns,=20= optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'N':=09=09=09/*=20= exclude=20schema(s)=20*/=0A+=09=09=09=09= simple_string_list_append(&schema_exclude_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'o':=0A+=09=09=09=09settings.on_error_stop=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'p':=0A+=09=09=09=09= connOpts->port=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09= =09case=20's':=0A+=09=09=09=09settings.strict_names=20=3D=20true;=0A+=09=09= =09=09break;=0A+=09=09=09case=20'r':=0A+=09=09=09=09settings.rootdescend=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'R':=0A+=09=09=09=09= settings.rootdescend=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20't':=09=09=09/*=20include=20table(s)=20*/=0A+=09=09=09=09= simple_string_list_append(&table_include_patterns,=20optarg);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'T':=09=09=09/*=20exclude=20table(s)=20*/=0A= +=09=09=09=09simple_string_list_append(&table_exclude_patterns,=20= optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'U':=0A+=09=09=09=09= connOpts->username=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09= =09=09case=20'v':=0A+=09=09=09=09settings.skip_visible=20=3D=20true;=0A+=09= =09=09=09break;=0A+=09=09=09case=20'V':=0A+=09=09=09=09showVersion();=0A= +=09=09=09=09exit(EXIT_SUCCESS);=0A+=09=09=09case=20'w':=0A+=09=09=09=09= settings.getPassword=20=3D=20TRI_NO;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'W':=0A+=09=09=09=09settings.getPassword=20=3D=20TRI_YES;=0A+=09=09= =09=09break;=0A+=09=09=09case=20'x':=0A+=09=09=09=09= settings.check_indexes=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'X':=0A+=09=09=09=09settings.check_indexes=20=3D=20false;=0A+=09=09= =09=09break;=0A+=09=09=09case=20'z':=0A+=09=09=09=09settings.check_toast=20= =3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20'Z':=0A+=09=09=09=09= settings.check_toast=20=3D=20false;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'?':=0A+=09=09=09=09if=20(optind=20<=3D=20argc=20&&=0A+=09=09=09=09= =09strcmp(argv[optind=20-=201],=20"-?")=20=3D=3D=200)=0A+=09=09=09=09{=0A= +=09=09=09=09=09/*=20actual=20help=20option=20given=20*/=0A+=09=09=09=09=09= usage();=0A+=09=09=09=09=09exit(EXIT_SUCCESS);=0A+=09=09=09=09}=0A+=09=09= =09=09else=0A+=09=09=09=09{=0A+=09=09=09=09=09/*=20getopt=20error=20= (unknown=20option=20or=20missing=20argument)=20*/=0A+=09=09=09=09=09goto=20= unknown_option;=0A+=09=09=09=09}=0A+=09=09=09=09break;=0A+=09=09=09case=20= 1:=0A+=09=09=09=09{=0A+=09=09=09=09=09if=20(!optarg=20||=20= strcmp(optarg,=20"options")=20=3D=3D=200)=0A+=09=09=09=09=09=09usage();=0A= +=09=09=09=09=09else=0A+=09=09=09=09=09=09goto=20unknown_option;=0A+=0A+=09= =09=09=09=09exit(EXIT_SUCCESS);=0A+=09=09=09=09}=0A+=09=09=09=09break;=0A= +=09=09=09default:=0A+=09=09unknown_option:=0A+=09=09=09=09= fprintf(stderr,=20_("Try=20\"%s=20--help\"=20for=20more=20= information.\n"),=0A+=09=09=09=09=09=09settings.progname);=0A+=09=09=09=09= exit(EXIT_FAILURE);=0A+=09=09=09=09break;=0A+=09=09}=0A+=09}=0A+=0A+=09= /*=0A+=09=20*=20if=20we=20still=20have=20arguments,=20use=20it=20as=20= the=20database=20name=20and=20username=0A+=09=20*/=0A+=09while=20(argc=20= -=20optind=20>=3D=201)=0A+=09{=0A+=09=09if=20(!connOpts->dbname)=0A+=09=09= =09connOpts->dbname=20=3D=20argv[optind];=0A+=09=09else=20if=20= (!connOpts->username)=0A+=09=09=09connOpts->username=20=3D=20= argv[optind];=0A+=09=09else=0A+=09=09=09pg_log_warning("extra=20= command-line=20argument=20\"%s\"=20ignored",=0A+=09=09=09=09=09=09=20=20=20= argv[optind]);=0A+=0A+=09=09optind++;=0A+=09}=0A+=0A+}=0A+=0A+/*=0A+=20*=20= usage=0A+=20*=0A+=20*=20print=20out=20command=20line=20arguments=0A+=20= */=0A+static=20void=0A+usage(void)=0A+{=0A+=09int=09=09=09lineno;=0A+=0A= +=09for=20(lineno=20=3D=200;=20usage_text[lineno];=20lineno++)=0A+=09=09= printf("%s\n",=20usage_text[lineno]);=0A+=09printf("Report=20bugs=20to=20= <%s>.\n",=20PACKAGE_BUGREPORT);=0A+=09printf("%s=20home=20page:=20= <%s>\n",=20PACKAGE_NAME,=20PACKAGE_URL);=0A+}=0A+=0A+static=20void=0A= +showVersion(void)=0A+{=0A+=09puts("pg_amcheck=20(PostgreSQL)=20"=20= PG_VERSION);=0A+}=0A+=0A+/*=0A+=20*=20for=20backend=20Notice=20messages=20= (INFO,=20WARNING,=20etc)=0A+=20*/=0A+static=20void=0A= +NoticeProcessor(void=20*arg,=20const=20char=20*message)=0A+{=0A+=09= (void)=20arg;=09=09=09=09=09/*=20not=20used=20*/=0A+=09pg_log_info("%s",=20= message);=0A+}=0A+=0A+/*=0A+=20*=20Helper=20function=20for=20= apply_filter,=20below.=0A+=20*/=0A+static=20void=0A= +append_csv_oids(PQExpBuffer=20querybuf,=20const=20SimpleOidList=20= *oids)=0A+{=0A+=09const=20SimpleOidListCell=20*cell;=0A+=09const=20char=20= *comma;=0A+=0A+=09for=20(comma=20=3D=20"",=20cell=20=3D=20oids->head;=20= cell;=20comma=20=3D=20",=20",=20cell=20=3D=20cell->next)=0A+=09=09= appendPQExpBuffer(querybuf,=20"%s%u",=20comma,=20cell->val);=0A+}=0A+=0A= +/*=0A+=20*=20Internal=20implementation=20of=20include_filter=20and=20= exclude_filter=0A+=20*/=0A+static=20void=0A+apply_filter(PQExpBuffer=20= querybuf,=20const=20char=20*lval,=20const=20SimpleOidList=20*oids,=0A+=09= =09=09=20bool=20include)=0A+{=0A+=09if=20(!oids=20||=20!oids->head)=0A+=09= =09return;=0A+=09if=20(include)=0A+=09=09appendPQExpBuffer(querybuf,=20= "\nAND=20%s=20OPERATOR(pg_catalog.=3D)=20ANY(array[",=20lval);=0A+=09= else=0A+=09=09appendPQExpBuffer(querybuf,=20"\nAND=20%s=20= OPERATOR(pg_catalog.!=3D)=20ALL(array[",=20lval);=0A+=09= append_csv_oids(querybuf,=20oids);=0A+=09appendPQExpBuffer(querybuf,=20= "]::OID[])");=0A+}=0A+=0A+/*=0A+=20*=20Find=20and=20append=20to=20the=20= given=20Oid=20list=20the=20Oids=20of=20all=20schemas=20matching=20the=0A= +=20*=20given=20list=20of=20patterns=20but=20not=20included=20in=20the=20= given=20list=20of=20excluded=20Oids.=0A+=20*/=0A+static=20void=0A= +expand_schema_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09= =09=09=09=09=09=09const=20SimpleOidList=20*exclude_nsp,=0A+=09=09=09=09=09= =09=09SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09bool=20= strict_names)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09PGresult=20=20=20= *res;=0A+=09SimpleStringListCell=20*cell;=0A+=09int=09=09=09i;=0A+=0A+=09= if=20(settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20= entry=20to=20expand_schema_name_patterns");=0A+=0A+=09if=20= (patterns->head=20=3D=3D=20NULL)=0A+=09=09return;=09=09=09=09=09/*=20= nothing=20to=20do=20*/=0A+=0A+=09querybuf=20=3D=20createPQExpBuffer();=0A= +=0A+=09/*=0A+=09=20*=20The=20loop=20below=20runs=20multiple=20SELECTs=20= might=20sometimes=20result=20in=0A+=09=20*=20duplicate=20entries=20in=20= the=20Oid=20list,=20but=20we=20don't=20care.=0A+=09=20*/=0A+=0A+=09for=20= (cell=20=3D=20patterns->head;=20cell;=20cell=20=3D=20cell->next)=0A+=09{=0A= +=09=09appendPQExpBufferStr(querybuf,=0A+=09=09=09=09=09=09=09=20"SELECT=20= oid=20FROM=20pg_catalog.pg_namespace=20n\n");=0A+=09=09= processSQLNamePattern(settings.db,=20querybuf,=20cell->val,=20false,=0A+=09= =09=09=09=09=09=09=20=20false,=20NULL,=20"n.nspname",=20NULL,=20NULL);=0A= +=09=09exclude_filter(querybuf,=20"n.oid",=20exclude_nsp);=0A+=0A+=09=09= res=20=3D=20ExecuteSqlQueryOrDie(querybuf->data);=0A+=09=09if=20= (strict_names=20&&=20PQntuples(res)=20=3D=3D=200)=0A+=09=09=09fatal("no=20= matching=20schemas=20were=20found=20for=20pattern=20\"%s\"",=0A+=09=09=09= =09=20=20cell->val);=0A+=0A+=09=09for=20(i=20=3D=200;=20i=20<=20= PQntuples(res);=20i++)=0A+=09=09{=0A+=09=09=09= simple_oid_list_append(oids,=20atooid(PQgetvalue(res,=20i,=200)));=0A+=09= =09}=0A+=0A+=09=09PQclear(res);=0A+=09=09resetPQExpBuffer(querybuf);=0A+=09= }=0A+=0A+=09destroyPQExpBuffer(querybuf);=0A+}=0A+=0A+/*=0A+=20*=20Find=20= and=20append=20to=20the=20given=20Oid=20list=20the=20Oids=20of=20all=20= relations=20matching=20the=0A+=20*=20given=20list=20of=20patterns=20but=20= not=20included=20in=20the=20given=20list=20of=20excluded=20Oids=0A+=20*=20= nor=20in=20one=20of=20the=20given=20excluded=20namespaces.=20=20The=20= relations=20are=20filtered=20by=0A+=20*=20the=20given=20schema_quals.=20=20= They=20are=20further=20filtered=20by=20the=20given=0A+=20*=20= relkind_quals,=20allowing=20the=20caller=20to=20restrict=20the=20= relations=20to=20just=20indexes=0A+=20*=20or=20tables.=20=20The=20= missing_errtext=20should=20be=20a=20message=20for=20use=20in=20error=0A+=20= *=20messages=20if=20no=20matching=20relations=20are=20found=20and=20= strict_names=20was=20specified.=0A+=20*/=0A+static=20void=0A= +expand_relkind_name_patterns(const=20SimpleStringList=20*patterns,=0A+=09= =09=09=09=09=09=09=20const=20SimpleOidList=20*exclude_nsp_oids,=0A+=09=09= =09=09=09=09=09=20const=20SimpleOidList=20*exclude_oids,=0A+=09=09=09=09=09= =09=09=20SimpleOidList=20*oids,=0A+=09=09=09=09=09=09=09=20bool=20= strict_names,=0A+=09=09=09=09=09=09=09=20const=20char=20= *missing_errtext,=0A+=09=09=09=09=09=09=09=20const=20char=20= *relkind_quals)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09PGresult=20=20=20= *res;=0A+=09SimpleStringListCell=20*cell;=0A+=09int=09=09=09i;=0A+=0A+=09= if=20(settings.db=20=3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20= entry=20to=20expand_relkind_name_patterns");=0A+=0A+=09if=20= (patterns->head=20=3D=3D=20NULL)=0A+=09=09return;=09=09=09=09=09/*=20= nothing=20to=20do=20*/=0A+=0A+=09querybuf=20=3D=20createPQExpBuffer();=0A= +=0A+=09/*=0A+=09=20*=20this=20might=20sometimes=20result=20in=20= duplicate=20entries=20in=20the=20Oid=20list,=20but=0A+=09=20*=20we=20= don't=20care.=0A+=09=20*/=0A+=0A+=09for=20(cell=20=3D=20patterns->head;=20= cell;=20cell=20=3D=20cell->next)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20= Query=20must=20remain=20ABSOLUTELY=20devoid=20of=20unqualified=20names.=20= =20This=0A+=09=09=20*=20would=20be=20unnecessary=20given=20a=20= pg_table_is_visible()=20variant=20taking=20a=0A+=09=09=20*=20search_path=20= argument.=0A+=09=09=20*/=0A+=09=09appendPQExpBuffer(querybuf,=0A+=09=09=09= =09=09=09=20=20"SELECT=20c.oid"=0A+=09=09=09=09=09=09=20=20"\nFROM=20= pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=09=20=20"\n=20=20=20=20LEFT=20= JOIN=20pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=09=20=20"\n=20=20=20= =20ON=20n.oid=20OPERATOR(pg_catalog.=3D)=20c.relnamespace"=0A+=09=09=09=09= =09=09=20=20"\nWHERE=20c.relkind=20OPERATOR(pg_catalog.=3D)=20%s\n",=0A+=09= =09=09=09=09=09=20=20relkind_quals);=0A+=09=09exclude_filter(querybuf,=20= "c.oid",=20exclude_oids);=0A+=09=09exclude_filter(querybuf,=20"n.oid",=20= exclude_nsp_oids);=0A+=09=09processSQLNamePattern(settings.db,=20= querybuf,=20cell->val,=20true,=0A+=09=09=09=09=09=09=09=20=20false,=20= "n.nspname",=20"c.relname",=20NULL,=20NULL);=0A+=09=09res=20=3D=20= ExecuteSqlQueryOrDie(querybuf->data);=0A+=09=09if=20(strict_names=20&&=20= PQntuples(res)=20=3D=3D=200)=0A+=09=09=09fatal("%s=20\"%s\"",=20= missing_errtext,=20cell->val);=0A+=0A+=09=09for=20(i=20=3D=200;=20i=20<=20= PQntuples(res);=20i++)=0A+=09=09=09simple_oid_list_append(oids,=20= atooid(PQgetvalue(res,=20i,=200)));=0A+=0A+=09=09PQclear(res);=0A+=09=09= resetPQExpBuffer(querybuf);=0A+=09}=0A+=0A+=09= destroyPQExpBuffer(querybuf);=0A+}=0A+=0A+/*=0A+=20*=20Find=20the=20Oids=20= of=20all=20tables=20matching=20the=20given=20list=20of=20patterns,=0A+=20= *=20and=20append=20them=20to=20the=20given=20Oid=20list.=0A+=20*/=0A= +static=20void=0A+expand_table_name_patterns(const=20SimpleStringList=20= *patterns,=20const=20SimpleOidList=20*exclude_nsp_oids,=0A+=09=09=09=09=09= =09=20=20=20const=20SimpleOidList=20*exclude_oids,=20SimpleOidList=20= *oids,=20bool=20strict_names)=0A+{=0A+=09= expand_relkind_name_patterns(patterns,=20exclude_nsp_oids,=20= exclude_oids,=20oids,=20strict_names,=0A+=09=09=09=09=09=09=09=09=20"no=20= matching=20tables=20were=20found=20for=20pattern",=0A+=09=09=09=09=09=09=09= =09=20get_table_relkind_quals());=0A+}=0A+=0A+/*=0A+=20*=20Find=20the=20= Oids=20of=20all=20indexes=20matching=20the=20given=20list=20of=20= patterns,=0A+=20*=20and=20append=20them=20to=20the=20given=20Oid=20list.=0A= +=20*/=0A+static=20void=0A+expand_index_name_patterns(const=20= SimpleStringList=20*patterns,=20const=20SimpleOidList=20= *exclude_nsp_oids,=0A+=09=09=09=09=09=09=20=20=20const=20SimpleOidList=20= *exclude_oids,=20SimpleOidList=20*oids,=20bool=20strict_names)=0A+{=0A+=09= expand_relkind_name_patterns(patterns,=20exclude_nsp_oids,=20= exclude_oids,=20oids,=20strict_names,=0A+=09=09=09=09=09=09=09=09=20"no=20= matching=20indexes=20were=20found=20for=20pattern",=0A+=09=09=09=09=09=09= =09=09=20get_index_relkind_quals());=0A+}=0A+=0A+static=20void=0A= +get_table_check_list(const=20SimpleOidList=20*include_nsp,=20const=20= SimpleOidList=20*exclude_nsp,=0A+=09=09=09=09=09=20const=20SimpleOidList=20= *include_tbl,=20const=20SimpleOidList=20*exclude_tbl,=0A+=09=09=09=09=09=20= SimpleOidList=20*checklist)=0A+{=0A+=09PQExpBuffer=20querybuf;=0A+=09= PGresult=20=20=20*res;=0A+=09int=09=09=09i;=0A+=0A+=09if=20(settings.db=20= =3D=3D=20NULL)=0A+=09=09fatal("no=20connection=20on=20entry=20to=20= get_table_check_list");=0A+=0A+=09querybuf=20=3D=20createPQExpBuffer();=0A= +=0A+=09appendPQExpBuffer(querybuf,=0A+=09=09=09=09=09=20=20"SELECT=20= c.oid"=0A+=09=09=09=09=09=20=20"\nFROM=20pg_catalog.pg_class=20c,=20= pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=20=20"\nWHERE=20n.oid=20= OPERATOR(pg_catalog.=3D)=20c.relnamespace"=0A+=09=09=09=09=09=20=20"\n=09= AND=20c.relkind=20OPERATOR(pg_catalog.=3D)=20%s\n",=0A+=09=09=09=09=09=20= =20get_table_relkind_quals());=0A+=09include_filter(querybuf,=20"n.oid",=20= include_nsp);=0A+=09exclude_filter(querybuf,=20"n.oid",=20exclude_nsp);=0A= +=09include_filter(querybuf,=20"c.oid",=20include_tbl);=0A+=09= exclude_filter(querybuf,=20"c.oid",=20exclude_tbl);=0A+=0A+=09res=20=3D=20= ExecuteSqlQueryOrDie(querybuf->data);=0A+=09for=20(i=20=3D=200;=20i=20<=20= PQntuples(res);=20i++)=0A+=09=09simple_oid_list_append(checklist,=20= atooid(PQgetvalue(res,=20i,=200)));=0A+=0A+=09PQclear(res);=0A+=09= destroyPQExpBuffer(querybuf);=0A+}=0A+=0A+static=20PGresult=20*=0A= +ExecuteSqlQueryOrDie(const=20char=20*query)=0A+{=0A+=09PGresult=20=20=20= *res;=0A+=0A+=09res=20=3D=20PQexec(settings.db,=20query);=0A+=09if=20= (PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09=09= die_on_query_failure(query);=0A+=09return=20res;=0A+}=0A+=0A+/*=0A+=20*=20= Execute=20the=20given=20SQL=20query.=20=20This=20function=20should=20= only=20be=20used=20for=20queries=0A+=20*=20which=20are=20not=20expected=20= to=20fail=20under=20normal=20circumstances,=20as=20failures=20will=0A+=20= *=20result=20in=20the=20printing=20of=20error=20messages,=20which=20will=20= look=20a=20bit=20messy=20when=0A+=20*=20interleaved=20with=20corruption=20= reports.=0A+=20*=0A+=20*=20On=20error,=20use=20the=20supplied=20= error_context=20string=20and=20the=20error=20string=0A+=20*=20returned=20= from=20the=20database=20connection=20to=20print=20an=20error=20message=20= for=20the=0A+=20*=20user.=0A+=20*=0A+=20*=20The=20error_context=20= argument=20is=20pfree'd=20by=20us=20at=20the=20end=20of=20the=20call.=0A= +=20*/=0A+static=20PGresult=20*=0A+ExecuteSqlQuery(const=20char=20= *query,=20char=20**error)=0A+{=0A+=09PGresult=20=20=20*res;=0A+=0A+=09= res=20=3D=20PQexec(settings.db,=20query);=0A+=09if=20= (PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09=09*error=20=3D=20= pstrdup(PQerrorMessage(settings.db));=0A+=09return=20res;=0A+}=0A+=0A+/*=0A= +=20*=20Return=20the=20cached=20relkind=20quals=20string=20for=20tables,=20= computing=20it=20first=20if=20we=0A+=20*=20don't=20have=20one=20cached.=0A= +=20*/=0A+static=20const=20char=20*=0A+get_table_relkind_quals(void)=0A= +{=0A+=09if=20(!table_relkind_quals)=0A+=09=09table_relkind_quals=20=3D=20= psprintf("ANY(array['%c',=20'%c',=20'%c'])",=0A+=09=09=09=09=09=09=09=09=09= =20=20=20RELKIND_RELATION,=20RELKIND_MATVIEW,=0A+=09=09=09=09=09=09=09=09= =09=20=20=20RELKIND_PARTITIONED_TABLE);=0A+=09return=20= table_relkind_quals;=0A+}=0A+=0A+/*=0A+=20*=20Return=20the=20cached=20= relkind=20quals=20string=20for=20indexes,=20computing=20it=20first=20if=20= we=0A+=20*=20don't=20have=20one=20cached.=0A+=20*/=0A+static=20const=20= char=20*=0A+get_index_relkind_quals(void)=0A+{=0A+=09if=20= (!index_relkind_quals)=0A+=09=09index_relkind_quals=20=3D=20= psprintf("'%c'",=20RELKIND_INDEX);=0A+=09return=20index_relkind_quals;=0A= +}=0Adiff=20--git=20a/contrib/pg_amcheck/pg_amcheck.control=20= b/contrib/pg_amcheck/pg_amcheck.control=0Anew=20file=20mode=20100644=0A= index=200000000000..395f368101=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/pg_amcheck.control=0A@@=20-0,0=20+1,5=20@@=0A+#=20= pg_amcheck=20extension=0A+comment=20=3D=20'command-line=20tool=20for=20= verifying=20relation=20integrity'=0A+default_version=20=3D=20'1.3'=0A= +module_pathname=20=3D=20'$libdir/pg_amcheck'=0A+relocatable=20=3D=20= true=0Adiff=20--git=20a/contrib/pg_amcheck/t/001_basic.pl=20= b/contrib/pg_amcheck/t/001_basic.pl=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..dfa0ae9e06=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/001_basic.pl=0A@@=20-0,0=20+1,9=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20TestLib;=0A+use=20Test::More=20= tests=20=3D>=208;=0A+=0A+program_help_ok('pg_amcheck');=0A= +program_version_ok('pg_amcheck');=0A= +program_options_handling_ok('pg_amcheck');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/002_nonesuch.pl=20= b/contrib/pg_amcheck/t/002_nonesuch.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..68be9c6585=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/002_nonesuch.pl=0A@@=20-0,0=20+1,60=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20TestLib;=0A= +use=20Test::More=20tests=20=3D>=2014;=0A+=0A+#=20Test=20set-up=0A+my=20= ($node,=20$port);=0A+$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+$port=20=3D=20$node->port;=0A+=0A+#=20Load=20the=20= amcheck=20extension,=20upon=20which=20pg_amcheck=20depends=0A= +$node->safe_psql('postgres',=20q(CREATE=20EXTENSION=20amcheck));=0A+=0A= +#########################################=0A+#=20Test=20connecting=20to=20= a=20non-existent=20database=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20"$port",=20'qqq'=20],=0A+=09qr/\Qpg_amcheck:=20= error:=20could=20not=20connect=20to=20server:=20FATAL:=20=20database=20= "qqq"=20does=20not=20exist\E/,=0A+=09'connecting=20to=20a=20non-existent=20= database');=0A+=0A+#########################################=0A+#=20Test=20= connecting=20with=20a=20non-existent=20user=0A+=0A+command_fails_like(=0A= +=09[=20'pg_amcheck',=20'-p',=20"$port",=20'-U=3Dno_such_user'=20],=0A+=09= qr/\Qpg_amcheck:=20error:=20could=20not=20connect=20to=20server:=20= FATAL:=20=20role=20"=3Dno_such_user"=20does=20not=20exist\E/,=0A+=09= 'connecting=20with=20a=20non-existent=20user');=0A+=0A= +#########################################=0A+#=20Test=20checking=20a=20= non-existent=20schema,=20table,=20and=20patterns=20with=20--strict-names=0A= +=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20"$port",=20= '-n',=20'nonexistent'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20no=20= matching=20schemas=20were=20found\E/,=0A+=09'checking=20a=20non-existent=20= schema');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20= "$port",=20'-t',=20'nonexistent'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20= no=20matching=20tables=20were=20found\E/,=0A+=09'checking=20a=20= non-existent=20table');=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20"$port",=20'--strict-names',=20'-n',=20= 'nonexistent*'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20no=20matching=20= schemas=20were=20found=20for=20pattern\E/,=0A+=09'no=20matching=20= schemas');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20= "$port",=20'--strict-names',=20'-t',=20'nonexistent*'=20],=0A+=09= qr/\Qpg_amcheck:=20error:=20no=20matching=20tables=20were=20found=20for=20= pattern\E/,=0A+=09'no=20matching=20tables');=0A+=0A+command_fails_like(=0A= +=09[=20'pg_amcheck',=20'-p',=20"$port",=20'--strict-names',=20'-i',=20= 'nonexistent*'=20],=0A+=09qr/\Qpg_amcheck:=20error:=20no=20matching=20= indexes=20were=20found=20for=20pattern\E/,=0A+=09'no=20matching=20= indexes');=0Adiff=20--git=20a/contrib/pg_amcheck/t/003_check.pl=20= b/contrib/pg_amcheck/t/003_check.pl=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..4d8e61d871=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/003_check.pl=0A@@=20-0,0=20+1,231=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20TestLib;=0A= +use=20Test::More=20tests=20=3D>=2039;=0A+=0A+my=20($node,=20$port);=0A+=0A= +#=20Returns=20the=20filesystem=20path=20for=20the=20named=20relation.=0A= +#=0A+#=20Assumes=20the=20test=20node=20is=20running=0A+sub=20= relation_filepath($)=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A+=0A+=09my=20= $pgdata=20=3D=20$node->data_dir;=0A+=09my=20$rel=20=3D=20= $node->safe_psql('postgres',=0A+=09=09=09=09=09=09=09=20=20=20qq(SELECT=20= pg_relation_filepath('$relname')));=0A+=09die=20"path=20not=20found=20= for=20relation=20$relname"=20unless=20defined=20$rel;=0A+=09return=20= "$pgdata/$rel";=0A+}=0A+=0A+#=20Stops=20the=20test=20node,=20corrupts=20= the=20first=20page=20of=20the=20named=20relation,=20and=0A+#=20restarts=20= the=20node.=0A+#=0A+#=20Assumes=20the=20node=20is=20running.=0A+sub=20= corrupt_first_page($)=0A+{=0A+=09my=20($relname)=20=3D=20@_;=0A+=09my=20= $relpath=20=3D=20relation_filepath($relname);=0A+=0A+=09$node->stop;=0A+=09= my=20$fh;=0A+=09open($fh,=20'+<',=20$relpath);=0A+=09binmode=20$fh;=0A+=09= seek($fh,=2032,=200);=0A+=09syswrite($fh,=20'\x77\x77\x77\x77',=20500);=0A= +=09close($fh);=0A+=09$node->start;=0A+}=0A+=0A+#=20Stops=20the=20test=20= node,=20unlinks=20the=20file=20from=20the=20filesystem=20that=20backs=20= the=0A+#=20relation,=20and=20restarts=20the=20node.=0A+#=0A+#=20Assumes=20= the=20test=20node=20is=20running=0A+sub=20remove_relation_file($)=0A+{=0A= +=09my=20($relname)=20=3D=20@_;=0A+=09my=20$relpath=20=3D=20= relation_filepath($relname);=0A+=0A+=09$node->stop();=0A+=09= unlink($relpath);=0A+=09$node->start;=0A+}=0A+=0A+#=20Test=20set-up=0A= +$node=20=3D=20get_new_node('test');=0A+$node->init;=0A+$node->start;=0A= +$port=20=3D=20$node->port;=0A+=0A+#=20Load=20the=20amcheck=20extension,=20= upon=20which=20pg_amcheck=20depends=0A+$node->safe_psql('postgres',=20= q(CREATE=20EXTENSION=20amcheck));=0A+=0A+#=20Create=20schemas=20and=20= tables=20for=20checking=20pg_amcheck's=20include=0A+#=20and=20exclude=20= schema=20and=20table=20command=20line=20options=0A= +$node->safe_psql('postgres',=20q(=0A+--=20We'll=20corrupt=20all=20= indexes=20in=20s1=0A+CREATE=20SCHEMA=20s1;=0A+CREATE=20TABLE=20s1.t1=20= (a=20TEXT);=0A+CREATE=20TABLE=20s1.t2=20(a=20TEXT);=0A+CREATE=20INDEX=20= i1=20ON=20s1.t1(a);=0A+CREATE=20INDEX=20i2=20ON=20s1.t2(a);=0A+INSERT=20= INTO=20s1.t1=20(a)=20(SELECT=20gs::TEXT=20FROM=20= generate_series(1,10000)=20AS=20gs);=0A+INSERT=20INTO=20s1.t2=20(a)=20= (SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A+=0A= +--=20We'll=20corrupt=20all=20tables=20in=20s2=0A+CREATE=20SCHEMA=20s2;=0A= +CREATE=20TABLE=20s2.t1=20(a=20TEXT);=0A+CREATE=20TABLE=20s2.t2=20(a=20= TEXT);=0A+CREATE=20INDEX=20i1=20ON=20s2.t1(a);=0A+CREATE=20INDEX=20i2=20= ON=20s2.t2(a);=0A+INSERT=20INTO=20s2.t1=20(a)=20(SELECT=20gs::TEXT=20= FROM=20generate_series(1,10000)=20AS=20gs);=0A+INSERT=20INTO=20s2.t2=20= (a)=20(SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A= +=0A+--=20We'll=20corrupt=20all=20tables=20and=20indexes=20in=20s3=0A= +CREATE=20SCHEMA=20s3;=0A+CREATE=20TABLE=20s3.t1=20(a=20TEXT);=0A+CREATE=20= TABLE=20s3.t2=20(a=20TEXT);=0A+CREATE=20INDEX=20i1=20ON=20s3.t1(a);=0A= +CREATE=20INDEX=20i2=20ON=20s3.t2(a);=0A+INSERT=20INTO=20s3.t1=20(a)=20= (SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A= +INSERT=20INTO=20s3.t2=20(a)=20(SELECT=20gs::TEXT=20FROM=20= generate_series(1,10000)=20AS=20gs);=0A+=0A+--=20We'll=20leave=20= everything=20in=20s4=20uncorrupted=0A+CREATE=20SCHEMA=20s4;=0A+CREATE=20= TABLE=20s4.t1=20(a=20TEXT);=0A+CREATE=20TABLE=20s4.t2=20(a=20TEXT);=0A= +CREATE=20INDEX=20i1=20ON=20s4.t1(a);=0A+CREATE=20INDEX=20i2=20ON=20= s4.t2(a);=0A+INSERT=20INTO=20s4.t1=20(a)=20(SELECT=20gs::TEXT=20FROM=20= generate_series(1,10000)=20AS=20gs);=0A+INSERT=20INTO=20s4.t2=20(a)=20= (SELECT=20gs::TEXT=20FROM=20generate_series(1,10000)=20AS=20gs);=0A+));=0A= +=0A+#=20Corrupt=20indexes=20in=20schema=20"s1"=0A= +remove_relation_file('s1.i1');=0A+corrupt_first_page('s1.i2');=0A+=0A+#=20= Corrupt=20tables=20in=20schema=20"s2"=0A+remove_relation_file('s2.t1');=0A= +corrupt_first_page('s2.t2');=0A+=0A+#=20Corrupt=20tables=20and=20= indexes=20in=20schema=20"s3"=0A+remove_relation_file('s3.i1');=0A= +corrupt_first_page('s3.i2');=0A+remove_relation_file('s3.t1');=0A= +corrupt_first_page('s3.t2');=0A+=0A+#=20Leave=20schema=20"s4"=20alone=0A= +=0A+=0A+#=20The=20pg_amcheck=20command=20itself=20should=20return=20a=20= success=20exit=20status,=20even=0A+#=20though=20tables=20and=20indexes=20= are=20corrupt.=20=20An=20error=20code=20returned=20would=20mean=20the=0A= +#=20pg_amcheck=20command=20itself=20failed,=20for=20example=20because=20= a=20connection=20to=20the=0A+#=20database=20could=20not=20be=20= established.=0A+#=0A+#=20For=20these=20checks,=20we're=20ignoring=20any=20= corruption=20reported=20and=20focusing=0A+#=20exclusively=20on=20the=20= exit=20code=20from=20pg_amcheck.=0A+#=0A+$node->command_ok(=0A+=09[=20= 'pg_amcheck',=20'-x',=20'-p',=20$port,=20'postgres'=20],=0A+=09= 'pg_amcheck=20all=20schemas=20and=20tables');=0A+=0A+$node->command_ok(=0A= +=09[=20'pg_amcheck',=20'-p',=20$port,=20'postgres'=20],=0A+=09= 'pg_amcheck=20all=20schemas,=20tables=20and=20indexes');=0A+=0A= +$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1'=20],=0A+=09'pg_amcheck=20all=20objects=20in=20= schema=20s1');=0A+=0A+$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-p',=20= $port,=20'postgres',=20'-n',=20's*',=20'-t',=20't1'=20],=0A+=09= 'pg_amcheck=20all=20tables=20named=20t1=20and=20their=20indexes');=0A+=0A= +$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20$port,=20= 'postgres',=20'-T',=20't1'=20],=0A+=09'pg_amcheck=20all=20tables=20not=20= named=20t1');=0A+=0A+$node->command_ok(=0A+=09[=20'pg_amcheck',=20'-x',=20= '-p',=20$port,=20'postgres',=20'-N',=20's1',=20'-T',=20't1'=20],=0A+=09= 'pg_amcheck=20all=20tables=20not=20named=20t1=20nor=20in=20schema=20= s1');=0A+=0A+#=20Scans=20of=20indexes=20in=20s1=20should=20detect=20the=20= specific=20corruption=20that=20we=20created=0A+#=20above.=20=20For=20= missing=20relation=20forks,=20we=20know=20what=20the=20error=20message=20= looks=0A+#=20like.=20=20For=20corrupted=20index=20pages,=20the=20error=20= might=20vary=20depending=20on=20how=20the=0A+#=20page=20was=20formatted=20= on=20disk,=20including=20variations=20due=20to=20alignment=20differences=0A= +#=20between=20platforms,=20so=20we=20accept=20any=20non-empty=20error=20= message.=0A+#=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20= '-p',=20$port,=20'postgres',=20'-n',=20's1',=20'-i',=20'i1'=20],=0A+=09= qr/index=20"i1"=20lacks=20a=20main=20relation=20fork/,=0A+=09'pg_amcheck=20= index=20s1.i1=20reports=20missing=20main=20relation=20fork');=0A+=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1',=20'-i',=20'i2'=20],=0A+=09qr/.+/,=09=09=09#=20= Any=20non-empty=20error=20message=20is=20acceptable=0A+=09'pg_amcheck=20= index=20s1.s2=20reports=20index=20corruption');=0A+=0A+=0A+#=20In=20= schema=20s3,=20the=20tables=20and=20indexes=20are=20both=20corrupt.=20=20= Ordinarily,=20checking=0A+#=20of=20indexes=20will=20not=20be=20performed=20= for=20corrupt=20tables,=20but=20the=20--check-corrupt=0A+#=20option=20= (-c)=20forces=20the=20indexes=20to=20also=20be=20checked.=0A+#=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-c',=20'-p',=20= $port,=20'postgres',=20'-n',=20's3',=20'-i',=20'i1'=20],=0A+=09qr/index=20= "i1"=20lacks=20a=20main=20relation=20fork/,=0A+=09'pg_amcheck=20index=20= s3.i1=20reports=20missing=20main=20relation=20fork');=0A+=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-c',=20'-p',=20= $port,=20'postgres',=20'-n',=20's3',=20'-i',=20'i2'=20],=0A+=09qr/.+/,=09= =09=09#=20Any=20non-empty=20error=20message=20is=20acceptable=0A+=09= 'pg_amcheck=20index=20s3.s2=20reports=20index=20corruption');=0A+=0A+=0A= +#=20Check=20that=20'-x'=20and=20'-X'=20work=20as=20expected.=20=20Since=20= only=20index=20corruption=0A+#=20(and=20not=20table=20corruption)=20= exists=20in=20s1,=20'-X'=20should=20give=20no=20errors,=20and=0A+#=20= '-x'=20should=20give=20errors=20about=20index=20corruption.=0A+#=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1'=20],=0A+=09qr/.+/,=09=09=09#=20Any=20= non-empty=20error=20message=20is=20acceptable=0A+=09'pg_amcheck=20over=20= tables=20and=20indexes=20in=20schema=20s1=20reports=20corruption');=0A+=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-X',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's1'=20],=0A+=09qr/^$/,=09=09=09#=20Empty=0A+=09= 'pg_amcheck=20over=20only=20tables=20in=20schema=20s1=20reports=20no=20= corruption');=0A+=0A+=0A+#=20Check=20that=20table=20corruption=20is=20= reported=20as=20expected,=20with=20or=20without=0A+#=20index=20checking=0A= +#=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20'-x',=20'-p',=20= $port,=20'postgres',=20'-n',=20's2'=20],=0A+=09qr/could=20not=20open=20= file/,=0A+=09'pg_amcheck=20over=20tables=20in=20schema=20s2=20reports=20= table=20corruption');=0A+=0A+$node->command_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'postgres',=20'-n',=20's2'=20],=0A+=09= qr/could=20not=20open=20file/,=0A+=09'pg_amcheck=20over=20tables=20and=20= indexes=20in=20schema=20s2=20reports=20table=20corruption');=0A+=0A+#=20= Check=20that=20no=20corruption=20is=20reported=20in=20schema=20s4=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-n',=20's4'=20],=0A+=09qr/^$/,=09=09=09#=20Empty=0A+=09= 'pg_amcheck=20over=20schema=20s4=20reports=20no=20corruption');=0A+=0A+#=20= Check=20that=20no=20corruption=20is=20reported=20if=20we=20exclude=20= corrupt=20schemas=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20= '-p',=20$port,=20'postgres',=20'-N',=20's1',=20'-N',=20's2',=20'-N',=20= 's3'=20],=0A+=09qr/^$/,=09=09=09#=20Empty=0A+=09'pg_amcheck=20excluding=20= corrupt=20schemas=20reports=20no=20corruption');=0A+=0A+#=20Check=20that=20= no=20corruption=20is=20reported=20if=20we=20exclude=20corrupt=20tables=0A= +$node->command_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-T',=20't1',=20'-T',=20't2'=20],=0A+=09qr/^$/,=09=09=09#=20= Empty=0A+=09'pg_amcheck=20excluding=20corrupt=20tables=20reports=20no=20= corruption');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/004_verify_heapam.pl=20= b/contrib/pg_amcheck/t/004_verify_heapam.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..1cc36b25b7=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/004_verify_heapam.pl=0A@@=20-0,0=20+1,489=20@@=0A= +use=20strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20= TestLib;=0A+=0A+use=20Test::More=20tests=20=3D>=2022;=0A+=0A+#=20This=20= regression=20test=20demonstrates=20that=20the=20pg_amcheck=20binary=20= supplied=20with=0A+#=20the=20pg_amcheck=20contrib=20module=20correctly=20= identifies=20specific=20kinds=20of=0A+#=20corruption=20within=20pages.=20= =20To=20test=20this,=20we=20need=20a=20mechanism=20to=20create=20corrupt=0A= +#=20pages=20with=20predictable,=20repeatable=20corruption.=20=20The=20= postgres=20backend=20cannot=0A+#=20be=20expected=20to=20help=20us=20with=20= this,=20as=20its=20design=20is=20not=20consistent=20with=20the=0A+#=20= goal=20of=20intentionally=20corrupting=20pages.=0A+#=0A+#=20Instead,=20= we=20create=20a=20table=20to=20corrupt,=20and=20with=20careful=20= consideration=20of=20how=0A+#=20postgresql=20lays=20out=20heap=20pages,=20= we=20seek=20to=20offsets=20within=20the=20page=20and=0A+#=20overwrite=20= deliberately=20chosen=20bytes=20with=20specific=20values=20calculated=20= to=0A+#=20corrupt=20the=20page=20in=20expected=20ways.=20=20We=20then=20= verify=20that=20pg_amcheck=20reports=0A+#=20the=20corruption,=20and=20= that=20it=20runs=20without=20crashing.=20=20Note=20that=20the=20backend=0A= +#=20cannot=20simply=20be=20started=20to=20run=20queries=20against=20the=20= corrupt=20table,=20as=20the=0A+#=20backend=20will=20crash,=20at=20least=20= for=20some=20of=20the=20corruption=20types=20we=20generate.=0A+#=0A+#=20= Autovacuum=20potentially=20touching=20the=20table=20in=20the=20= background=20makes=20the=20exact=0A+#=20behavior=20of=20this=20test=20= harder=20to=20reason=20about.=20=20We=20turn=20it=20off=20to=20keep=20= things=0A+#=20simpler.=20=20We=20use=20a=20"belt=20and=20suspenders"=20= approach,=20turning=20it=20off=20for=20the=0A+#=20system=20generally=20= in=20postgresql.conf,=20and=20turning=20it=20off=20specifically=20for=20= the=0A+#=20test=20table.=0A+#=0A+#=20This=20test=20depends=20on=20the=20= table=20being=20written=20to=20the=20heap=20file=20exactly=20as=20we=0A= +#=20expect=20it=20to=20be,=20so=20we=20take=20care=20to=20arrange=20the=20= columns=20of=20the=20table,=20and=0A+#=20insert=20rows=20of=20the=20= table,=20that=20give=20predictable=20sizes=20and=20locations=20within=0A= +#=20the=20table=20page.=0A+#=0A+#=20The=20HeapTupleHeaderData=20has=20= 23=20bytes=20of=20fixed=20size=20fields=20before=20the=20variable=0A+#=20= length=20t_bits[]=20array.=20=20We=20have=20exactly=203=20columns=20in=20= the=20table,=20so=20natts=20=3D=203,=0A+#=20t_bits=20is=201=20byte=20= long,=20and=20t_hoff=20=3D=20MAXALIGN(23=20+=201)=20=3D=2024.=0A+#=0A+#=20= We're=20not=20too=20fussy=20about=20which=20datatypes=20we=20use=20for=20= the=20test,=20but=20we=20do=20care=0A+#=20about=20some=20specific=20= properties.=20=20We'd=20like=20to=20test=20both=20fixed=20size=20and=0A= +#=20varlena=20types.=20=20We'd=20like=20some=20varlena=20data=20inline=20= and=20some=20toasted.=20=20And=0A+#=20we'd=20like=20the=20layout=20of=20= the=20table=20such=20that=20the=20datums=20land=20at=20predictable=0A+#=20= offsets=20within=20the=20tuple.=20=20We=20choose=20a=20structure=20= without=20padding=20on=20all=0A+#=20supported=20architectures:=0A+#=0A+#=20= =09a=20BIGINT=0A+#=09b=20TEXT=0A+#=09c=20TEXT=0A+#=0A+#=20We=20always=20= insert=20a=207-ascii=20character=20string=20into=20field=20'b',=20which=20= with=20a=0A+#=201-byte=20varlena=20header=20gives=20an=208=20byte=20= inline=20value.=20=20We=20always=20insert=20a=20long=0A+#=20text=20= string=20in=20field=20'c',=20long=20enough=20to=20force=20toast=20= storage.=0A+#=0A+#=20We=20choose=20to=20read=20and=20write=20binary=20= copies=20of=20our=20table's=20tuples,=20using=20perl's=0A+#=20pack()=20= and=20unpack()=20functions.=20=20Perl=20uses=20a=20packing=20code=20= system=20in=20which:=0A+#=0A+#=09L=20=3D=20"Unsigned=2032-bit=20Long",=0A= +#=09S=20=3D=20"Unsigned=2016-bit=20Short",=0A+#=09C=20=3D=20"Unsigned=20= 8-bit=20Octet",=0A+#=09c=20=3D=20"signed=208-bit=20octet",=0A+#=09q=20=3D=20= "signed=2064-bit=20quadword"=0A+#=0A+#=20Each=20tuple=20in=20our=20table=20= has=20a=20layout=20as=20follows:=0A+#=0A+#=20=20=20=20xx=20xx=20xx=20xx=20= =20=20=20=20=20=20=20=20=20=20=20t_xmin:=20xxxx=09=09offset=20=3D=200=09=09= L=0A+#=20=20=20=20xx=20xx=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20= t_xmax:=20xxxx=09=09offset=20=3D=204=09=09L=0A+#=20=20=20=20xx=20xx=20xx=20= xx=20=20=20=20=20=20=20=20=20=20t_field3:=20xxxx=09=09offset=20=3D=208=09= =09L=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20bi_hi:=20xx=09=09=09offset=20=3D=2012=09=09S=0A+#=20=20=20=20= xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20bi_lo:=20= xx=09=09=09offset=20=3D=2014=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20ip_posid:=20xx=09=09=09offset=20=3D=20= 16=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20= t_infomask2:=20xx=09=09=09offset=20=3D=2018=09=09S=0A+#=20=20=20=20xx=20= xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20t_infomask:=20xx=09=09=09= offset=20=3D=2020=09=09S=0A+#=20=20=20=20xx=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20t_hoff:=20x=09=09=09offset=20=3D=2022=09= =09C=0A+#=20=20=20=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20t_bits:=20x=09=09=09offset=20=3D=2023=09=09C=0A+#=20=20=20=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'a':=20xxxxxxxx=09offset=20= =3D=2024=09=09q=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20= =20'b':=20xxxxxxxx=09offset=20=3D=2032=09=09Cccccccc=0A+#=20=20=20=20xx=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'c':=20xxxxxxxx=09offset=20=3D=20= 40=09=09SSSS=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20= =20=20=20:=20xxxxxxxx=09=20...continued=09SSSS=0A+#=20=20=20=20xx=20xx=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20:=20= xx=20=20=20=20=20=20=09=20...continued=09S=0A+#=0A+#=20We=20could=20= choose=20to=20read=20and=20write=20columns=20'b'=20and=20'c'=20in=20= other=20ways,=20but=0A+#=20it=20is=20convenient=20enough=20to=20do=20it=20= this=20way.=20=20We=20define=20packing=20code=0A+#=20constants=20here,=20= where=20they=20can=20be=20compared=20easily=20against=20the=20layout.=0A= +=0A+use=20constant=20HEAPTUPLE_PACK_CODE=20=3D>=20= 'LLLSSSSSCCqCcccccccSSSSSSSSS';=0A+use=20constant=20= HEAPTUPLE_PACK_LENGTH=20=3D>=2058;=20=20=20=20=20#=20Total=20size=0A+=0A= +#=20Read=20a=20tuple=20of=20our=20table=20from=20a=20heap=20page.=0A+#=0A= +#=20Takes=20an=20open=20filehandle=20to=20the=20heap=20file,=20and=20= the=20offset=20of=20the=20tuple.=0A+#=0A+#=20Rather=20than=20returning=20= the=20binary=20data=20from=20the=20file,=20unpacks=20the=20data=20into=20= a=0A+#=20perl=20hash=20with=20named=20fields.=20=20These=20fields=20= exactly=20match=20the=20ones=20understood=0A+#=20by=20write_tuple(),=20= below.=20=20Returns=20a=20reference=20to=20this=20hash.=0A+#=0A+sub=20= read_tuple=20($$)=0A+{=0A+=09my=20($fh,=20$offset)=20=3D=20@_;=0A+=09my=20= ($buffer,=20%tup);=0A+=09seek($fh,=20$offset,=200);=0A+=09sysread($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=0A+=09@_=20=3D=20= unpack(HEAPTUPLE_PACK_CODE,=20$buffer);=0A+=09%tup=20=3D=20(t_xmin=20=3D>=20= shift,=0A+=09=09=09t_xmax=20=3D>=20shift,=0A+=09=09=09t_field3=20=3D>=20= shift,=0A+=09=09=09bi_hi=20=3D>=20shift,=0A+=09=09=09bi_lo=20=3D>=20= shift,=0A+=09=09=09ip_posid=20=3D>=20shift,=0A+=09=09=09t_infomask2=20=3D>= =20shift,=0A+=09=09=09t_infomask=20=3D>=20shift,=0A+=09=09=09t_hoff=20=3D>= =20shift,=0A+=09=09=09t_bits=20=3D>=20shift,=0A+=09=09=09a=20=3D>=20= shift,=0A+=09=09=09b_header=20=3D>=20shift,=0A+=09=09=09b_body1=20=3D>=20= shift,=0A+=09=09=09b_body2=20=3D>=20shift,=0A+=09=09=09b_body3=20=3D>=20= shift,=0A+=09=09=09b_body4=20=3D>=20shift,=0A+=09=09=09b_body5=20=3D>=20= shift,=0A+=09=09=09b_body6=20=3D>=20shift,=0A+=09=09=09b_body7=20=3D>=20= shift,=0A+=09=09=09c1=20=3D>=20shift,=0A+=09=09=09c2=20=3D>=20shift,=0A+=09= =09=09c3=20=3D>=20shift,=0A+=09=09=09c4=20=3D>=20shift,=0A+=09=09=09c5=20= =3D>=20shift,=0A+=09=09=09c6=20=3D>=20shift,=0A+=09=09=09c7=20=3D>=20= shift,=0A+=09=09=09c8=20=3D>=20shift,=0A+=09=09=09c9=20=3D>=20shift);=0A= +=09#=20Stitch=20together=20the=20text=20for=20column=20'b'=0A+=09= $tup{b}=20=3D=20join('',=20map=20{=20chr($tup{"b_body$_"})=20}=20= (1..7));=0A+=09return=20\%tup;=0A+}=0A+=0A+#=20Write=20a=20tuple=20of=20= our=20table=20to=20a=20heap=20page.=0A+#=0A+#=20Takes=20an=20open=20= filehandle=20to=20the=20heap=20file,=20the=20offset=20of=20the=20tuple,=20= and=20a=0A+#=20reference=20to=20a=20hash=20with=20the=20tuple=20values,=20= as=20returned=20by=20read_tuple().=0A+#=20Writes=20the=20tuple=20fields=20= from=20the=20hash=20into=20the=20heap=20file.=0A+#=0A+#=20The=20purpose=20= of=20this=20function=20is=20to=20write=20a=20tuple=20back=20to=20disk=20= with=20some=0A+#=20subset=20of=20fields=20modified.=20=20The=20function=20= does=20no=20error=20checking.=20=20Use=0A+#=20cautiously.=0A+#=0A+sub=20= write_tuple($$$)=0A+{=0A+=09my=20($fh,=20$offset,=20$tup)=20=3D=20@_;=0A= +=09my=20$buffer=20=3D=20pack(HEAPTUPLE_PACK_CODE,=0A+=09=09=09=09=09= $tup->{t_xmin},=0A+=09=09=09=09=09$tup->{t_xmax},=0A+=09=09=09=09=09= $tup->{t_field3},=0A+=09=09=09=09=09$tup->{bi_hi},=0A+=09=09=09=09=09= $tup->{bi_lo},=0A+=09=09=09=09=09$tup->{ip_posid},=0A+=09=09=09=09=09= $tup->{t_infomask2},=0A+=09=09=09=09=09$tup->{t_infomask},=0A+=09=09=09=09= =09$tup->{t_hoff},=0A+=09=09=09=09=09$tup->{t_bits},=0A+=09=09=09=09=09= $tup->{a},=0A+=09=09=09=09=09$tup->{b_header},=0A+=09=09=09=09=09= $tup->{b_body1},=0A+=09=09=09=09=09$tup->{b_body2},=0A+=09=09=09=09=09= $tup->{b_body3},=0A+=09=09=09=09=09$tup->{b_body4},=0A+=09=09=09=09=09= $tup->{b_body5},=0A+=09=09=09=09=09$tup->{b_body6},=0A+=09=09=09=09=09= $tup->{b_body7},=0A+=09=09=09=09=09$tup->{c1},=0A+=09=09=09=09=09= $tup->{c2},=0A+=09=09=09=09=09$tup->{c3},=0A+=09=09=09=09=09$tup->{c4},=0A= +=09=09=09=09=09$tup->{c5},=0A+=09=09=09=09=09$tup->{c6},=0A+=09=09=09=09= =09$tup->{c7},=0A+=09=09=09=09=09$tup->{c8},=0A+=09=09=09=09=09= $tup->{c9});=0A+=09seek($fh,=20$offset,=200);=0A+=09syswrite($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=09return;=0A+}=0A+=0A+#=20Set=20= umask=20so=20test=20directories=20and=20files=20are=20created=20with=20= default=20permissions=0A+umask(0077);=0A+=0A+#=20Set=20up=20the=20node.=20= =20Once=20we=20create=20and=20corrupt=20the=20table,=0A+#=20autovacuum=20= workers=20visiting=20the=20table=20could=20crash=20the=20backend.=0A+#=20= Disable=20autovacuum=20so=20that=20won't=20happen.=0A+my=20$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A= +$node->append_conf('postgresql.conf',=20'autovacuum=3Doff');=0A+=0A+#=20= Start=20the=20node=20and=20load=20the=20extensions.=20=20We=20depend=20= on=20both=0A+#=20amcheck=20and=20pageinspect=20for=20this=20test.=0A= +$node->start;=0A+my=20$port=20=3D=20$node->port;=0A+my=20$pgdata=20=3D=20= $node->data_dir;=0A+$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20= amcheck");=0A+$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20= pageinspect");=0A+=0A+#=20Get=20a=20non-zero=20datfrozenxid=0A= +$node->safe_psql('postgres',=20qq(VACUUM=20FREEZE));=0A+=0A+#=20Create=20= the=20test=20table=20with=20precisely=20the=20schema=20that=20our=20= corruption=20function=0A+#=20expects.=0A+$node->safe_psql(=0A+=09= 'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20public.test=20(a=20BIGINT,=20= b=20TEXT,=20c=20TEXT);=0A+=09=09ALTER=20TABLE=20public.test=20SET=20= (autovacuum_enabled=3Dfalse);=0A+=09=09ALTER=20TABLE=20public.test=20= ALTER=20COLUMN=20c=20SET=20STORAGE=20EXTERNAL;=0A+=09=09CREATE=20INDEX=20= test_idx=20ON=20public.test(a,=20b);=0A+=09));=0A+=0A+#=20We=20want=20(0=20= <=20datfrozenxid=20<=20test.relfrozenxid).=20=20To=20achieve=20this,=20= we=20freeze=0A+#=20an=20otherwise=20unused=20table,=20public.junk,=20= prior=20to=20inserting=20data=20and=20freezing=0A+#=20public.test=0A= +$node->safe_psql(=0A+=09'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20= public.junk=20AS=20SELECT=20'junk'::TEXT=20AS=20junk_column;=0A+=09=09= ALTER=20TABLE=20public.junk=20SET=20(autovacuum_enabled=3Dfalse);=0A+=09=09= VACUUM=20FREEZE=20public.junk=0A+=09));=0A+=0A+my=20$rel=20=3D=20= $node->safe_psql('postgres',=20qq(SELECT=20= pg_relation_filepath('public.test')));=0A+my=20$relpath=20=3D=20= "$pgdata/$rel";=0A+=0A+#=20Insert=20data=20and=20freeze=20public.test=0A= +use=20constant=20ROWCOUNT=20=3D>=2016;=0A+$node->safe_psql('postgres',=20= qq(=0A+=09INSERT=20INTO=20public.test=20(a,=20b,=20c)=0A+=09=09VALUES=20= (=0A+=09=09=0912345678,=0A+=09=09=09'abcdefg',=0A+=09=09=09repeat('w',=20= 10000)=0A+=09=09);=0A+=09VACUUM=20FREEZE=20public.test=0A+=09))=20for=20= (1..ROWCOUNT);=0A+=0A+my=20$relfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A+=09q(select=20relfrozenxid=20from=20= pg_class=20where=20relname=20=3D=20'test'));=0A+my=20$datfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A+=09q(select=20datfrozenxid=20from=20= pg_database=20where=20datname=20=3D=20'postgres'));=0A+=0A+#=20Find=20= where=20each=20of=20the=20tuples=20is=20located=20on=20the=20page.=0A+my=20= @lp_off;=0A+for=20my=20$tup=20(0..ROWCOUNT-1)=0A+{=0A+=09push=20= (@lp_off,=20$node->safe_psql('postgres',=20qq(=0A+select=20lp_off=20from=20= heap_page_items(get_raw_page('test',=20'main',=200))=0A+=09offset=20$tup=20= limit=201)));=0A+}=0A+=0A+#=20Check=20that=20pg_amcheck=20runs=20against=20= the=20uncorrupted=20table=20without=20error.=0A= +$node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A+=09=09= =09=09=20=20'pg_amcheck=20test=20table,=20prior=20to=20corruption');=0A+=0A= +#=20Check=20that=20pg_amcheck=20runs=20against=20the=20uncorrupted=20= table=20and=20index=20without=20error.=0A= +$node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A+=09=09= =09=09=20=20'pg_amcheck=20test=20table=20and=20index,=20prior=20to=20= corruption');=0A+=0A+$node->stop;=0A+=0A+#=20Sanity=20check=20that=20our=20= 'test'=20table=20has=20a=20relfrozenxid=20newer=20than=20the=0A+#=20= datfrozenxid=20for=20the=20database,=20and=20that=20the=20datfrozenxid=20= is=20greater=20than=20the=0A+#=20first=20normal=20xid.=20=20We=20rely=20= on=20these=20invariants=20in=20some=20of=20our=20tests.=0A+if=20= ($datfrozenxid=20<=3D=203=20||=20$datfrozenxid=20>=3D=20$relfrozenxid)=0A= +{=0A+=09fail('Xid=20thresholds=20not=20as=20expected');=0A+=09= $node->clean_node;=0A+=09exit;=0A+}=0A+=0A+#=20Some=20#define=20= constants=20from=20access/htup_details.h=20for=20use=20while=20= corrupting.=0A+use=20constant=20HEAP_HASNULL=20=20=20=20=20=20=20=20=20=20= =20=20=3D>=200x0001;=0A+use=20constant=20HEAP_XMAX_LOCK_ONLY=20=20=20=20=20= =3D>=200x0080;=0A+use=20constant=20HEAP_XMIN_COMMITTED=20=20=20=20=20=3D>=20= 0x0100;=0A+use=20constant=20HEAP_XMIN_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0200;=0A+use=20constant=20HEAP_XMAX_COMMITTED=20=20=20=20=20=3D>=20= 0x0400;=0A+use=20constant=20HEAP_XMAX_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0800;=0A+use=20constant=20HEAP_NATTS_MASK=20=20=20=20=20=20=20=20=20=3D>= =200x07FF;=0A+use=20constant=20HEAP_XMAX_IS_MULTI=20=20=20=20=20=20=3D>=20= 0x1000;=0A+use=20constant=20HEAP_KEYS_UPDATED=20=20=20=20=20=20=20=3D>=20= 0x2000;=0A+=0A+#=20Helper=20functions=0A+sub=20header=0A+{=0A+=09my=20= ($blkno,=20$offnum,=20$attnum)=20=3D=20@_;=0A+=09= qr/\(relname=3Dtest,blkno=3D$blkno,offnum=3D$offnum,attnum=3D$attnum\)\s+/= ms;=0A+}=0A+=0A+#=20Corrupt=20the=20tuples,=20one=20type=20of=20= corruption=20per=20tuple.=20=20Some=20types=20of=0A+#=20corruption=20= cause=20verify_heapam=20to=20skip=20to=20the=20next=20tuple=20without=0A= +#=20performing=20any=20remaining=20checks,=20so=20we=20can't=20exercise=20= the=20system=20properly=20if=0A+#=20we=20focus=20all=20our=20corruption=20= on=20a=20single=20tuple.=0A+#=0A+my=20@expected;=0A+my=20$file;=0A= +open($file,=20'+<',=20$relpath);=0A+binmode=20$file;=0A+=0A+for=20(my=20= $tupidx=20=3D=200;=20$tupidx=20<=20ROWCOUNT;=20$tupidx++)=0A+{=0A+=09my=20= $offnum=20=3D=20$tupidx=20+=201;=20=20#=20offnum=20is=201-based,=20not=20= zero-based=0A+=09my=20$offset=20=3D=20$lp_off[$tupidx];=0A+=09my=20$tup=20= =3D=20read_tuple($file,=20$offset);=0A+=0A+=09#=20Sanity-check=20that=20= the=20data=20appears=20on=20the=20page=20where=20we=20expect.=0A+=09if=20= ($tup->{a}=20ne=20'12345678'=20||=20$tup->{b}=20ne=20'abcdefg')=0A+=09{=0A= +=09=09fail('Page=20layout=20differs=20from=20our=20expectations');=0A+=09= =09$node->clean_node;=0A+=09=09exit;=0A+=09}=0A+=0A+=09my=20$header=20=3D=20= header(0,=20$offnum,=20'');=0A+=09if=20($offnum=20=3D=3D=201)=0A+=09{=0A= +=09=09#=20Corruptly=20set=20xmin=20<=20relfrozenxid=0A+=09=09my=20$xmin=20= =3D=20$relfrozenxid=20-=201;=0A+=09=09$tup->{t_xmin}=20=3D=20$xmin;=0A+=09= =09$tup->{t_infomask}=20&=3D=20~HEAP_XMIN_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20&=3D=20~HEAP_XMIN_INVALID;=0A+=0A+=09=09#=20= Expected=20corruption=20report=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}xmin=20$xmin=20precedes=20relation=20freeze=20threshold=20= 0:\d+/;=0A+=09}=0A+=09if=20($offnum=20=3D=3D=202)=0A+=09{=0A+=09=09#=20= Corruptly=20set=20xmin=20<=20datfrozenxid=0A+=09=09my=20$xmin=20=3D=203;=0A= +=09=09$tup->{t_xmin}=20=3D=20$xmin;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmin=20$xmin=20precedes=20oldest=20valid=20transaction=20ID=20= 0:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=203)=0A+=09{=0A+=09=09#=20= Corruptly=20set=20xmin=20<=20datfrozenxid,=20further=20back,=20noting=20= circularity=0A+=09=09#=20of=20xid=20comparison.=20=20For=20a=20new=20= cluster=20with=20epoch=20=3D=200,=20the=20corrupt=0A+=09=09#=20xmin=20= will=20be=20interpreted=20as=20in=20the=20future=0A+=09=09$tup->{t_xmin}=20= =3D=204026531839;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmin=204026531839=20equals=20or=20exceeds=20next=20valid=20= transaction=20ID=200:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=204)=0A= +=09{=0A+=09=09#=20Corruptly=20set=20xmax=20<=20relminmxid;=0A+=09=09= $tup->{t_xmax}=20=3D=204026531839;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMAX_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmax=204026531839=20equals=20or=20exceeds=20next=20valid=20= transaction=20ID=200:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=205)=0A= +=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20but=20keep=20it=20= aligned=20properly=0A+=09=09$tup->{t_hoff}=20+=3D=20128;=0A+=0A+=09=09= push=20@expected,=0A+=09=09=09qr/${$header}data=20begins=20at=20offset=20= 152=20beyond=20the=20tuple=20length=2058/,=0A+=09=09=09= qr/${$header}tuple=20data=20should=20begin=20at=20byte=2024,=20but=20= actually=20begins=20at=20byte=20152=20\(3=20attributes,=20no=20nulls\)/;=0A= +=09}=0A+=09elsif=20($offnum=20=3D=3D=206)=0A+=09{=0A+=09=09#=20Corrupt=20= the=20tuple=20t_hoff,=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20+=3D=20= 3;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 27=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=207)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20= underflow=20but=20correct=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=20= 8;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 16=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=208)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20= underflow=20and=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=203;=0A= +=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 21=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=209)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20to=20look=20= like=20it=20has=20lots=20of=20attributes,=20not=20just=203=0A+=09=09= $tup->{t_infomask2}=20|=3D=20HEAP_NATTS_MASK;=0A+=0A+=09=09push=20= @expected,=0A+=09=09=09qr/${$header}number=20of=20attributes=202047=20= exceeds=20maximum=20expected=20for=20table=203/;=0A+=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2010)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20to=20= look=20like=20it=20has=20lots=20of=20attributes,=20some=20of=0A+=09=09#=20= them=20null.=20=20This=20falsely=20creates=20the=20impression=20that=20= the=20t_bits=0A+=09=09#=20array=20is=20longer=20than=20just=20one=20= byte,=20but=20t_hoff=20still=20says=20otherwise.=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09$tup->{t_infomask2}=20= |=3D=20HEAP_NATTS_MASK;=0A+=09=09$tup->{t_bits}=20=3D=200xAA;=0A+=0A+=09=09= push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20should=20begin=20= at=20byte=20280,=20but=20actually=20begins=20at=20byte=2024=20\(2047=20= attributes,=20has=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=20= 11)=0A+=09{=0A+=09=09#=20Same=20as=20above,=20but=20this=20time=20t_hoff=20= plays=20along=0A+=09=09$tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09= $tup->{t_infomask2}=20|=3D=20(HEAP_NATTS_MASK=20&=200x40);=0A+=09=09= $tup->{t_bits}=20=3D=200xAA;=0A+=09=09$tup->{t_hoff}=20=3D=2032;=0A+=0A+=09= =09push=20@expected,=0A+=09=09=09qr/${$header}number=20of=20attributes=20= 67=20exceeds=20maximum=20expected=20for=20table=203/;=0A+=09}=0A+=09= elsif=20($offnum=20=3D=3D=2012)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= bits=20in=20column=20'b'=201-byte=20varlena=20header=0A+=09=09= $tup->{b_header}=20=3D=200x80;=0A+=0A+=09=09$header=20=3D=20header(0,=20= $offnum,=201);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}attribute=201=20with=20length=204294967295=20ends=20at=20= offset=20416848000=20beyond=20total=20tuple=20length=2058/;=0A+=09}=0A+=09= elsif=20($offnum=20=3D=3D=2013)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= bits=20in=20column=20'c'=20toast=20pointer=0A+=09=09$tup->{c6}=20=3D=20= 41;=0A+=09=09$tup->{c7}=20=3D=2041;=0A+=0A+=09=09$header=20=3D=20= header(0,=20$offnum,=202);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}final=20toast=20chunk=20number=200=20differs=20from=20= expected=20value=206/,=0A+=09=09=09qr/${header}toasted=20value=20for=20= attribute=202=20missing=20from=20toast=20table/;=0A+=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2014)=0A+=09{=0A+=09=09#=20Set=20both=20= HEAP_XMAX_LOCK_ONLY=20and=20HEAP_KEYS_UPDATED=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_LOCK_ONLY;=0A+=09=09= $tup->{t_infomask2}=20|=3D=20HEAP_KEYS_UPDATED;=0A+=0A+=09=09push=20= @expected,=0A+=09=09=09qr/${header}tuple=20is=20marked=20as=20only=20= locked,=20but=20also=20claims=20key=20columns=20were=20updated/;=0A+=09}=0A= +=09elsif=20($offnum=20=3D=3D=2015)=0A+=09{=0A+=09=09#=20Set=20both=20= HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_IS_MULTI;=0A+=09=09$tup->{t_xmax}=20= =3D=204;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}multitransaction=20ID=204=20equals=20or=20exceeds=20next=20= valid=20multitransaction=20ID=201/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D= =2016)=09#=20Last=20offnum=20must=20equal=20ROWCOUNT=0A+=09{=0A+=09=09#=20= Set=20both=20HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_IS_MULTI;=0A+=09=09$tup->{t_xmax}=20= =3D=204000000000;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}multitransaction=20ID=204000000000=20precedes=20relation=20= minimum=20multitransaction=20ID=20threshold=201/;=0A+=09}=0A+=09= write_tuple($file,=20$offset,=20$tup);=0A+}=0A+close($file);=0A= +$node->start;=0A+=0A+#=20Run=20pg_amcheck=20against=20the=20corrupt=20= table=20with=20epoch=3D0,=20comparing=20actual=0A+#=20corruption=20= messages=20against=20the=20expected=20messages=0A= +$node->command_checks_all(=0A+=09['pg_amcheck',=20'--check-toast',=20= '--skip-indexes',=20'-p',=20$port,=20'postgres'],=0A+=090,=0A+=09[=20= @expected=20],=0A+=09[=20qr/^$/=20],=0A+=09'Expected=20corruption=20= message=20output');=0A+=0A+$node->teardown_node;=0A+$node->clean_node;=0A= diff=20--git=20a/contrib/pg_amcheck/t/005_opclass_damage.pl=20= b/contrib/pg_amcheck/t/005_opclass_damage.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..fdbb1ea402=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/005_opclass_damage.pl=0A@@=20-0,0=20+1,52=20@@=0A= +#=20This=20regression=20test=20checks=20the=20behavior=20of=20the=20= btree=20validation=20in=20the=0A+#=20presence=20of=20breaking=20sort=20= order=20changes.=0A+#=0A+use=20strict;=0A+use=20warnings;=0A+use=20= PostgresNode;=0A+use=20TestLib;=0A+use=20Test::More=20tests=20=3D>=206;=0A= +=0A+my=20$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+=0A+#=20Create=20a=20custom=20operator=20class=20and=20= an=20index=20which=20uses=20it.=0A+$node->safe_psql('postgres',=20q(=0A+=09= CREATE=20EXTENSION=20amcheck;=0A+=0A+=09CREATE=20FUNCTION=20int4_asc_cmp=20= (a=20int4,=20b=20int4)=20RETURNS=20int=20LANGUAGE=20sql=20AS=20$$=0A+=09=09= SELECT=20CASE=20WHEN=20$1=20=3D=20$2=20THEN=200=20WHEN=20$1=20>=20$2=20= THEN=201=20ELSE=20-1=20END;=20$$;=0A+=0A+=09CREATE=20OPERATOR=20CLASS=20= int4_fickle_ops=20FOR=20TYPE=20int4=20USING=20btree=20AS=0A+=09=20=20=20=20= OPERATOR=201=20<=20(int4,=20int4),=20OPERATOR=202=20<=3D=20(int4,=20= int4),=0A+=09=20=20=20=20OPERATOR=203=20=3D=20(int4,=20int4),=20OPERATOR=20= 4=20>=3D=20(int4,=20int4),=0A+=09=20=20=20=20OPERATOR=205=20>=20(int4,=20= int4),=20FUNCTION=201=20int4_asc_cmp(int4,=20int4);=0A+=0A+=09CREATE=20= TABLE=20int4tbl=20(i=20int4);=0A+=09INSERT=20INTO=20int4tbl=20(SELECT=20= *=20FROM=20generate_series(1,1000)=20gs);=0A+=09CREATE=20INDEX=20= fickleidx=20ON=20int4tbl=20USING=20btree=20(i=20int4_fickle_ops);=0A+));=0A= +=0A+#=20We=20have=20not=20yet=20broken=20the=20index,=20so=20we=20= should=20get=20no=20corruption=0A+$node->command_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$node->port,=20'postgres'=20],=0A+=09qr/^$/,=0A+=09= 'pg_amcheck=20all=20schemas,=20tables=20and=20indexes=20reports=20no=20= corruption');=0A+=0A+#=20Change=20the=20operator=20class=20to=20use=20a=20= function=20which=20sorts=20in=20a=20different=0A+#=20order=20to=20= corrupt=20the=20btree=20index=0A+$node->safe_psql('postgres',=20q(=0A+=09= CREATE=20FUNCTION=20int4_desc_cmp=20(int4,=20int4)=20RETURNS=20int=20= LANGUAGE=20sql=20AS=20$$=0A+=09=09SELECT=20CASE=20WHEN=20$1=20=3D=20$2=20= THEN=200=20WHEN=20$1=20>=20$2=20THEN=20-1=20ELSE=201=20END;=20$$;=0A+=09= UPDATE=20pg_catalog.pg_amproc=0A+=09=09SET=20amproc=20=3D=20= 'int4_desc_cmp'::regproc=0A+=09=09WHERE=20amproc=20=3D=20= 'int4_asc_cmp'::regproc=0A+));=0A+=0A+#=20Index=20corruption=20should=20= now=20be=20reported=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20= '-x',=20'-p',=20$node->port,=20'postgres'=20],=0A+=09qr/item=20order=20= invariant=20violated=20for=20index=20"fickleidx"/,=0A+=09'pg_amcheck=20= all=20schemas,=20tables=20and=20indexes=20reports=20fickleidx=20= corruption'=0A+);=0Adiff=20--git=20a/doc/src/sgml/contrib.sgml=20= b/doc/src/sgml/contrib.sgml=0Aindex=204e833d79ef..1efca8adc4=20100644=0A= ---=20a/doc/src/sgml/contrib.sgml=0A+++=20b/doc/src/sgml/contrib.sgml=0A= @@=20-119,6=20+119,7=20@@=20CREATE=20EXTENSION=20= module_name;=0A=20=20&oldsnapshot;=0A=20=20= &pageinspect;=0A=20=20&passwordcheck;=0A+=20&pgamcheck;=0A=20=20= &pgbuffercache;=0A=20=20&pgcrypto;=0A=20=20&pgfreespacemap;=0Adiff=20= --git=20a/doc/src/sgml/filelist.sgml=20b/doc/src/sgml/filelist.sgml=0A= index=2038e8aa0bbf..a4e1b28b38=20100644=0A---=20= a/doc/src/sgml/filelist.sgml=0A+++=20b/doc/src/sgml/filelist.sgml=0A@@=20= -133,6=20+133,7=20@@=0A=20=0A=20=0A=20=0A+=0A=20=0A=20=0A=20=0Adiff=20--git=20a/doc/src/sgml/pgamcheck.sgml=20= b/doc/src/sgml/pgamcheck.sgml=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..fc36447dda=0A---=20/dev/null=0A+++=20= b/doc/src/sgml/pgamcheck.sgml=0A@@=20-0,0=20+1,228=20@@=0A+=0A+=0A+=0A+=20pg_amcheck=0A+=0A+=20= =0A+=20=20pg_amcheck=0A= +=20=0A+=0A+=20=0A+=20=20The=20= pg_amcheck=20module=20provides=20a=20command=20line=20= interface=0A+=20=20to=20the=20=20corruption=20= checking=20functionality.=0A+=20=0A+=0A+=20=0A+=20=20= pg_amcheck=20is=20a=20regular=0A+=20=20= PostgreSQL=20client=20application.=20You=20= can=20perform=0A+=20=20corruption=20checks=20from=20any=20remote=20host=20= that=20has=20access=20to=20the=20database=0A+=20=20connecting=20as=20a=20= user=20with=20sufficient=20privileges=20to=20check=20tables=20and=20= indexes.=0A+=20=20Currently,=20this=20requires=20execute=20privileges=20= on=20's=0A+=20=20= bt_index_parent_check=20and=20= verify_heapam=0A+=20=20functions,=20and=20on=20= having=20privileges=20to=20access=20the=20relations=20being=20checked.=0A= +=20=0A+=0A+=0A+pg_amcheck=20[OPTION]...=20[DBNAME=20= [USERNAME]]=0A+=20=20General=20options:=0A+=20=20=20=20-V,=20--version=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20output=20version=20= information,=20then=20exit=0A+=20=20=20=20-?,=20--help=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20show=20this=20help,=20then=20exit=0A= +=20=20=20=20-s,=20--strict-names=20=20=20=20=20=20=20=20=20=20=20= require=20include=20patterns=20to=20match=20at=20least=20one=20entity=20= each=0A+=20=20=20=20-o,=20--on-error-stop=20=20=20=20=20=20=20=20=20=20= stop=20checking=20at=20end=20of=20first=20corrupt=20page=0A+=0A+=20=20= Schema=20checking=20options:=0A+=20=20=20=20-n,=20--schema=3DPATTERN=20=20= =20=20=20=20=20=20=20check=20relations=20in=20the=20specified=20= schema(s)=20only=0A+=20=20=20=20-N,=20--exclude-schema=3DPATTERN=20do=20= NOT=20check=20relations=20in=20the=20specified=20schema(s)=0A+=0A+=20=20= Table=20checking=20options:=0A+=20=20=20=20-t,=20--table=3DPATTERN=20=20=20= =20=20=20=20=20=20=20check=20the=20specified=20table(s)=20only=0A+=20=20=20= =20-T,=20--exclude-table=3DPATTERN=20=20do=20NOT=20check=20the=20= specified=20table(s)=0A+=20=20=20=20-b,=20--startblock=20=20=20=20=20=20=20= =20=20=20=20=20=20begin=20checking=20table(s)=20at=20the=20given=20= starting=20block=20number=0A+=20=20=20=20-e,=20--endblock=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20check=20table(s)=20only=20up=20to=20the=20= given=20ending=20block=20number=0A+=20=20=20=20-f,=20--skip-all-frozen=20= =20=20=20=20=20=20=20do=20NOT=20check=20blocks=20marked=20as=20= all-frozen=0A+=20=20=20=20-v,=20--skip-all-visible=20=20=20=20=20=20=20= do=20NOT=20check=20blocks=20marked=20as=20all-visible=0A+=0A+=20=20TOAST=20= table=20checking=20options:=0A+=20=20=20=20-z,=20--check-toast=20=20=20=20= =20=20=20=20=20=20=20=20check=20associated=20toast=20tables=20and=20= toast=20indexes=0A+=20=20=20=20-Z,=20--skip-toast=20=20=20=20=20=20=20=20= =20=20=20=20=20do=20NOT=20check=20associated=20toast=20tables=20and=20= toast=20indexes=0A+=20=20=20=20-B,=20--toast-startblock=20=20=20=20=20=20= =20begin=20checking=20toast=20table(s)=20at=20the=20given=20starting=20= block=0A+=20=20=20=20-E,=20--toast-endblock=20=20=20=20=20=20=20=20=20= check=20toast=20table(s)=20only=20up=20to=20the=20given=20ending=20block=0A= +=0A+=20=20Index=20checking=20options:=0A+=20=20=20=20-x,=20= --check-indexes=20=20=20=20=20=20=20=20=20=20check=20btree=20indexes=20= associated=20with=20tables=20being=20checked=0A+=20=20=20=20-X,=20= --skip-indexes=20=20=20=20=20=20=20=20=20=20=20do=20NOT=20check=20any=20= btree=20indexes=0A+=20=20=20=20-i,=20--index=3DPATTERN=20=20=20=20=20=20=20= =20=20=20check=20the=20specified=20index(es)=20only=0A+=20=20=20=20-I,=20= --exclude-index=3DPATTERN=20=20do=20NOT=20check=20the=20specified=20= index(es)=0A+=20=20=20=20-c,=20--check-corrupt=20=20=20=20=20=20=20=20=20= =20check=20indexes=20even=20if=20their=20associated=20table=20is=20= corrupt=0A+=20=20=20=20-C,=20--skip-corrupt=20=20=20=20=20=20=20=20=20=20= =20do=20NOT=20check=20indexes=20if=20their=20associated=20table=20is=20= corrupt=0A+=20=20=20=20-a,=20--heapallindexed=20=20=20=20=20=20=20=20=20= check=20index=20tuples=20against=20the=20table=20tuples=0A+=20=20=20=20= -A,=20--no-heapallindexed=20=20=20=20=20=20do=20NOT=20check=20index=20= tuples=20against=20the=20table=20tuples=0A+=20=20=20=20-r,=20= --rootdescend=20=20=20=20=20=20=20=20=20=20=20=20search=20from=20the=20= root=20page=20for=20each=20index=20tuple=0A+=20=20=20=20-R,=20= --no-rootdescend=20=20=20=20=20=20=20=20=20do=20NOT=20search=20from=20= the=20root=20page=20for=20each=20index=20tuple=0A+=0A+=20=20Connection=20= options:=0A+=20=20=20=20-d,=20--dbname=3DDBNAME=20=20=20=20=20=20=20=20=20= =20database=20name=20to=20connect=20to=0A+=20=20=20=20-h,=20= --host=3DHOSTNAME=20=20=20=20=20=20=20=20=20=20database=20server=20host=20= or=20socket=20directory=0A+=20=20=20=20-p,=20--port=3DPORT=20=20=20=20=20= =20=20=20=20=20=20=20=20=20database=20server=20port=0A+=20=20=20=20-U,=20= --username=3DUSERNAME=20=20=20=20=20=20database=20user=20name=0A+=20=20=20= =20-w,=20--no-password=20=20=20=20=20=20=20=20=20=20=20=20never=20prompt=20= for=20password=0A+=20=20=20=20-W,=20--password=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20force=20password=20prompt=20(should=20happen=20= automatically)=0A+=0A+=0A+=20=0A+=20=20= Options=0A+=0A+=20=20=0A+=20=20=20To=20specify=20= which=20database=20server=20pg_amcheck=20= should=0A+=20=20=20contact,=20use=20the=20command=20line=20options=20= =20or=0A+=20=20=20=20and=20= =20or=0A+=20=20=20.=20The=20= default=20host=20is=20the=20local=20host=0A+=20=20=20or=20whatever=20= your=20PGHOST=20environment=20variable=20specifies.=0A+=20= =20=20Similarly,=20the=20default=20port=20is=20indicated=20by=20the=20= PGPORT=0A+=20=20=20environment=20variable=20or,=20failing=20= that,=20by=20the=20compiled-in=20default.=0A+=20=20=0A+=20=0A+=20=20= =0A+=20=20=20Like=20any=20other=20= PostgreSQL=20client=20application,=0A+=20=20=20= pg_amcheck=20will=20by=20default=20connect=20= with=20the=0A+=20=20=20database=20user=20name=20that=20is=20equal=20to=20= the=20current=20operating=20system=20user=20name.=0A+=20=20=20To=20= override=20this,=20either=20specify=20the=20=20option=20= or=20set=20the=0A+=20=20=20environment=20variable=20= PGUSER.=20Remember=20that=0A+=20=20=20= pg_amcheck=20connections=20are=20subject=20to=20= the=20normal=0A+=20=20=20client=20authentication=20mechanisms=20(which=20= are=20described=20in=20).=0A+=20=20=0A+=20=0A+=20=20= =0A+=20=20=20To=20restrict=20checking=20of=20tables=20and=20= indexes=20to=20specific=20schemas,=20specify=20the=0A+=20=20=20= =20or=20=20option=20with=20a=20= pattern.=0A+=20=20=20To=20exclude=20checking=20of=20tables=20and=20= indexes=20within=20specific=20schemas,=20specify=0A+=20=20=20the=20= =20or=20=20option=20= with=0A+=20=20=20a=20pattern.=0A+=20=20=0A+=20=0A+=20=20=0A= +=20=20=20To=20specify=20which=20tables=20are=20checked,=20specify=20the=0A= +=20=20=20=20or=20=20option=20= with=20a=20pattern.=0A+=20=20=20To=20exclude=20checking=20of=20tables,=20= specify=20the=0A+=20=20=20=20or=20= =20option=20with=20a=0A+=20=20=20= pattern.=0A+=20=20=0A+=20=0A+=20=20=0A+=20=20=20To=20check=20= indexes=20associated=20with=20checked=20tables,=20specify=20the=0A+=20=20= =20=20or=20=20option.=20= =20Only=0A+=20=20=20indexes=20on=20tables=20which=20are=20being=20= checked=20will=20themselves=20be=20checked.=20=20To=0A+=20=20=20check=20= all=20indexes=20in=20a=20database,=20all=20tables=20on=20which=20the=20= indexes=20exist=20must=0A+=20=20=20also=20be=20checked.=20=20This=20= restriction=20may=20be=20relaxed=20in=20the=20future.=0A+=20=20=0A= +=20=0A+=20=20=0A+=20=20=20To=20restrict=20the=20range=20of=20= blocks=20within=20a=20table=20that=20are=20checked,=20specify=20the=0A+=20= =20=20=20or=20=20and/or=0A= +=20=20=20=20or=20=20= options=20with=20numeric=0A+=20=20=20values=20for=20the=20starting=20and=20= ending=20block=20numbers.=20=20Although=20these=20options=0A+=20=20=20= make=20the=20most=20sense=20when=20applied=20to=20a=20single=20table,=20= if=20specified=20along=20with=0A+=20=20=20options=20that=20select=20= multiple=20tables,=20each=20table=20check=20will=20be=20restricted=20to=0A= +=20=20=20the=20specified=20blocks.=20=20If=20= =20is=20omitted,=20checking=0A+=20=20=20= begins=20with=20the=20first=20block.=20If=20=20= is=20omitted,=0A+=20=20=20checking=20continues=20to=20the=20end=20of=20= the=20relation.=0A+=20=20=0A+=20=0A+=20=20=0A+=20=20=20Some=20= users=20may=20wish=20to=20periodically=20check=20tables=20without=20= incurring=20the=20cost=0A+=20=20=20of=20rechecking=20older=20table=20= blocks,=20presumably=20because=20those=20blocks=20have=0A+=20=20=20= already=20been=20checked=20in=20the=20past.=20=20There=20is=20at=20= present=20no=20perfect=20way=20to=20do=0A+=20=20=20this.=20=20Although=20= the=20=20and=20=0A= +=20=20=20options=20can=20be=20used=20to=20restrict=20blocks,=20the=20= user=20is=20not=20expected=20to=20have=0A+=20=20=20perfect=20knowledge=20= of=20which=20blocks=20have=20already=20been=20checked,=20and=20in=20any=0A= +=20=20=20event,=20some=20blocks=20that=20were=20previously=20checked=20= may=20have=20been=20subject=20to=0A+=20=20=20modification=20since=20the=20= last=20check.=20=20As=20an=20approximation=20to=20the=20desired=0A+=20=20= =20functionality,=20one=20can=20specify=20the=0A+=20=20=20= =20or=20=20option,=20= or=0A+=20=20=20alternatively=20the=0A+=20=20=20=20or=20= =20option=20to=20skip=0A+=20=20=20= blocks=20marked=20in=20the=20visibility=20map=20as=20all-frozen=20or=20= all-visible,=0A+=20=20=20respectively.=0A+=20=20=0A+=20=0A= +=0A+=20=0A+=20=20Example=20Usage=0A+=0A+=20=20= =0A+=20=20=20For=20table=20corruption,=20each=20detected=20= corruption=20is=20reported=20on=20two=20lines,=20the=0A+=20=20=20first=20= line=20shows=20the=20location=20and=20the=20second=20line=20shows=20a=20= message=20describing=0A+=20=20=20the=20problem.=0A+=20=20=0A+=0A+=20= =20=0A+=20=20=20Checking=20an=20entire=20database=20which=20= contains=20one=20corrupt=20table,=20"mytable",=0A+=20=20=20along=20with=20= the=20output:=0A+=20=20=0A+=0A+=0A+%=20pg_amcheck=20= --check-toast=20--skip-indexes=20mydb=0A= +(relname=3Dmytable,blkno=3D17,offnum=3D12,attnum=3D)=0A+xmin=20= 4294967295=20precedes=20relation=20freeze=20threshold=2017:1134217582=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D4,attnum=3D)=0A+data=20begins=20= at=20offset=20152=20beyond=20the=20tuple=20length=2058=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D4,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 152=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D5,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 27=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D6,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 16=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D960,offnum=3D7,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 21=20(3=20attributes,=20no=20nulls)=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D2,attnum=3D)=0A+number=20of=20= attributes=202047=20exceeds=20maximum=20expected=20for=20table=203=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D10,attnum=3D)=0A+tuple=20data=20= should=20begin=20at=20byte=20280,=20but=20actually=20begins=20at=20byte=20= 24=20(2047=20attributes,=20has=20nulls)=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D15,attnum=3D)=0A+number=20of=20= attributes=2067=20exceeds=20maximum=20expected=20for=20table=203=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D16,attnum=3D1)=0A+attribute=20= 1=20with=20length=204294967295=20ends=20at=20offset=20416848000=20beyond=20= total=20tuple=20length=2058=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D18,attnum=3D2)=0A+final=20= toast=20chunk=20number=200=20differs=20from=20expected=20value=206=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D19,attnum=3D2)=0A+toasted=20= value=20for=20attribute=202=20missing=20from=20toast=20table=0A= +(relname=3Dmytable,blkno=3D1147,offnum=3D21,attnum=3D)=0A+tuple=20is=20= marked=20as=20only=20locked,=20but=20also=20claims=20key=20columns=20= were=20updated=0A+(relname=3Dmytable,blkno=3D1147,offnum=3D22,attnum=3D)=0A= +multitransaction=20ID=201775655=20is=20from=20before=20relation=20= cutoff=202355572=0A+=0A+=0A+=20=20=0A+=20=20=20For=20= index=20corruption,=20the=20output=20is=20more=20free-form,=20and=20may=20= span=20differing=0A+=20=20=20numbers=20of=20lines=20per=20corruption=20= detected.=0A+=20=20=0A+=0A+=20=20=0A+=20=20=20Checking=20an=20= entire=20database=20which=20contains=20one=20corrupt=20index,=0A+=20=20=20= "corrupt_index",=20with=20corruption=20in=20the=20page=20header,=20along=20= with=20the=20output:=0A+=20=20=0A+=0A+=0A+%=20pg_amcheck=20= --check-toast=20--check-indexes=20--schema=3Dpublic=20= --table=3Dtable_with_corrupt_index=20mydb=0A+index=20check=20failed=20= for=20index=20corrupt_index=20of=20table=20table_with_corrupt_index:=0A= +ERROR:=20=20XX002:=20index=20"corrupt_index"=20is=20not=20a=20btree=0A= +LOCATION:=20=20_bt_getmeta,=20nbtpage.c:152=0A+=0A+=0A+=20=20= =0A+=20=20=20Checking=20again=20after=20rebuilding=20the=20index=20= but=20corrupting=20the=20contents,=0A+=20=20=20along=20with=20the=20= output:=0A+=20=20=0A+=0A+=0A+%=20pg_amcheck=20= --check-toast=20--check-indexes=20--schema=3Dpublic=20= --table=3Dtable_with_corrupt_index=20mydb=0A+index=20check=20failed=20= for=20index=20corrupt_index=20of=20table=20table_with_corrupt_index:=0A= +ERROR:=20=20XX002:=20index=20tuple=20size=20does=20not=20equal=20lp_len=20= in=20index=20"corrupt_index"=0A+DETAIL:=20=20Index=20tid=3D(39,49)=20= tuple=20size=3D3373=20lp_len=3D24=20page=20lsn=3D0/2B548C0.=0A+HINT:=20=20= This=20could=20be=20a=20torn=20page=20problem.=0A+LOCATION:=20=20= bt_target_page_check,=20verify_nbtree.c:1125=0A+=0A+=0A+=20= =0A+=0Adiff=20--git=20a/src/tools/msvc/Mkvcbuild.pm=20= b/src/tools/msvc/Mkvcbuild.pm=0Aindex=2089e1b39036..8cf0554823=20100644=0A= ---=20a/src/tools/msvc/Mkvcbuild.pm=0A+++=20= b/src/tools/msvc/Mkvcbuild.pm=0A@@=20-33,9=20+33,9=20@@=20my=20= @unlink_on_exit;=0A=20=0A=20#=20Set=20of=20variables=20for=20modules=20= in=20contrib/=20and=20src/test/modules/=0A=20my=20$contrib_defines=20=3D=20= {=20'refint'=20=3D>=20'REFINT_VERBOSE'=20};=0A-my=20@contrib_uselibpq=20= =3D=20('dblink',=20'oid2name',=20'postgres_fdw',=20'vacuumlo');=0A-my=20= @contrib_uselibpgport=20=20=20=3D=20('oid2name',=20'pg_standby',=20= 'vacuumlo');=0A-my=20@contrib_uselibpgcommon=20=3D=20('oid2name',=20= 'pg_standby',=20'vacuumlo');=0A+my=20@contrib_uselibpq=20=3D=20= ('dblink',=20'oid2name',=20'pg_amcheck',=20'postgres_fdw',=20= 'vacuumlo');=0A+my=20@contrib_uselibpgport=20=20=20=3D=20('oid2name',=20= 'pg_amcheck',=20'pg_standby',=20'vacuumlo');=0A+my=20= @contrib_uselibpgcommon=20=3D=20('oid2name',=20'pg_amcheck',=20= 'pg_standby',=20'vacuumlo');=0A=20my=20$contrib_extralibs=20=20=20=20=20=20= =3D=20undef;=0A=20my=20$contrib_extraincludes=20=3D=20{=20'dblink'=20=3D>=20= ['src/backend']=20};=0A=20my=20$contrib_extrasource=20=3D=20{=0Adiff=20= --git=20a/src/tools/pgindent/typedefs.list=20= b/src/tools/pgindent/typedefs.list=0Aindex=20bca30f3dde..369b8e7c6f=20= 100644=0A---=20a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-102,6=20+102,7=20@@=20= AlterUserMappingStmt=0A=20AlteredTableInfo=0A=20AlternativeSubPlan=0A=20= AlternativeSubPlanState=0A+AmCheckSettings=0A=20= AnalyzeAttrComputeStatsFunc=0A=20AnalyzeAttrFetchFunc=0A=20= AnalyzeForeignTable_function=0A@@=20-403,6=20+404,7=20@@=20ConfigData=0A=20= ConfigVariable=0A=20ConnCacheEntry=0A=20ConnCacheKey=0A+ConnectOptions=0A= =20ConnStatusType=0A=20ConnType=0A=20ConnectionStateEnum=0A--=20=0A= 2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479 Content-Disposition: attachment; filename=v17-0003-Creating-non-throwing-interface-to-clog-and-slru.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v17-0003-Creating-non-throwing-interface-to-clog-and-slru.patch" Content-Transfer-Encoding: quoted-printable =46rom=20f34c48af2746b752d11c3d55ce7604785dc44068=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Tue,=206=20Oct=202020=2011:28:18=20-0700=0ASubject:=20[PATCH=20= v17=203/4]=20Creating=20non-throwing=20interface=20to=20clog=20and=20= slru.=0A=0A---=0A=20src/backend/access/transam/clog.c=20=20=20=20=20=20|=20= 21=20+++---=0A=20src/backend/access/transam/commit_ts.c=20|=20=204=20+-=0A= =20src/backend/access/transam/multixact.c=20|=2016=20++---=0A=20= src/backend/access/transam/slru.c=20=20=20=20=20=20|=2023=20+++---=0A=20= src/backend/access/transam/subtrans.c=20=20|=20=204=20+-=0A=20= src/backend/access/transam/transam.c=20=20=20|=2098=20= +++++++++-----------------=0A=20src/backend/commands/async.c=20=20=20=20=20= =20=20=20=20=20=20|=20=204=20+-=0A=20= src/backend/storage/lmgr/predicate.c=20=20=20|=20=204=20+-=0A=20= src/include/access/clog.h=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= 18=20+----=0A=20src/include/access/clogdefs.h=20=20=20=20=20=20=20=20=20=20= |=2033=20+++++++++=0A=20src/include/access/slru.h=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=206=20+-=0A=20src/include/access/transam.h=20=20=20= =20=20=20=20=20=20=20=20|=20=203=20+=0A=2012=20files=20changed,=20122=20= insertions(+),=20112=20deletions(-)=0A=20create=20mode=20100644=20= src/include/access/clogdefs.h=0A=0Adiff=20--git=20= a/src/backend/access/transam/clog.c=20= b/src/backend/access/transam/clog.c=0Aindex=20034349aa7b..a2eb3e2983=20= 100644=0A---=20a/src/backend/access/transam/clog.c=0A+++=20= b/src/backend/access/transam/clog.c=0A@@=20-357,7=20+357,7=20@@=20= TransactionIdSetPageStatusInternal(TransactionId=20xid,=20int=20= nsubxids,=0A=20=09=20*=20write-busy,=20since=20we=20don't=20care=20if=20= the=20update=20reaches=20disk=20sooner=20than=0A=20=09=20*=20we=20think.=0A= =20=09=20*/=0A-=09slotno=20=3D=20SimpleLruReadPage(XactCtl,=20pageno,=20= XLogRecPtrIsInvalid(lsn),=20xid);=0A+=09slotno=20=3D=20= SimpleLruReadPage(XactCtl,=20pageno,=20XLogRecPtrIsInvalid(lsn),=20xid,=20= true);=0A=20=0A=20=09/*=0A=20=09=20*=20Set=20the=20main=20transaction=20= id,=20if=20any.=0A@@=20-631,7=20+631,7=20@@=20= TransactionIdSetStatusBit(TransactionId=20xid,=20XidStatus=20status,=20= XLogRecPtr=20lsn,=20i=0A=20=20*=20for=20most=20uses;=20= TransactionLogFetch()=20in=20transam.c=20is=20the=20intended=20caller.=0A= =20=20*/=0A=20XidStatus=0A-TransactionIdGetStatus(TransactionId=20xid,=20= XLogRecPtr=20*lsn)=0A+TransactionIdGetStatus(TransactionId=20xid,=20= XLogRecPtr=20*lsn,=20bool=20throwError)=0A=20{=0A=20=09int=09=09=09= pageno=20=3D=20TransactionIdToPage(xid);=0A=20=09int=09=09=09byteno=20=3D=20= TransactionIdToByte(xid);=0A@@=20-643,13=20+643,18=20@@=20= TransactionIdGetStatus(TransactionId=20xid,=20XLogRecPtr=20*lsn)=0A=20=0A= =20=09/*=20lock=20is=20acquired=20by=20SimpleLruReadPage_ReadOnly=20*/=0A= =20=0A-=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(XactCtl,=20pageno,=20= xid);=0A-=09byteptr=20=3D=20XactCtl->shared->page_buffer[slotno]=20+=20= byteno;=0A+=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(XactCtl,=20= pageno,=20xid,=20throwError);=0A+=09if=20(slotno=20=3D=3D=20= InvalidSlotNo)=0A+=09=09status=20=3D=20TRANSACTION_STATUS_UNKNOWN;=0A+=09= else=0A+=09{=0A+=09=09byteptr=20=3D=20= XactCtl->shared->page_buffer[slotno]=20+=20byteno;=0A=20=0A-=09status=20= =3D=20(*byteptr=20>>=20bshift)=20&=20CLOG_XACT_BITMASK;=0A+=09=09status=20= =3D=20(*byteptr=20>>=20bshift)=20&=20CLOG_XACT_BITMASK;=0A=20=0A-=09= lsnindex=20=3D=20GetLSNIndex(slotno,=20xid);=0A-=09*lsn=20=3D=20= XactCtl->shared->group_lsn[lsnindex];=0A+=09=09lsnindex=20=3D=20= GetLSNIndex(slotno,=20xid);=0A+=09=09*lsn=20=3D=20= XactCtl->shared->group_lsn[lsnindex];=0A+=09}=0A=20=0A=20=09= LWLockRelease(XactSLRULock);=0A=20=0A@@=20-796,7=20+801,7=20@@=20= TrimCLOG(void)=0A=20=09=09int=09=09=09slotno;=0A=20=09=09char=09=20=20=20= *byteptr;=0A=20=0A-=09=09slotno=20=3D=20SimpleLruReadPage(XactCtl,=20= pageno,=20false,=20xid);=0A+=09=09slotno=20=3D=20= SimpleLruReadPage(XactCtl,=20pageno,=20false,=20xid,=20true);=0A=20=09=09= byteptr=20=3D=20XactCtl->shared->page_buffer[slotno]=20+=20byteno;=0A=20=0A= =20=09=09/*=20Zero=20so-far-unused=20positions=20in=20the=20current=20= byte=20*/=0Adiff=20--git=20a/src/backend/access/transam/commit_ts.c=20= b/src/backend/access/transam/commit_ts.c=0Aindex=20= cb8a968801..98c685405c=20100644=0A---=20= a/src/backend/access/transam/commit_ts.c=0A+++=20= b/src/backend/access/transam/commit_ts.c=0A@@=20-237,7=20+237,7=20@@=20= SetXidCommitTsInPage(TransactionId=20xid,=20int=20nsubxids,=0A=20=0A=20=09= LWLockAcquire(CommitTsSLRULock,=20LW_EXCLUSIVE);=0A=20=0A-=09slotno=20=3D=20= SimpleLruReadPage(CommitTsCtl,=20pageno,=20true,=20xid);=0A+=09slotno=20= =3D=20SimpleLruReadPage(CommitTsCtl,=20pageno,=20true,=20xid,=20true);=0A= =20=0A=20=09TransactionIdSetCommitTs(xid,=20ts,=20nodeid,=20slotno);=0A=20= =09for=20(i=20=3D=200;=20i=20<=20nsubxids;=20i++)=0A@@=20-342,7=20+342,7=20= @@=20TransactionIdGetCommitTsData(TransactionId=20xid,=20TimestampTz=20= *ts,=0A=20=09}=0A=20=0A=20=09/*=20lock=20is=20acquired=20by=20= SimpleLruReadPage_ReadOnly=20*/=0A-=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(CommitTsCtl,=20pageno,=20xid);=0A+=09slotno=20= =3D=20SimpleLruReadPage_ReadOnly(CommitTsCtl,=20pageno,=20xid,=20true);=0A= =20=09memcpy(&entry,=0A=20=09=09=20=20=20= CommitTsCtl->shared->page_buffer[slotno]=20+=0A=20=09=09=20=20=20= SizeOfCommitTimestampEntry=20*=20entryno,=0Adiff=20--git=20= a/src/backend/access/transam/multixact.c=20= b/src/backend/access/transam/multixact.c=0Aindex=20= 81752b68eb..5c0213b06d=20100644=0A---=20= a/src/backend/access/transam/multixact.c=0A+++=20= b/src/backend/access/transam/multixact.c=0A@@=20-881,7=20+881,7=20@@=20= RecordNewMultiXact(MultiXactId=20multi,=20MultiXactOffset=20offset,=0A=20= =09=20*=20enough=20that=20a=20MultiXactId=20is=20really=20involved.=20=20= Perhaps=20someday=20we'll=0A=20=09=20*=20take=20the=20trouble=20to=20= generalize=20the=20slru.c=20error=20reporting=20code.=0A=20=09=20*/=0A-=09= slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20= multi);=0A+=09slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20= pageno,=20true,=20multi,=20true);=0A=20=09offptr=20=3D=20= (MultiXactOffset=20*)=20MultiXactOffsetCtl->shared->page_buffer[slotno];=0A= =20=09offptr=20+=3D=20entryno;=0A=20=0A@@=20-914,7=20+914,7=20@@=20= RecordNewMultiXact(MultiXactId=20multi,=20MultiXactOffset=20offset,=0A=20= =0A=20=09=09if=20(pageno=20!=3D=20prev_pageno)=0A=20=09=09{=0A-=09=09=09= slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20true,=20= multi);=0A+=09=09=09slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20= pageno,=20true,=20multi,=20true);=0A=20=09=09=09prev_pageno=20=3D=20= pageno;=0A=20=09=09}=0A=20=0A@@=20-1345,7=20+1345,7=20@@=20retry:=0A=20=09= pageno=20=3D=20MultiXactIdToOffsetPage(multi);=0A=20=09entryno=20=3D=20= MultiXactIdToOffsetEntry(multi);=0A=20=0A-=09slotno=20=3D=20= SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20multi);=0A+=09= slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20= multi,=20true);=0A=20=09offptr=20=3D=20(MultiXactOffset=20*)=20= MultiXactOffsetCtl->shared->page_buffer[slotno];=0A=20=09offptr=20+=3D=20= entryno;=0A=20=09offset=20=3D=20*offptr;=0A@@=20-1377,7=20+1377,7=20@@=20= retry:=0A=20=09=09entryno=20=3D=20MultiXactIdToOffsetEntry(tmpMXact);=0A=20= =0A=20=09=09if=20(pageno=20!=3D=20prev_pageno)=0A-=09=09=09slotno=20=3D=20= SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20tmpMXact);=0A+=09= =09=09slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20= true,=20tmpMXact,=20true);=0A=20=0A=20=09=09offptr=20=3D=20= (MultiXactOffset=20*)=20MultiXactOffsetCtl->shared->page_buffer[slotno];=0A= =20=09=09offptr=20+=3D=20entryno;=0A@@=20-1418,7=20+1418,7=20@@=20retry:=0A= =20=0A=20=09=09if=20(pageno=20!=3D=20prev_pageno)=0A=20=09=09{=0A-=09=09=09= slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20true,=20= multi);=0A+=09=09=09slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20= pageno,=20true,=20multi,=20true);=0A=20=09=09=09prev_pageno=20=3D=20= pageno;=0A=20=09=09}=0A=20=0A@@=20-2063,7=20+2063,7=20@@=20= TrimMultiXact(void)=0A=20=09=09int=09=09=09slotno;=0A=20=09=09= MultiXactOffset=20*offptr;=0A=20=0A-=09=09slotno=20=3D=20= SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20true,=20nextMXact);=0A= +=09=09slotno=20=3D=20SimpleLruReadPage(MultiXactOffsetCtl,=20pageno,=20= true,=20nextMXact,=20true);=0A=20=09=09offptr=20=3D=20(MultiXactOffset=20= *)=20MultiXactOffsetCtl->shared->page_buffer[slotno];=0A=20=09=09offptr=20= +=3D=20entryno;=0A=20=0A@@=20-2095,7=20+2095,7=20@@=20= TrimMultiXact(void)=0A=20=09=09int=09=09=09memberoff;=0A=20=0A=20=09=09= memberoff=20=3D=20MXOffsetToMemberOffset(offset);=0A-=09=09slotno=20=3D=20= SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20true,=20offset);=0A+=09= =09slotno=20=3D=20SimpleLruReadPage(MultiXactMemberCtl,=20pageno,=20= true,=20offset,=20true);=0A=20=09=09xidptr=20=3D=20(TransactionId=20*)=0A= =20=09=09=09(MultiXactMemberCtl->shared->page_buffer[slotno]=20+=20= memberoff);=0A=20=0A@@=20-2749,7=20+2749,7=20@@=20= find_multixact_start(MultiXactId=20multi,=20MultiXactOffset=20*result)=0A= =20=09=09return=20false;=0A=20=0A=20=09/*=20lock=20is=20acquired=20by=20= SimpleLruReadPage_ReadOnly=20*/=0A-=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(MultiXactOffsetCtl,=20pageno,=20multi);=0A+=09= slotno=20=3D=20SimpleLruReadPage_ReadOnly(MultiXactOffsetCtl,=20pageno,=20= multi,=20true);=0A=20=09offptr=20=3D=20(MultiXactOffset=20*)=20= MultiXactOffsetCtl->shared->page_buffer[slotno];=0A=20=09offptr=20+=3D=20= entryno;=0A=20=09offset=20=3D=20*offptr;=0Adiff=20--git=20= a/src/backend/access/transam/slru.c=20= b/src/backend/access/transam/slru.c=0Aindex=2016a7898697..daa145eeff=20= 100644=0A---=20a/src/backend/access/transam/slru.c=0A+++=20= b/src/backend/access/transam/slru.c=0A@@=20-385,14=20+385,15=20@@=20= SimpleLruWaitIO(SlruCtl=20ctl,=20int=20slotno)=0A=20=20*=20The=20= passed-in=20xid=20is=20used=20only=20for=20error=20reporting,=20and=20= may=20be=0A=20=20*=20InvalidTransactionId=20if=20no=20specific=20xid=20= is=20associated=20with=20the=20action.=0A=20=20*=0A-=20*=20Return=20= value=20is=20the=20shared-buffer=20slot=20number=20now=20holding=20the=20= page.=0A-=20*=20The=20buffer's=20LRU=20access=20info=20is=20updated.=0A+=20= *=20On=20error,=20when=20throwError=20is=20false,=20the=20return=20value=20= is=20negative.=0A+=20*=20Otherwise,=20return=20value=20is=20the=20= shared-buffer=20slot=20number=20now=20holding=20the=0A+=20*=20page,=20= and=20the=20buffer's=20LRU=20access=20info=20is=20updated.=0A=20=20*=0A=20= =20*=20Control=20lock=20must=20be=20held=20at=20entry,=20and=20will=20be=20= held=20at=20exit.=0A=20=20*/=0A=20int=0A=20SimpleLruReadPage(SlruCtl=20= ctl,=20int=20pageno,=20bool=20write_ok,=0A-=09=09=09=09=20=20= TransactionId=20xid)=0A+=09=09=09=09=20=20TransactionId=20xid,=20bool=20= throwError)=0A=20{=0A=20=09SlruShared=09shared=20=3D=20ctl->shared;=0A=20= =0A@@=20-465,7=20+466,11=20@@=20SimpleLruReadPage(SlruCtl=20ctl,=20int=20= pageno,=20bool=20write_ok,=0A=20=0A=20=09=09/*=20Now=20it's=20okay=20to=20= ereport=20if=20we=20failed=20*/=0A=20=09=09if=20(!ok)=0A-=09=09=09= SlruReportIOError(ctl,=20pageno,=20xid);=0A+=09=09{=0A+=09=09=09if=20= (throwError)=0A+=09=09=09=09SlruReportIOError(ctl,=20pageno,=20xid);=0A+=09= =09=09return=20InvalidSlotNo;=0A+=09=09}=0A=20=0A=20=09=09= SlruRecentlyUsed(shared,=20slotno);=0A=20=0A@@=20-484,14=20+489,16=20@@=20= SimpleLruReadPage(SlruCtl=20ctl,=20int=20pageno,=20bool=20write_ok,=0A=20= =20*=20The=20passed-in=20xid=20is=20used=20only=20for=20error=20= reporting,=20and=20may=20be=0A=20=20*=20InvalidTransactionId=20if=20no=20= specific=20xid=20is=20associated=20with=20the=20action.=0A=20=20*=0A-=20= *=20Return=20value=20is=20the=20shared-buffer=20slot=20number=20now=20= holding=20the=20page.=0A-=20*=20The=20buffer's=20LRU=20access=20info=20= is=20updated.=0A+=20*=20On=20error,=20when=20throwError=20is=20false,=20= the=20return=20value=20is=20negative.=0A+=20*=20Otherwise,=20return=20= value=20is=20the=20shared-buffer=20slot=20number=20now=20holding=20the=0A= +=20*=20page,=20and=20the=20buffer's=20LRU=20access=20info=20is=20= updated.=0A=20=20*=0A=20=20*=20Control=20lock=20must=20NOT=20be=20held=20= at=20entry,=20but=20will=20be=20held=20at=20exit.=0A=20=20*=20It=20is=20= unspecified=20whether=20the=20lock=20will=20be=20shared=20or=20= exclusive.=0A=20=20*/=0A=20int=0A-SimpleLruReadPage_ReadOnly(SlruCtl=20= ctl,=20int=20pageno,=20TransactionId=20xid)=0A= +SimpleLruReadPage_ReadOnly(SlruCtl=20ctl,=20int=20pageno,=20= TransactionId=20xid,=0A+=09=09=09=09=09=09=20=20=20bool=20throwError)=0A=20= {=0A=20=09SlruShared=09shared=20=3D=20ctl->shared;=0A=20=09int=09=09=09= slotno;=0A@@=20-520,7=20+527,7=20@@=20SimpleLruReadPage_ReadOnly(SlruCtl=20= ctl,=20int=20pageno,=20TransactionId=20xid)=0A=20=09= LWLockRelease(shared->ControlLock);=0A=20=09= LWLockAcquire(shared->ControlLock,=20LW_EXCLUSIVE);=0A=20=0A-=09return=20= SimpleLruReadPage(ctl,=20pageno,=20true,=20xid);=0A+=09return=20= SimpleLruReadPage(ctl,=20pageno,=20true,=20xid,=20throwError);=0A=20}=0A=20= =0A=20/*=0Adiff=20--git=20a/src/backend/access/transam/subtrans.c=20= b/src/backend/access/transam/subtrans.c=0Aindex=200111e867c7..353b946731=20= 100644=0A---=20a/src/backend/access/transam/subtrans.c=0A+++=20= b/src/backend/access/transam/subtrans.c=0A@@=20-83,7=20+83,7=20@@=20= SubTransSetParent(TransactionId=20xid,=20TransactionId=20parent)=0A=20=0A= =20=09LWLockAcquire(SubtransSLRULock,=20LW_EXCLUSIVE);=0A=20=0A-=09= slotno=20=3D=20SimpleLruReadPage(SubTransCtl,=20pageno,=20true,=20xid);=0A= +=09slotno=20=3D=20SimpleLruReadPage(SubTransCtl,=20pageno,=20true,=20= xid,=20true);=0A=20=09ptr=20=3D=20(TransactionId=20*)=20= SubTransCtl->shared->page_buffer[slotno];=0A=20=09ptr=20+=3D=20entryno;=0A= =20=0A@@=20-123,7=20+123,7=20@@=20SubTransGetParent(TransactionId=20xid)=0A= =20=0A=20=09/*=20lock=20is=20acquired=20by=20SimpleLruReadPage_ReadOnly=20= */=0A=20=0A-=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(SubTransCtl,=20= pageno,=20xid);=0A+=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(SubTransCtl,=20pageno,=20xid,=20true);=0A=20=09= ptr=20=3D=20(TransactionId=20*)=20= SubTransCtl->shared->page_buffer[slotno];=0A=20=09ptr=20+=3D=20entryno;=0A= =20=0Adiff=20--git=20a/src/backend/access/transam/transam.c=20= b/src/backend/access/transam/transam.c=0Aindex=20a28918657c..88f867e5ef=20= 100644=0A---=20a/src/backend/access/transam/transam.c=0A+++=20= b/src/backend/access/transam/transam.c=0A@@=20-35,7=20+35,8=20@@=20= static=20XidStatus=20cachedFetchXidStatus;=0A=20static=20XLogRecPtr=20= cachedCommitLSN;=0A=20=0A=20/*=20Local=20functions=20*/=0A-static=20= XidStatus=20TransactionLogFetch(TransactionId=20transactionId);=0A= +static=20XidStatus=20TransactionLogFetch(TransactionId=20transactionId,=0A= +=09=09=09=09=09=09=09=09=09=20bool=20throwError);=0A=20=0A=20=0A=20/*=20= ----------------------------------------------------------------=0A@@=20= -49,7=20+50,7=20@@=20static=20XidStatus=20= TransactionLogFetch(TransactionId=20transactionId);=0A=20=20*=20= TransactionLogFetch=20---=20fetch=20commit=20status=20of=20specified=20= transaction=20id=0A=20=20*/=0A=20static=20XidStatus=0A= -TransactionLogFetch(TransactionId=20transactionId)=0A= +TransactionLogFetch(TransactionId=20transactionId,=20bool=20throwError)=0A= =20{=0A=20=09XidStatus=09xidstatus;=0A=20=09XLogRecPtr=09xidlsn;=0A@@=20= -76,14=20+77,16=20@@=20TransactionLogFetch(TransactionId=20= transactionId)=0A=20=09/*=0A=20=09=20*=20Get=20the=20transaction=20= status.=0A=20=09=20*/=0A-=09xidstatus=20=3D=20= TransactionIdGetStatus(transactionId,=20&xidlsn);=0A+=09xidstatus=20=3D=20= TransactionIdGetStatus(transactionId,=20&xidlsn,=20throwError);=0A=20=0A=20= =09/*=0A=20=09=20*=20Cache=20it,=20but=20DO=20NOT=20cache=20status=20for=20= unfinished=20or=20sub-committed=0A=20=09=20*=20transactions!=20=20We=20= only=20cache=20status=20that=20is=20guaranteed=20not=20to=20change.=0A+=09= =20*=20Likewise,=20DO=20NOT=20cache=20when=20the=20status=20is=20= unknown.=0A=20=09=20*/=0A=20=09if=20(xidstatus=20!=3D=20= TRANSACTION_STATUS_IN_PROGRESS=20&&=0A-=09=09xidstatus=20!=3D=20= TRANSACTION_STATUS_SUB_COMMITTED)=0A+=09=09xidstatus=20!=3D=20= TRANSACTION_STATUS_SUB_COMMITTED=20&&=0A+=09=09xidstatus=20!=3D=20= TRANSACTION_STATUS_UNKNOWN)=0A=20=09{=0A=20=09=09cachedFetchXid=20=3D=20= transactionId;=0A=20=09=09cachedFetchXidStatus=20=3D=20xidstatus;=0A@@=20= -96,6=20+99,7=20@@=20TransactionLogFetch(TransactionId=20transactionId)=0A= =20/*=20----------------------------------------------------------------=0A= =20=20*=09=09=09=09=09=09Interface=20functions=0A=20=20*=0A+=20*=09=09= TransactionIdResolveStatus=0A=20=20*=09=09TransactionIdDidCommit=0A=20=20= *=09=09TransactionIdDidAbort=0A=20=20*=09=09=3D=3D=3D=3D=3D=3D=3D=3D=0A= @@=20-115,24=20+119,17=20@@=20TransactionLogFetch(TransactionId=20= transactionId)=0A=20=20*/=0A=20=0A=20/*=0A-=20*=20TransactionIdDidCommit=0A= -=20*=09=09True=20iff=20transaction=20associated=20with=20the=20= identifier=20did=20commit.=0A-=20*=0A-=20*=20Note:=0A-=20*=09=09Assumes=20= transaction=20identifier=20is=20valid=20and=20exists=20in=20clog.=0A+=20= *=20TransactionIdResolveStatus=0A+=20*=20=09=09Returns=20the=20status=20= of=20the=20transaction=20associated=20with=20the=20identifer,=0A+=20*=09=09= recursively=20resolving=20sub-committed=20transaction=20status=20by=20= checking=0A+=20*=09=09the=20parent=20transaction.=0A=20=20*/=0A-bool=09=09= =09=09=09=09=09/*=20true=20if=20given=20transaction=20committed=20*/=0A= -TransactionIdDidCommit(TransactionId=20transactionId)=0A+XidStatus=0A= +TransactionIdResolveStatus(TransactionId=20transactionId,=20bool=20= throwError)=0A=20{=0A=20=09XidStatus=09xidstatus;=0A=20=0A-=09xidstatus=20= =3D=20TransactionLogFetch(transactionId);=0A-=0A-=09/*=0A-=09=20*=20If=20= it's=20marked=20committed,=20it's=20committed.=0A-=09=20*/=0A-=09if=20= (xidstatus=20=3D=3D=20TRANSACTION_STATUS_COMMITTED)=0A-=09=09return=20= true;=0A+=09xidstatus=20=3D=20TransactionLogFetch(transactionId,=20= throwError);=0A=20=0A=20=09/*=0A=20=09=20*=20If=20it's=20marked=20= subcommitted,=20we=20have=20to=20check=20the=20parent=20recursively.=0A= @@=20-153,21=20+150,31=20@@=20TransactionIdDidCommit(TransactionId=20= transactionId)=0A=20=09=09TransactionId=20parentXid;=0A=20=0A=20=09=09if=20= (TransactionIdPrecedes(transactionId,=20TransactionXmin))=0A-=09=09=09= return=20false;=0A+=09=09=09return=20TRANSACTION_STATUS_ABORTED;=0A=20=09= =09parentXid=20=3D=20SubTransGetParent(transactionId);=0A=20=09=09if=20= (!TransactionIdIsValid(parentXid))=0A=20=09=09{=0A=20=09=09=09= elog(WARNING,=20"no=20pg_subtrans=20entry=20for=20subcommitted=20XID=20= %u",=0A=20=09=09=09=09=20transactionId);=0A-=09=09=09return=20false;=0A+=09= =09=09return=20TRANSACTION_STATUS_ABORTED;=0A=20=09=09}=0A-=09=09return=20= TransactionIdDidCommit(parentXid);=0A+=09=09return=20= TransactionIdResolveStatus(parentXid,=20throwError);=0A=20=09}=0A+=09= return=20xidstatus;=0A+}=0A=20=0A-=09/*=0A-=09=20*=20It's=20not=20= committed.=0A-=09=20*/=0A-=09return=20false;=0A+/*=0A+=20*=20= TransactionIdDidCommit=0A+=20*=09=09True=20iff=20transaction=20= associated=20with=20the=20identifier=20did=20commit.=0A+=20*=0A+=20*=20= Note:=0A+=20*=09=09Assumes=20transaction=20identifier=20is=20valid=20and=20= exists=20in=20clog.=0A+=20*/=0A+bool=09=09=09=09=09=09=09/*=20true=20if=20= given=20transaction=20committed=20*/=0A= +TransactionIdDidCommit(TransactionId=20transactionId)=0A+{=0A+=09return=20= (TransactionIdResolveStatus(transactionId,=20true)=20=3D=3D=0A+=09=09=09= TRANSACTION_STATUS_COMMITTED);=0A=20}=0A=20=0A=20/*=0A@@=20-180,43=20= +187,8=20@@=20TransactionIdDidCommit(TransactionId=20transactionId)=0A=20= bool=09=09=09=09=09=09=09/*=20true=20if=20given=20transaction=20aborted=20= */=0A=20TransactionIdDidAbort(TransactionId=20transactionId)=0A=20{=0A-=09= XidStatus=09xidstatus;=0A-=0A-=09xidstatus=20=3D=20= TransactionLogFetch(transactionId);=0A-=0A-=09/*=0A-=09=20*=20If=20it's=20= marked=20aborted,=20it's=20aborted.=0A-=09=20*/=0A-=09if=20(xidstatus=20= =3D=3D=20TRANSACTION_STATUS_ABORTED)=0A-=09=09return=20true;=0A-=0A-=09= /*=0A-=09=20*=20If=20it's=20marked=20subcommitted,=20we=20have=20to=20= check=20the=20parent=20recursively.=0A-=09=20*=20However,=20if=20it's=20= older=20than=20TransactionXmin,=20we=20can't=20look=20at=0A-=09=20*=20= pg_subtrans;=20instead=20assume=20that=20the=20parent=20crashed=20= without=20cleaning=20up=0A-=09=20*=20its=20children.=0A-=09=20*/=0A-=09= if=20(xidstatus=20=3D=3D=20TRANSACTION_STATUS_SUB_COMMITTED)=0A-=09{=0A-=09= =09TransactionId=20parentXid;=0A-=0A-=09=09if=20= (TransactionIdPrecedes(transactionId,=20TransactionXmin))=0A-=09=09=09= return=20true;=0A-=09=09parentXid=20=3D=20= SubTransGetParent(transactionId);=0A-=09=09if=20= (!TransactionIdIsValid(parentXid))=0A-=09=09{=0A-=09=09=09/*=20see=20= notes=20in=20TransactionIdDidCommit=20*/=0A-=09=09=09elog(WARNING,=20"no=20= pg_subtrans=20entry=20for=20subcommitted=20XID=20%u",=0A-=09=09=09=09=20= transactionId);=0A-=09=09=09return=20true;=0A-=09=09}=0A-=09=09return=20= TransactionIdDidAbort(parentXid);=0A-=09}=0A-=0A-=09/*=0A-=09=20*=20It's=20= not=20aborted.=0A-=09=20*/=0A-=09return=20false;=0A+=09return=20= (TransactionIdResolveStatus(transactionId,=20true)=20=3D=3D=0A+=09=09=09= TRANSACTION_STATUS_ABORTED);=0A=20}=0A=20=0A=20/*=0A@@=20-419,7=20+391,7=20= @@=20TransactionIdGetCommitLSN(TransactionId=20xid)=0A=20=09/*=0A=20=09=20= *=20Get=20the=20transaction=20status.=0A=20=09=20*/=0A-=09(void)=20= TransactionIdGetStatus(xid,=20&result);=0A+=09(void)=20= TransactionIdGetStatus(xid,=20&result,=20true);=0A=20=0A=20=09return=20= result;=0A=20}=0Adiff=20--git=20a/src/backend/commands/async.c=20= b/src/backend/commands/async.c=0Aindex=208dbcace3f9..a49126dba0=20100644=0A= ---=20a/src/backend/commands/async.c=0A+++=20= b/src/backend/commands/async.c=0A@@=20-1477,7=20+1477,7=20@@=20= asyncQueueAddEntries(ListCell=20*nextNotify)=0A=20=09=09slotno=20=3D=20= SimpleLruZeroPage(NotifyCtl,=20pageno);=0A=20=09else=0A=20=09=09slotno=20= =3D=20SimpleLruReadPage(NotifyCtl,=20pageno,=20true,=0A-=09=09=09=09=09=09= =09=09=20=20=20InvalidTransactionId);=0A+=09=09=09=09=09=09=09=09=20=20=20= InvalidTransactionId,=20true);=0A=20=0A=20=09/*=20Note=20we=20mark=20the=20= page=20dirty=20before=20writing=20in=20it=20*/=0A=20=09= NotifyCtl->shared->page_dirty[slotno]=20=3D=20true;=0A@@=20-2010,7=20= +2010,7=20@@=20asyncQueueReadAllNotifications(void)=0A=20=09=09=09=20*=20= part=20of=20the=20page=20we=20will=20actually=20inspect.=0A=20=09=09=09=20= */=0A=20=09=09=09slotno=20=3D=20SimpleLruReadPage_ReadOnly(NotifyCtl,=20= curpage,=0A-=09=09=09=09=09=09=09=09=09=09=09=09InvalidTransactionId);=0A= +=09=09=09=09=09=09=09=09=09=09=09=09InvalidTransactionId,=20true);=0A=20= =09=09=09if=20(curpage=20=3D=3D=20QUEUE_POS_PAGE(head))=0A=20=09=09=09{=0A= =20=09=09=09=09/*=20we=20only=20want=20to=20read=20as=20far=20as=20head=20= */=0Adiff=20--git=20a/src/backend/storage/lmgr/predicate.c=20= b/src/backend/storage/lmgr/predicate.c=0Aindex=208a365b400c..6cf12e46f6=20= 100644=0A---=20a/src/backend/storage/lmgr/predicate.c=0A+++=20= b/src/backend/storage/lmgr/predicate.c=0A@@=20-904,7=20+904,7=20@@=20= SerialAdd(TransactionId=20xid,=20SerCommitSeqNo=20= minConflictCommitSeqNo)=0A=20=09=09slotno=20=3D=20= SimpleLruZeroPage(SerialSlruCtl,=20targetPage);=0A=20=09}=0A=20=09else=0A= -=09=09slotno=20=3D=20SimpleLruReadPage(SerialSlruCtl,=20targetPage,=20= true,=20xid);=0A+=09=09slotno=20=3D=20SimpleLruReadPage(SerialSlruCtl,=20= targetPage,=20true,=20xid,=20true);=0A=20=0A=20=09SerialValue(slotno,=20= xid)=20=3D=20minConflictCommitSeqNo;=0A=20=09= SerialSlruCtl->shared->page_dirty[slotno]=20=3D=20true;=0A@@=20-946,7=20= +946,7=20@@=20SerialGetMinConflictCommitSeqNo(TransactionId=20xid)=0A=20=09= =20*=20but=20will=20return=20with=20that=20lock=20held,=20which=20must=20= then=20be=20released.=0A=20=09=20*/=0A=20=09slotno=20=3D=20= SimpleLruReadPage_ReadOnly(SerialSlruCtl,=0A-=09=09=09=09=09=09=09=09=09=09= SerialPage(xid),=20xid);=0A+=09=09=09=09=09=09=09=09=09=09= SerialPage(xid),=20xid,=20true);=0A=20=09val=20=3D=20SerialValue(slotno,=20= xid);=0A=20=09LWLockRelease(SerialSLRULock);=0A=20=09return=20val;=0A= diff=20--git=20a/src/include/access/clog.h=20b/src/include/access/clog.h=0A= index=206c840cbf29..cf299cd8f6=20100644=0A---=20= a/src/include/access/clog.h=0A+++=20b/src/include/access/clog.h=0A@@=20= -11,24=20+11,11=20@@=0A=20#ifndef=20CLOG_H=0A=20#define=20CLOG_H=0A=20=0A= +#include=20"access/clogdefs.h"=0A=20#include=20"access/xlogreader.h"=0A=20= #include=20"storage/sync.h"=0A=20#include=20"lib/stringinfo.h"=0A=20=0A= -/*=0A-=20*=20Possible=20transaction=20statuses=20---=20note=20that=20= all-zeroes=20is=20the=20initial=0A-=20*=20state.=0A-=20*=0A-=20*=20A=20= "subcommitted"=20transaction=20is=20a=20committed=20subtransaction=20= whose=20parent=0A-=20*=20hasn't=20committed=20or=20aborted=20yet.=0A-=20= */=0A-typedef=20int=20XidStatus;=0A-=0A-#define=20= TRANSACTION_STATUS_IN_PROGRESS=09=090x00=0A-#define=20= TRANSACTION_STATUS_COMMITTED=09=090x01=0A-#define=20= TRANSACTION_STATUS_ABORTED=09=09=090x02=0A-#define=20= TRANSACTION_STATUS_SUB_COMMITTED=090x03=0A-=0A=20typedef=20struct=20= xl_clog_truncate=0A=20{=0A=20=09int=09=09=09pageno;=0A@@=20-38,7=20+25,8=20= @@=20typedef=20struct=20xl_clog_truncate=0A=20=0A=20extern=20void=20= TransactionIdSetTreeStatus(TransactionId=20xid,=20int=20nsubxids,=0A=20=09= =09=09=09=09=09=09=09=09=20=20=20TransactionId=20*subxids,=20XidStatus=20= status,=20XLogRecPtr=20lsn);=0A-extern=20XidStatus=20= TransactionIdGetStatus(TransactionId=20xid,=20XLogRecPtr=20*lsn);=0A= +extern=20XidStatus=20TransactionIdGetStatus(TransactionId=20xid,=20= XLogRecPtr=20*lsn,=0A+=09=09=09=09=09=09=09=09=09=09bool=20throwError);=0A= =20=0A=20extern=20Size=20CLOGShmemBuffers(void);=0A=20extern=20Size=20= CLOGShmemSize(void);=0Adiff=20--git=20a/src/include/access/clogdefs.h=20= b/src/include/access/clogdefs.h=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..0f9996bb08=0A---=20/dev/null=0A+++=20= b/src/include/access/clogdefs.h=0A@@=20-0,0=20+1,33=20@@=0A+/*=0A+=20*=20= clogdefs.h=0A+=20*=0A+=20*=20PostgreSQL=20transaction-commit-log=20= manager=0A+=20*=0A+=20*=20Portions=20Copyright=20(c)=201996-2020,=20= PostgreSQL=20Global=20Development=20Group=0A+=20*=20Portions=20Copyright=20= (c)=201994,=20Regents=20of=20the=20University=20of=20California=0A+=20*=0A= +=20*=20src/include/access/clogdefs.h=0A+=20*/=0A+#ifndef=20CLOGDEFS_H=0A= +#define=20CLOGDEFS_H=0A+=0A+/*=0A+=20*=20Possible=20transaction=20= statuses=20---=20note=20that=20all-zeroes=20is=20the=20initial=0A+=20*=20= state.=0A+=20*=0A+=20*=20A=20"subcommitted"=20transaction=20is=20a=20= committed=20subtransaction=20whose=20parent=0A+=20*=20hasn't=20committed=20= or=20aborted=20yet.=0A+=20*=0A+=20*=20An=20"unknown"=20status=20= indicates=20an=20error=20condition,=20such=20as=20when=20the=20clog=20= has=0A+=20*=20been=20erroneously=20truncated=20and=20the=20commit=20= status=20of=20a=20transaction=20cannot=20be=0A+=20*=20determined.=0A+=20= */=0A+typedef=20enum=20XidStatus=20{=0A+=09= TRANSACTION_STATUS_IN_PROGRESS=20=3D=200x00,=0A+=09= TRANSACTION_STATUS_COMMITTED=20=3D=200x01,=0A+=09= TRANSACTION_STATUS_ABORTED=20=3D=200x02,=0A+=09= TRANSACTION_STATUS_SUB_COMMITTED=20=3D=200x03,=0A+=09= TRANSACTION_STATUS_UNKNOWN=20=3D=200x04=09/*=20error=20condition=20*/=0A= +}=20XidStatus;=0A+=0A+#endif=09=09=09=09=09=09=09/*=20CLOG_H=20*/=0A= diff=20--git=20a/src/include/access/slru.h=20b/src/include/access/slru.h=0A= index=20b39b43504d..0b6a5669d8=20100644=0A---=20= a/src/include/access/slru.h=0A+++=20b/src/include/access/slru.h=0A@@=20= -133,6=20+133,8=20@@=20typedef=20struct=20SlruCtlData=0A=20=0A=20typedef=20= SlruCtlData=20*SlruCtl;=0A=20=0A+#define=20InvalidSlotNo=20((int)=20-1)=0A= +=0A=20=0A=20extern=20Size=20SimpleLruShmemSize(int=20nslots,=20int=20= nlsns);=0A=20extern=20void=20SimpleLruInit(SlruCtl=20ctl,=20const=20char=20= *name,=20int=20nslots,=20int=20nlsns,=0A@@=20-140,9=20+142,9=20@@=20= extern=20void=20SimpleLruInit(SlruCtl=20ctl,=20const=20char=20*name,=20= int=20nslots,=20int=20nlsns,=0A=20=09=09=09=09=09=09=20=20= SyncRequestHandler=20sync_handler);=0A=20extern=20int=09= SimpleLruZeroPage(SlruCtl=20ctl,=20int=20pageno);=0A=20extern=20int=09= SimpleLruReadPage(SlruCtl=20ctl,=20int=20pageno,=20bool=20write_ok,=0A-=09= =09=09=09=09=09=09=20=20TransactionId=20xid);=0A+=09=09=09=09=09=09=09=20= =20TransactionId=20xid,=20bool=20throwError);=0A=20extern=20int=09= SimpleLruReadPage_ReadOnly(SlruCtl=20ctl,=20int=20pageno,=0A-=09=09=09=09= =09=09=09=09=09=20=20=20TransactionId=20xid);=0A+=09=09=09=09=09=09=09=09= =09=20=20=20TransactionId=20xid,=20bool=20throwError);=0A=20extern=20= void=20SimpleLruWritePage(SlruCtl=20ctl,=20int=20slotno);=0A=20extern=20= void=20SimpleLruWriteAll(SlruCtl=20ctl,=20bool=20allow_redirtied);=0A=20= extern=20void=20SimpleLruTruncate(SlruCtl=20ctl,=20int=20cutoffPage);=0A= diff=20--git=20a/src/include/access/transam.h=20= b/src/include/access/transam.h=0Aindex=202f1f144db4..7d5e2f614d=20100644=0A= ---=20a/src/include/access/transam.h=0A+++=20= b/src/include/access/transam.h=0A@@=20-14,6=20+14,7=20@@=0A=20#ifndef=20= TRANSAM_H=0A=20#define=20TRANSAM_H=0A=20=0A+#include=20= "access/clogdefs.h"=0A=20#include=20"access/xlogdefs.h"=0A=20=0A=20=0A@@=20= -264,6=20+265,8=20@@=20extern=20PGDLLIMPORT=20VariableCache=20= ShmemVariableCache;=0A=20/*=0A=20=20*=20prototypes=20for=20functions=20= in=20transam/transam.c=0A=20=20*/=0A+extern=20XidStatus=20= TransactionIdResolveStatus(TransactionId=20transactionId,=0A+=09=09=09=09= =09=09=09=09=09=09=09bool=20throwError);=0A=20extern=20bool=20= TransactionIdDidCommit(TransactionId=20transactionId);=0A=20extern=20= bool=20TransactionIdDidAbort(TransactionId=20transactionId);=0A=20extern=20= bool=20TransactionIdIsKnownCompleted(TransactionId=20transactionId);=0A= --=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479 Content-Disposition: attachment; filename=v17-0004-Using-non-throwing-clog-interface-from-amcheck.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v17-0004-Using-non-throwing-clog-interface-from-amcheck.patch" Content-Transfer-Encoding: quoted-printable =46rom=20574d7cb2d8a169fcd50653ec36a7ad919540d808=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Tue,=206=20Oct=202020=2013:33:23=20-0700=0ASubject:=20[PATCH=20= v17=204/4]=20Using=20non-throwing=20clog=20interface=20from=20amcheck=0A=0A= Converting=20the=20heap=20checking=20functions=20to=20use=20the=20= recently=20introduced=0Anon-throwing=20interface=20to=20clog=20when=20= checking=20transaction=20commit=20status,=20and=0Aadding=20corruption=20= reports=20about=20missing=20clog=20rather=20than=20aborting.=0A---=0A=20= contrib/amcheck/verify_heapam.c=20=20|=2084=20= ++++++++++++++++++++------------=0A=20src/tools/pgindent/typedefs.list=20= |=20=201=20-=0A=202=20files=20changed,=2054=20insertions(+),=2031=20= deletions(-)=0A=0Adiff=20--git=20a/contrib/amcheck/verify_heapam.c=20= b/contrib/amcheck/verify_heapam.c=0Aindex=20b46cce798e..9885a9d55c=20= 100644=0A---=20a/contrib/amcheck/verify_heapam.c=0A+++=20= b/contrib/amcheck/verify_heapam.c=0A@@=20-10,6=20+10,7=20@@=0A=20=20*/=0A= =20#include=20"postgres.h"=0A=20=0A+#include=20"access/clogdefs.h"=0A=20= #include=20"access/detoast.h"=0A=20#include=20"access/genam.h"=0A=20= #include=20"access/heapam.h"=0A@@=20-40,13=20+41,6=20@@=20typedef=20enum=20= XidBoundsViolation=0A=20=09XID_BOUNDS_OK=0A=20}=20XidBoundsViolation;=0A=20= =0A-typedef=20enum=20XidCommitStatus=0A-{=0A-=09XID_COMMITTED,=0A-=09= XID_IN_PROGRESS,=0A-=09XID_ABORTED=0A-}=20XidCommitStatus;=0A-=0A=20= typedef=20enum=20SkipPages=0A=20{=0A=20=09SKIP_PAGES_ALL_FROZEN,=0A@@=20= -80,7=20+74,7=20@@=20typedef=20struct=20HeapCheckContext=0A=20=09=20*=20= Cached=20copies=20of=20the=20most=20recently=20checked=20xid=20and=20its=20= status.=0A=20=09=20*/=0A=20=09TransactionId=20cached_xid;=0A-=09= XidCommitStatus=20cached_status;=0A+=09XidStatus=09cached_status;=0A=20=0A= =20=09/*=20Values=20concerning=20the=20heap=20relation=20being=20checked=20= */=0A=20=09Relation=09rel;=0A@@=20-138,7=20+132,7=20@@=20static=20void=20= update_cached_xid_range(HeapCheckContext=20*ctx);=0A=20static=20void=20= update_cached_mxid_range(HeapCheckContext=20*ctx);=0A=20static=20= XidBoundsViolation=20check_mxid_in_range(MultiXactId=20mxid,=20= HeapCheckContext=20*ctx);=0A=20static=20XidBoundsViolation=20= check_mxid_valid_in_rel(MultiXactId=20mxid,=20HeapCheckContext=20*ctx);=0A= -static=20XidBoundsViolation=20get_xid_status(TransactionId=20xid,=20= HeapCheckContext=20*ctx,=20XidCommitStatus=20*status);=0A+static=20= XidBoundsViolation=20get_xid_status(TransactionId=20xid,=20= HeapCheckContext=20*ctx,=20XidStatus=20*status);=0A=20=0A=20/*=0A=20=20*=20= Scan=20and=20report=20corruption=20in=20heap=20pages,=20optionally=20= reconciling=20toasted=0A@@=20-642,7=20+636,7=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09else=20if=20(infomask=20&=20= HEAP_MOVED_OFF=20||=0A=20=09=09=09=09=20infomask=20&=20HEAP_MOVED_IN)=0A=20= =09=09{=0A-=09=09=09XidCommitStatus=20status;=0A+=09=09=09XidStatus=09= status;=0A=20=09=09=09TransactionId=20xvac=20=3D=20= HeapTupleHeaderGetXvac(tuphdr);=0A=20=0A=20=09=09=09switch=20= (get_xid_status(xvac,=20ctx,=20&status))=0A@@=20-686,17=20+680,27=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09=09=09case=20XID_BOUNDS_OK:=0A=20=09=09= =09=09=09switch=20(status)=0A=20=09=09=09=09=09{=0A-=09=09=09=09=09=09= case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_IN_PROGRESS:=0A=20=09=09=09=09=09=09=09return=20true;=09= /*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A-=09=09=09=09=09=09case=20= XID_COMMITTED:=0A-=09=09=09=09=09=09case=20XID_ABORTED:=0A+=09=09=09=09=09= =09case=20TRANSACTION_STATUS_COMMITTED:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_ABORTED:=0A=20=09=09=09=09=09=09=09return=20false;=09= /*=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_UNKNOWN:=0A+=09=09=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09/*------=0A+=09=09=09=09=09= =09=09translator:=20%u=20is=20a=20transaction=20identifier=20*/=0A+=09=09= =09=09=09=09=09=09=09=09=09=20=20psprintf(_("old-style=20VACUUM=20FULL=20= transaction=20ID=20%u=20commit=20status=20is=20lost"),=0A+=09=09=09=09=09= =09=09=09=09=09=09=09=09=20=20=20xvac));=0A+=09=09=09=09=09=09=09return=20= false;=09/*=20corruption=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_SUB_COMMITTED:=0A+=09=09=09=09=09=09=09elog(ERROR,=20= "get_xid_status=20failed=20to=20resolve=20parent=20transaction=20= status");=0A+=09=09=09=09=09=09=09return=20false;=09/*=20not=20reached=20= */=0A=20=09=09=09=09=09}=0A=20=09=09=09}=0A=20=09=09}=0A=20=09=09else=0A=20= =09=09{=0A-=09=09=09XidCommitStatus=20status;=0A+=09=09=09XidStatus=09= status;=0A=20=0A=20=09=09=09switch=20(get_xid_status(raw_xmin,=20ctx,=20= &status))=0A=20=09=09=09{=0A@@=20-737,12=20+741,22=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09=09=09case=20XID_BOUNDS_OK:=0A=20=09=09= =09=09=09switch=20(status)=0A=20=09=09=09=09=09{=0A-=09=09=09=09=09=09= case=20XID_COMMITTED:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_COMMITTED:=0A=20=09=09=09=09=09=09=09break;=0A-=09=09=09= =09=09=09case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_IN_PROGRESS:=0A=20=09=09=09=09=09=09=09return=20true;=09= /*=20insert=20or=20delete=20in=20progress=20*/=0A-=09=09=09=09=09=09case=20= XID_ABORTED:=0A+=09=09=09=09=09=09case=20TRANSACTION_STATUS_ABORTED:=0A=20= =09=09=09=09=09=09=09return=20false;=09/*=20HEAPTUPLE_DEAD=20*/=0A+=09=09= =09=09=09=09case=20TRANSACTION_STATUS_UNKNOWN:=0A+=09=09=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09/*------=0A+=09=09=09=09=09= =09=09translator:=20%u=20is=20a=20transaction=20identifier=20*/=0A+=09=09= =09=09=09=09=09=09=09=09=09=20=20psprintf(_("raw=20xmin=20%u=20commit=20= status=20is=20lost"),=0A+=09=09=09=09=09=09=09=09=09=09=09=09=09=20=20=20= raw_xmin));=0A+=09=09=09=09=09=09=09return=20false;=09/*=20corruption=20= */=0A+=09=09=09=09=09=09case=20TRANSACTION_STATUS_SUB_COMMITTED:=0A+=09=09= =09=09=09=09=09elog(ERROR,=20"get_xid_status=20failed=20to=20resolve=20= parent=20transaction=20status");=0A+=09=09=09=09=09=09=09return=20false;=09= /*=20not=20reached=20*/=0A=20=09=09=09=09=09}=0A=20=09=09=09}=0A=20=09=09= }=0A@@=20-752,7=20+766,7=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09{=0A=20=09=09if=20(infomask=20&=20= HEAP_XMAX_IS_MULTI)=0A=20=09=09{=0A-=09=09=09XidCommitStatus=20status;=0A= +=09=09=09XidStatus=09status;=0A=20=09=09=09TransactionId=20xmax=20=3D=20= HeapTupleGetUpdateXid(tuphdr);=0A=20=0A=20=09=09=09switch=20= (get_xid_status(xmax,=20ctx,=20&status))=0A@@=20-795,12=20+809,22=20@@=20= check_tuple_header_and_visibilty(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*ctx)=0A=20=09=09=09=09case=20XID_BOUNDS_OK:=0A=20=09=09= =09=09=09switch=20(status)=0A=20=09=09=09=09=09{=0A-=09=09=09=09=09=09= case=20XID_IN_PROGRESS:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_IN_PROGRESS:=0A=20=09=09=09=09=09=09=09return=20true;=09= /*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A-=09=09=09=09=09=09case=20= XID_COMMITTED:=0A-=09=09=09=09=09=09case=20XID_ABORTED:=0A+=09=09=09=09=09= =09case=20TRANSACTION_STATUS_COMMITTED:=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_ABORTED:=0A=20=09=09=09=09=09=09=09return=20false;=09= /*=20HEAPTUPLE_RECENTLY_DEAD=20or=0A=20=09=09=09=09=09=09=09=09=09=09=09=20= *=20HEAPTUPLE_DEAD=20*/=0A+=09=09=09=09=09=09case=20= TRANSACTION_STATUS_UNKNOWN:=0A+=09=09=09=09=09=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=09/*------=0A+=09=09=09=09=09= =09=09translator:=20%u=20is=20a=20transaction=20identifier=20*/=0A+=09=09= =09=09=09=09=09=09=09=09=09=20=20psprintf(_("xmax=20%u=20commit=20status=20= is=20lost"),=0A+=09=09=09=09=09=09=09=09=09=09=09=09=09=20=20=20xmax));=0A= +=09=09=09=09=09=09=09return=20false;=09/*=20corruption=20*/=0A+=09=09=09= =09=09=09case=20TRANSACTION_STATUS_SUB_COMMITTED:=0A+=09=09=09=09=09=09=09= elog(ERROR,=20"get_xid_status=20failed=20to=20resolve=20parent=20= transaction=20status");=0A+=09=09=09=09=09=09=09return=20false;=09/*=20= not=20reached=20*/=0A=20=09=09=09=09=09}=0A=20=09=09=09}=0A=20=0A@@=20= -1260,7=20+1284,11=20@@=20check_tuple(HeapCheckContext=20*ctx)=0A=20=09=09= =09=09break;=0A=20=09=09}=0A=20=09}=0A-=09/*=20If=20xmax=20is=20not=20a=20= multixact=20and=20is=20normal,=20it=20should=20be=20within=20valid=20= range=20*/=0A+=0A+=09/*=0A+=09=20*=20If=20xmax=20is=20not=20a=20= multixact=20and=20is=20normal,=20it=20should=20be=20within=20valid=0A+=09= =20*=20range=0A+=09=20*/=0A=20=09else=0A=20=09{=0A=20=09=09switch=20= (get_xid_status(xmax,=20ctx,=20NULL))=0A@@=20-1465,7=20+1493,7=20@@=20= check_mxid_valid_in_rel(MultiXactId=20mxid,=20HeapCheckContext=20*ctx)=0A= =20=20*=20status=20argument=20will=20be=20set=20with=20the=20status=20of=20= the=20transaction=20ID.=0A=20=20*/=0A=20static=20XidBoundsViolation=0A= -get_xid_status(TransactionId=20xid,=20HeapCheckContext=20*ctx,=20= XidCommitStatus=20*status)=0A+get_xid_status(TransactionId=20xid,=20= HeapCheckContext=20*ctx,=20XidStatus=20*status)=0A=20{=0A=20=09= XidBoundsViolation=20result;=0A=20=09FullTransactionId=20fxid;=0A@@=20= -1516,19=20+1544,15=20@@=20get_xid_status(TransactionId=20xid,=20= HeapCheckContext=20*ctx,=20XidCommitStatus=20*status=0A=20=09=09return=20= result;=0A=20=09}=0A=20=0A-=09*status=20=3D=20XID_COMMITTED;=0A+=09= *status=20=3D=20TRANSACTION_STATUS_COMMITTED;=0A=20=09= LWLockAcquire(XactTruncationLock,=20LW_SHARED);=0A=20=09clog_horizon=20=3D= =20FullTransactionIdFromXidAndCtx(ShmemVariableCache->oldestClogXid,=20= ctx);=0A=20=09if=20(FullTransactionIdPrecedesOrEquals(clog_horizon,=20= fxid))=0A=20=09{=0A=20=09=09if=20= (TransactionIdIsCurrentTransactionId(xid))=0A-=09=09=09*status=20=3D=20= XID_IN_PROGRESS;=0A-=09=09else=20if=20(TransactionIdDidCommit(xid))=0A-=09= =09=09*status=20=3D=20XID_COMMITTED;=0A-=09=09else=20if=20= (TransactionIdDidAbort(xid))=0A-=09=09=09*status=20=3D=20XID_ABORTED;=0A= +=09=09=09*status=20=3D=20TRANSACTION_STATUS_IN_PROGRESS;=0A=20=09=09= else=0A-=09=09=09*status=20=3D=20XID_IN_PROGRESS;=0A+=09=09=09*status=20= =3D=20TransactionIdResolveStatus(xid,=20false);=0A=20=09}=0A=20=09= LWLockRelease(XactTruncationLock);=0A=20=09ctx->cached_xid=20=3D=20xid;=0A= diff=20--git=20a/src/tools/pgindent/typedefs.list=20= b/src/tools/pgindent/typedefs.list=0Aindex=20369b8e7c6f..6ca1bac21a=20= 100644=0A---=20a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-2793,7=20+2793,6=20@@=20= XactCallbackItem=0A=20XactEvent=0A=20XactLockTableWaitInfo=0A=20= XidBoundsViolation=0A-XidCommitStatus=0A=20XidHorizonPrefetchState=0A=20= XidStatus=0A=20XmlExpr=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 =E2=80=94 Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --Apple-Mail=_8FC41164-396D-49DA-A8F1-39272E327479--