Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQahn-0005X6-JX for pgsql-hackers@arkaria.postgresql.org; Mon, 20 Apr 2020 17:59:40 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jQahm-0006GG-8U for pgsql-hackers@arkaria.postgresql.org; Mon, 20 Apr 2020 17:59:38 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQahl-0006G9-Og for pgsql-hackers@lists.postgresql.org; Mon, 20 Apr 2020 17:59:38 +0000 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1jQahg-00082O-CP for pgsql-hackers@postgresql.org; Mon, 20 Apr 2020 17:59:36 +0000 Received: by mail-pj1-x1031.google.com with SMTP id hi11so197085pjb.3 for ; Mon, 20 Apr 2020 10:59:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=from:mime-version:subject:message-id:date:cc:to; bh=mPQ47u2tNPrbXCy86ecDFzThKiZgbm2Qph3WnRpUfik=; b=XW1iW0Pjne+/47WrS4f23d10SZ+FhtRey+0xC/hk6ToPmShHpoiyCke37qy4r45u+4 caJuR9fRWOh1DgD4ESR9FENVUJrx1QJWgJI71emdq5XI+JqNal1s1KUbXiMYJtDCwPZU XF5b6Xd02YRtx9jERh/K2g5NjiWsk4ro0GRiRvJCERpQ1CO5IbTKLXmShb8T16uyCu6B ykdjgabjfKdSCfMWs9BkkEPkQ1POvS5bwM3UrcJEs5MddYege5uKFhPmqD5fjlAy2pAF TwwG4/9h0n48xSywlQO1M9666q7hem2zByOIInYlr7eSKhdO4cRcWCV6bBWgkIhhHsIw GzbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:cc:to; bh=mPQ47u2tNPrbXCy86ecDFzThKiZgbm2Qph3WnRpUfik=; b=AOolTKwacAqbqqxqQode8LINlbW8Fljb0v6ozvjvB0UK6IkZGFXCn7bOJ0cFRf3UMw 03rJp5TBKuGxB3KCQboGXf9u0J8LWkdW0+LResyekjQ4N2Mb1SqGVIw4zGp/N5AKhDVe z+4o/Q0KpZOpxP0ibvOFuObte5XHDpsAI2lY8f7MdR+pFvXAOjl2OCkEsG7qE+XShRjX BFa5N3Xoed2QStQvVsg56HeLnbQcZOcki/3vmogWKeS+vXw7hWiFT8Ua9a/GRAI8hiK6 hF6H+eW/KEGm+lCZvyedth560MZRiIM8QAi7Pw03HPBS8d1agkNuttEaBc1PUb1uAXc2 /1dg== X-Gm-Message-State: AGi0PuZH0nU4uhXa2v4Cg2Tr7xVF2a6CBbGLGivZm7B7OowvSvU9O4r8 fnT/hdW5QeHuOYncxgGz1Gb79l3jukhyMX2YnTB2gD05Nd/u2PWkC6gAIhwHUAYNgjvWLXEzOxS ihzacgXji5ob2tSSGt4dAWhaqBNGiqsmtLGnClCVvgnpUJRbvoUh9SzizE1ocCELqT9jQnHyzot y3728GlJNEwcLw9P8FUVS4xPRMbuTFRpllFfdpkQ7jatywzJSCxQ2nJMEY1ajSD2FmOPzajL6wp 8TLrnWOcXH1zJk4GKAmWl94gYBahz9Lp8qV X-Google-Smtp-Source: APiQypI9Nr4ND4yICpnxfLjxlSHd95Gp/QkSwfoKDmwnTUBLPKbLkhNnStRj9NrkmSyFQbplwG2w5w== X-Received: by 2002:a17:90a:db91:: with SMTP id h17mr659957pjv.7.1587405569645; Mon, 20 Apr 2020 10:59:29 -0700 (PDT) Received: from [192.168.0.29] (24-113-193-150.wavecable.com. [24.113.193.150]) by smtp.gmail.com with ESMTPSA id 1sm113194pff.151.2020.04.20.10.59.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Apr 2020 10:59:28 -0700 (PDT) From: Mark Dilger Content-Type: multipart/mixed; boundary="Apple-Mail=_4BC3E30B-3A68-4D41-9C2D-C21A8D3F28D4" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: new heapcheck contrib module Message-Id: <12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com> Date: Mon, 20 Apr 2020 10:59:28 -0700 Cc: Robert Haas To: PostgreSQL-development X-Mailer: Apple Mail (2.3608.80.23.2.2) X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --Apple-Mail=_4BC3E30B-3A68-4D41-9C2D-C21A8D3F28D4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hackers, I have been talking with Robert about table corruption that occurs from = time to time. The page checksum feature seems sufficient to detect most = random corruption problems, but it can't detect "logical" corruption, = where the page is valid but inconsistent with the rest of the database = cluster. This can happen due to faulty or ill-conceived backup and = restore tools, or bad storage, or user error, or bugs in the server = itself. (Also, not everyone enables checksums.) The attached module provides the means to scan a relation and sanity = check it. Currently, it checks xmin and xmax values against relfrozenxid = and relminmxid, and also validates TOAST pointers. If people like this, = it could be expanded to perform additional checks. There was a prior v1 patch, discussed offlist with Robert, not posted. = Here is v2: --Apple-Mail=_4BC3E30B-3A68-4D41-9C2D-C21A8D3F28D4 Content-Disposition: attachment; filename=v2-0001-Adding-heapcheck-contrib-module.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v2-0001-Adding-heapcheck-contrib-module.patch" Content-Transfer-Encoding: quoted-printable =46rom=202a1bc0bb9fa94bd929adc1a408900cb925ebcdd5=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Mon,=2020=20Apr=202020=2008:05:58=20-0700=0ASubject:=20[PATCH=20= v2]=20Adding=20heapcheck=20contrib=20module.=0A=0AThe=20heapcheck=20= module=20introduces=20a=20new=20function=20for=20checking=20a=20heap=0A= relation=20and=20associated=20toast=20relation,=20if=20any,=20for=20= corruption.=0A=0AThe=20postgres=20backend=20already=20defends=20against=20= certain=20forms=20of=0Acorruption,=20by=20checking=20the=20page=20header=20= of=20each=20page=20before=20allowing=0Ait=20into=20the=20page=20cache,=20= and=20by=20checking=20the=20page=20checksum,=20if=20enabled.=0A= Experience=20shows=20that=20broken=20or=20ill-conceived=20backup=20and=20= restore=0Amechanisms=20can=20result=20in=20a=20page,=20or=20an=20entire=20= file,=20being=20overwritten=0Awith=20an=20earlier=20version=20of=20= itself,=20restored=20from=20backup.=20=20Pages=20thus=0Aoverwritten=20= will=20appear=20to=20have=20valid=20page=20headers=20and=20checksums,=0A= while=20potentially=20containing=20xmin,=20xmax,=20and=20toast=20= pointers=20that=20are=0Ainvalid.=0A=0Acontrib/heapcheck=20introduces=20a=20= function,=20heapcheck_relation,=20that=0Atakes=20a=20regclass=20= argument,=20scans=20the=20given=20heap=20relation,=20and=20returns=0A= rows=20containing=20information=20about=20corruption=20found=20within=20= the=20table.=0AThe=20main=20focus=20of=20the=20scan=20is=20to=20find=20= invalid=20xmin,=20xmax,=20and=20toast=0Apointer=20values.=20=20It=20also=20= checks=20for=20structural=20corruption=20within=20the=0Apage=20(such=20= as=20invalid=20t_hoff=20values)=20that=20could=20lead=20to=20the=20= backend=0Aaborting=20should=20the=20function=20blindly=20trust=20the=20= data=20as=20it=20finds=20it.=0A---=0A=20contrib/Makefile=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=201=20+=0A=20contrib/heapcheck/.gitignore=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20|=20=20=20=204=20+=0A=20= contrib/heapcheck/Makefile=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20=2025=20+=0A=20.../expected/001_create_extension.out=20= =20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= .../expected/002_disallowed_reltypes.out=20=20=20=20=20=20|=20=20=2027=20= +=0A=20contrib/heapcheck/heapcheck--1.0.sql=20=20=20=20=20=20=20=20=20=20= |=20=20=2021=20+=0A=20contrib/heapcheck/heapcheck.c=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20|=201167=20+++++++++++++++++=0A=20= contrib/heapcheck/heapcheck.control=20=20=20=20=20=20=20=20=20=20=20|=20=20= =20=205=20+=0A=20.../heapcheck/sql/001_create_extension.sql=20=20=20=20|=20= =20=20=201=20+=0A=20.../heapcheck/sql/002_disallowed_reltypes.sql=20|=20=20= =2029=20+=0A=20contrib/heapcheck/t/003_heapcheck_relation.pl=20|=20=20= 361=20+++++=0A=20doc/src/sgml/contrib.sgml=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= doc/src/sgml/filelist.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20|=20=20=20=201=20+=0A=20doc/src/sgml/heapcheck.sgml=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20133=20++=0A=2014=20= files=20changed,=201777=20insertions(+)=0A=20create=20mode=20100644=20= contrib/heapcheck/.gitignore=0A=20create=20mode=20100644=20= contrib/heapcheck/Makefile=0A=20create=20mode=20100644=20= contrib/heapcheck/expected/001_create_extension.out=0A=20create=20mode=20= 100644=20contrib/heapcheck/expected/002_disallowed_reltypes.out=0A=20= create=20mode=20100644=20contrib/heapcheck/heapcheck--1.0.sql=0A=20= create=20mode=20100644=20contrib/heapcheck/heapcheck.c=0A=20create=20= mode=20100644=20contrib/heapcheck/heapcheck.control=0A=20create=20mode=20= 100644=20contrib/heapcheck/sql/001_create_extension.sql=0A=20create=20= mode=20100644=20contrib/heapcheck/sql/002_disallowed_reltypes.sql=0A=20= create=20mode=20100644=20contrib/heapcheck/t/003_heapcheck_relation.pl=0A= =20create=20mode=20100644=20doc/src/sgml/heapcheck.sgml=0A=0Adiff=20= --git=20a/contrib/Makefile=20b/contrib/Makefile=0Aindex=20= 1846d415b6..27ac131526=20100644=0A---=20a/contrib/Makefile=0A+++=20= b/contrib/Makefile=0A@@=20-20,6=20+20,7=20@@=20SUBDIRS=20=3D=20\=0A=20=09= =09earthdistance=09\=0A=20=09=09file_fdw=09\=0A=20=09=09fuzzystrmatch=09= \=0A+=09=09heapcheck=20\=0A=20=09=09hstore=09=09\=0A=20=09=09intagg=09=09= \=0A=20=09=09intarray=09\=0Adiff=20--git=20= a/contrib/heapcheck/.gitignore=20b/contrib/heapcheck/.gitignore=0Anew=20= file=20mode=20100644=0Aindex=200000000000..5dcb3ff972=0A---=20/dev/null=0A= +++=20b/contrib/heapcheck/.gitignore=0A@@=20-0,0=20+1,4=20@@=0A+#=20= Generated=20subdirectories=0A+/log/=0A+/results/=0A+/tmp_check/=0Adiff=20= --git=20a/contrib/heapcheck/Makefile=20b/contrib/heapcheck/Makefile=0A= new=20file=20mode=20100644=0Aindex=200000000000..8d780a41ab=0A---=20= /dev/null=0A+++=20b/contrib/heapcheck/Makefile=0A@@=20-0,0=20+1,25=20@@=0A= +#=20contrib/heapcheck/Makefile=0A+=0A+MODULE_big=20=3D=20heapcheck=0A= +OBJS=20=3D=20\=0A+=09$(WIN32RES)=20\=0A+=09heapcheck.o=0A+=0A+EXTENSION=20= =3D=20heapcheck=0A+DATA=20=3D=20heapcheck--1.0.sql=0A+PGFILEDESC=20=3D=20= "heapcheck=20-=20page=20corruption=20information"=0A+=0A+REGRESS=20=3D=20= 001_create_extension=20002_disallowed_reltypes=0A+=0A+TAP_TESTS=20=3D=20= 1=0A+=0A+ifdef=20USE_PGXS=0A+PG_CONFIG=20=3D=20pg_config=0A+PGXS=20:=3D=20= $(shell=20$(PG_CONFIG)=20--pgxs)=0A+include=20$(PGXS)=0A+else=0A+subdir=20= =3D=20contrib/heapcheck=0A+top_builddir=20=3D=20../..=0A+include=20= $(top_builddir)/src/Makefile.global=0A+include=20= $(top_srcdir)/contrib/contrib-global.mk=0A+endif=0Adiff=20--git=20= a/contrib/heapcheck/expected/001_create_extension.out=20= b/contrib/heapcheck/expected/001_create_extension.out=0Anew=20file=20= mode=20100644=0Aindex=200000000000..0ca79c22be=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/expected/001_create_extension.out=0A@@=20-0,0=20+1=20= @@=0A+create=20extension=20heapcheck;=0Adiff=20--git=20= a/contrib/heapcheck/expected/002_disallowed_reltypes.out=20= b/contrib/heapcheck/expected/002_disallowed_reltypes.out=0Anew=20file=20= mode=20100644=0Aindex=200000000000..8e0b18dfc3=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/expected/002_disallowed_reltypes.out=0A@@=20-0,0=20= +1,27=20@@=0A+--=0A+--=20check=20that=20using=20the=20module's=20= functions=20with=20unsupported=20relations=20will=20fail=0A+--=0A+--=20= partitioned=20tables=20(the=20parent=20ones)=20don't=20have=20visibility=20= maps=0A+create=20table=20test_partitioned=20(a=20int,=20b=20text=20= default=20repeat('x',=205000))=20partition=20by=20list=20(a);=0A+--=20= these=20should=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_partitioned');=0A+ERROR:=20=20= "test_partitioned"=20is=20not=20a=20table,=20materialized=20view,=20or=20= TOAST=20table=0A+create=20table=20test_partition=20partition=20of=20= test_partitioned=20for=20values=20in=20(1);=0A+create=20index=20= test_index=20on=20test_partition=20(a);=0A+--=20indexes=20do=20not,=20so=20= these=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_index');=0A+ERROR:=20=20"test_index"=20is=20not=20= a=20table,=20materialized=20view,=20or=20TOAST=20table=0A+create=20view=20= test_view=20as=20select=201;=0A+--=20views=20do=20not=20have=20vms,=20so=20= these=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_view');=0A+ERROR:=20=20"test_view"=20is=20not=20= a=20table,=20materialized=20view,=20or=20TOAST=20table=0A+create=20= sequence=20test_sequence;=0A+--=20sequences=20do=20not=20have=20vms,=20= so=20these=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_sequence');=0A+ERROR:=20=20"test_sequence"=20is=20= not=20a=20table,=20materialized=20view,=20or=20TOAST=20table=0A+create=20= foreign=20data=20wrapper=20dummy;=0A+create=20server=20dummy_server=20= foreign=20data=20wrapper=20dummy;=0A+create=20foreign=20table=20= test_foreign_table=20()=20server=20dummy_server;=0A+--=20foreign=20= tables=20do=20not=20have=20vms,=20so=20these=20all=20fail=0A+select=20*=20= from=20heapcheck_relation('test_foreign_table');=0A+ERROR:=20=20= "test_foreign_table"=20is=20not=20a=20table,=20materialized=20view,=20or=20= TOAST=20table=0Adiff=20--git=20a/contrib/heapcheck/heapcheck--1.0.sql=20= b/contrib/heapcheck/heapcheck--1.0.sql=0Anew=20file=20mode=20100644=0A= index=200000000000..48251e6781=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/heapcheck--1.0.sql=0A@@=20-0,0=20+1,21=20@@=0A+/*=20= contrib/heapcheck/heapcheck--1.0.sql=20*/=0A+=0A+--=20complain=20if=20= script=20is=20sourced=20in=20psql,=20rather=20than=20via=20CREATE=20= EXTENSION=0A+\echo=20Use=20"CREATE=20EXTENSION=20heapcheck"=20to=20load=20= this=20file.=20\quit=0A+=0A+--=20Show=20visibility=20map=20and=20= page-level=20visibility=20information=20for=20each=20block.=0A+CREATE=20= FUNCTION=20heapcheck_relation(regclass,=0A+=09=09=09=09=09=09=09=09=20=20= blkno=20OUT=20bigint,=0A+=09=09=09=09=09=09=09=09=20=20offnum=20OUT=20= integer,=0A+=09=09=09=09=09=09=09=09=20=20lp_off=20OUT=20smallint,=0A+=09= =09=09=09=09=09=09=09=20=20lp_flags=20OUT=20smallint,=0A+=09=09=09=09=09=09= =09=09=20=20lp_len=20OUT=20smallint,=0A+=09=09=09=09=09=09=09=09=20=20= attnum=20OUT=20integer,=0A+=09=09=09=09=09=09=09=09=20=20chunk=20OUT=20= integer,=0A+=09=09=09=09=09=09=09=09=20=20msg=20OUT=20text=0A+=09=09=09=09= =09=09=09=09=20=20)=0A+RETURNS=20SETOF=20record=0A+AS=20= 'MODULE_PATHNAME',=20'heapcheck_relation'=0A+LANGUAGE=20C=20STRICT;=0A= +REVOKE=20ALL=20ON=20FUNCTION=20heapcheck_relation(regclass)=20FROM=20= PUBLIC;=0A+GRANT=20EXECUTE=20ON=20FUNCTION=20= heapcheck_relation(regclass)=20TO=20pg_stat_scan_tables;=0Adiff=20--git=20= a/contrib/heapcheck/heapcheck.c=20b/contrib/heapcheck/heapcheck.c=0Anew=20= file=20mode=20100644=0Aindex=200000000000..7cd4690f98=0A---=20/dev/null=0A= +++=20b/contrib/heapcheck/heapcheck.c=0A@@=20-0,0=20+1,1167=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20heapcheck.c=0A+=20*=09=20=20Functions=20to=20check=20= postgresql=20relations=20for=20corruption=0A+=20*=0A+=20*=20Copyright=20= (c)=202016-2020,=20PostgreSQL=20Global=20Development=20Group=0A+=20*=0A+=20= *=09=20=20contrib/heapcheck/heapcheck.c=0A+=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres.h"=0A+=0A+#include=20= "access/detoast.h"=0A+#include=20"access/genam.h"=0A+#include=20= "access/heapam.h"=0A+#include=20"access/heaptoast.h"=0A+#include=20= "access/htup_details.h"=0A+#include=20"access/multixact.h"=0A+#include=20= "access/toast_internals.h"=0A+#include=20"access/visibilitymap.h"=0A= +#include=20"access/xact.h"=0A+#include=20"catalog/pg_am.h"=0A+#include=20= "catalog/pg_type.h"=0A+#include=20"catalog/storage_xlog.h"=0A+#include=20= "funcapi.h"=0A+#include=20"miscadmin.h"=0A+#include=20"storage/bufmgr.h"=0A= +#include=20"storage/procarray.h"=0A+#include=20"storage/smgr.h"=0A= +#include=20"utils/builtins.h"=0A+#include=20"utils/fmgroids.h"=0A= +#include=20"utils/lsyscache.h"=0A+#include=20"utils/rel.h"=0A+#include=20= "utils/snapmgr.h"=0A+#include=20"utils/syscache.h"=0A+=0A= +PG_MODULE_MAGIC;=0A+=0A+PG_FUNCTION_INFO_V1(heapcheck_relation);=0A+=0A= +typedef=20struct=20CorruptionInfo=0A+{=0A+=09BlockNumber=20blkno;=0A+=09= OffsetNumber=20offnum;=0A+=09int16=09=09lp_off;=0A+=09int16=09=09= lp_flags;=0A+=09int16=09=09lp_len;=0A+=09int32=09=09attnum;=0A+=09int32=09= =09chunk;=0A+=09char=09=20=20=20*msg;=0A+}=09=09=09CorruptionInfo;=0A+=0A= +typedef=20struct=20HeapCheckContext=0A+{=0A+=09/*=20Values=20concerning=20= the=20heap=20relation=20being=20checked=20*/=0A+=09Oid=09=09=09relid;=0A= +=09Relation=09rel;=0A+=09TupleDesc=09relDesc;=0A+=09TransactionId=20= relfrozenxid;=0A+=09MultiXactId=20relminmxid;=0A+=09int=09=09=09= rel_natts;=0A+=09bool=09=09has_toastrel;=0A+=09Relation=09toastrel;=0A+=09= Relation=20=20=20*toast_indexes;=0A+=09Relation=09valid_toast_index;=0A+=09= int=09=09=09num_toast_indexes;=0A+=0A+=09/*=20Values=20for=20iterating=20= over=20pages=20in=20the=20relation=20*/=0A+=09BlockNumber=20nblocks;=0A+=09= BlockNumber=20blkno;=0A+=09BufferAccessStrategy=20bstrategy;=0A+=09= Buffer=09=09buffer;=0A+=09Page=09=09page;=0A+=0A+=09/*=20Values=20for=20= iterating=20over=20tuples=20within=20a=20page=20*/=0A+=09OffsetNumber=20= offnum;=0A+=09OffsetNumber=20maxoff;=0A+=09ItemId=09=09itemid;=0A+=09= uint16=09=09lp_len;=0A+=09HeapTupleHeader=20tuphdr;=0A+=09TransactionId=20= xmin;=0A+=09TransactionId=20xmax;=0A+=09uint16=09=09infomask;=0A+=09int=09= =09=09natts;=0A+=09bool=09=09hasnulls;=0A+=0A+=09/*=20Values=20for=20= iterating=20over=20attributes=20within=20the=20tuple=20*/=0A+=09uint32=09= =09offset;=09=09=09/*=20offset=20in=20tuple=20data=20*/=0A+=09AttrNumber=09= attnum;=0A+=09char=09=20=20=20*tp;=09=09=09=09/*=20pointer=20to=20the=20= tuple=20data=20*/=0A+=09bits8=09=20=20=20*bp;=09=09=09=09/*=20ptr=20to=20= null=20bitmap=20in=20tuple=20*/=0A+=09Form_pg_attribute=20thisatt;=0A+=0A= +=09/*=20Values=20for=20iterating=20over=20toast=20for=20the=20attribute=20= */=0A+=09ScanKeyData=20toastkey;=0A+=09SysScanDesc=20toastscan;=0A+=09= SnapshotData=20SnapshotToast;=0A+=09int32=09=09chunkno;=0A+=09HeapTuple=09= toasttup;=0A+=09int32=09=09attrsize;=0A+=09int32=09=09endchunk;=0A+=09= int32=09=09totalchunks;=0A+=09TupleDesc=09toasttupDesc;=0A+=09bool=09=09= found_toasttup;=0A+=0A+=09/*=20List=20of=20CorruptionInfo=20*/=0A+=09= List=09=20=20=20*corruption;=0A+}=09=09=09HeapCheckContext;=0A+=0A+/*=20= Public=20API=20*/=0A+typedef=20struct=20CheckRelCtx=0A+{=0A+=09List=09=20= =20=20*corruption;=0A+=09int=09=09=09idx;=0A+}=09=09=09CheckRelCtx;=0A+=0A= +Datum=09=09heapcheck_relation(PG_FUNCTION_ARGS);=0A+=0A+/*=20Internal=20= implementation=20*/=0A+void=09=09record_corruption(HeapCheckContext=20*=20= ctx,=20char=20*msg);=0A+TupleDesc=09heapcheck_relation_tupdesc(void);=0A= +=0A+void=09=09beginRelBlockIteration(HeapCheckContext=20*=20ctx);=0A= +bool=09=09relBlockIteration_next(HeapCheckContext=20*=20ctx);=0A+void=09= =09endRelBlockIteration(HeapCheckContext=20*=20ctx);=0A+=0A+void=09=09= beginPageTupleIteration(HeapCheckContext=20*=20ctx);=0A+bool=09=09= pageTupleIteration_next(HeapCheckContext=20*=20ctx);=0A+void=09=09= endPageTupleIteration(HeapCheckContext=20*=20ctx);=0A+=0A+void=09=09= beginTupleAttributeIteration(HeapCheckContext=20*=20ctx);=0A+bool=09=09= tupleAttributeIteration_next(HeapCheckContext=20*=20ctx);=0A+void=09=09= endTupleAttributeIteration(HeapCheckContext=20*=20ctx);=0A+=0A+void=09=09= beginToastTupleIteration(HeapCheckContext=20*=20ctx,=0A+=09=09=09=09=09=09= =09=09=09=20struct=20varatt_external=20*toast_pointer);=0A+void=09=09= endToastTupleIteration(HeapCheckContext=20*=20ctx);=0A+bool=09=09= toastTupleIteration_next(HeapCheckContext=20*=20ctx);=0A+=0A+bool=09=09= TransactionIdStillValid(TransactionId=20xid,=20FullTransactionId=20= *fxid);=0A+bool=09=09HeapTupleIsVisible(HeapTupleHeader=20tuphdr,=20= HeapCheckContext=20*=20ctx);=0A+void=09=09= check_toast_tuple(HeapCheckContext=20*=20ctx);=0A+bool=09=09= check_tuple_attribute(HeapCheckContext=20*=20ctx);=0A+void=09=09= check_tuple(HeapCheckContext=20*=20ctx);=0A+=0A+List=09=20=20=20= *check_relation(Oid=20relid);=0A+void=09=09= check_relation_relkind(Relation=20rel);=0A+=0A+/*=0A+=20*=20= record_corruption=0A+=20*=0A+=20*=20=20=20Record=20a=20message=20about=20= corruption,=20including=20information=0A+=20*=20=20=20about=20where=20in=20= the=20relation=20the=20corruption=20was=20found.=0A+=20*/=0A+void=0A= +record_corruption(HeapCheckContext=20*=20ctx,=20char=20*msg)=0A+{=0A+=09= CorruptionInfo=20*info=20=3D=20(CorruptionInfo=20*)=20= palloc0(sizeof(CorruptionInfo));=0A+=0A+=09info->blkno=20=3D=20= ctx->blkno;=0A+=09info->offnum=20=3D=20ctx->offnum;=0A+=09info->lp_off=20= =3D=20ItemIdGetOffset(ctx->itemid);=0A+=09info->lp_flags=20=3D=20= ItemIdGetFlags(ctx->itemid);=0A+=09info->lp_len=20=3D=20= ItemIdGetLength(ctx->itemid);=0A+=09info->attnum=20=3D=20ctx->attnum;=0A= +=09info->chunk=20=3D=20ctx->chunkno;=0A+=09info->msg=20=3D=20msg;=0A+=0A= +=09ctx->corruption=20=3D=20lappend(ctx->corruption,=20info);=0A+}=0A+=0A= +/*=0A+=20*=20Helper=20function=20to=20construct=20the=20TupleDesc=20= needed=20by=20heapcheck_relation.=0A+=20*/=0A+TupleDesc=0A= +heapcheck_relation_tupdesc()=0A+{=0A+=09TupleDesc=09tupdesc;=0A+=09= AttrNumber=09maxattr=20=3D=208;=0A+=09AttrNumber=09a=20=3D=200;=0A+=0A+=09= tupdesc=20=3D=20CreateTemplateTupleDesc(maxattr);=0A+=09= TupleDescInitEntry(tupdesc,=20++a,=20"blkno",=20INT8OID,=20-1,=200);=0A+=09= TupleDescInitEntry(tupdesc,=20++a,=20"offnum",=20INT4OID,=20-1,=200);=0A= +=09TupleDescInitEntry(tupdesc,=20++a,=20"lp_off",=20INT2OID,=20-1,=20= 0);=0A+=09TupleDescInitEntry(tupdesc,=20++a,=20"lp_flags",=20INT2OID,=20= -1,=200);=0A+=09TupleDescInitEntry(tupdesc,=20++a,=20"lp_len",=20= INT2OID,=20-1,=200);=0A+=09TupleDescInitEntry(tupdesc,=20++a,=20= "attnum",=20INT4OID,=20-1,=200);=0A+=09TupleDescInitEntry(tupdesc,=20= ++a,=20"chunk",=20INT4OID,=20-1,=200);=0A+=09TupleDescInitEntry(tupdesc,=20= ++a,=20"msg",=20TEXTOID,=20-1,=200);=0A+=09Assert(a=20=3D=3D=20maxattr);=0A= +=0A+=09return=20BlessTupleDesc(tupdesc);=0A+}=0A+=0A+/*=0A+=20*=20= heapcheck_relation=0A+=20*=0A+=20*=20=20=20Scan=20and=20report=20= corruption=20in=20heap=20pages=20or=20in=20associated=20toast=20= relation.=0A+=20*/=0A+Datum=0A+heapcheck_relation(PG_FUNCTION_ARGS)=0A+{=0A= +=09FuncCallContext=20*funcctx;=0A+=09CheckRelCtx=20*ctx;=0A+=0A+=09if=20= (SRF_IS_FIRSTCALL())=0A+=09{=0A+=09=09Oid=09=09=09relid=20=3D=20= PG_GETARG_OID(0);=0A+=09=09MemoryContext=20oldcontext;=0A+=0A+=09=09/*=0A= +=09=09=20*=20Scan=20the=20entire=20relation,=20building=20up=20a=20list=20= of=20corruption=20found=20in=0A+=09=09=20*=20ctx->corruption,=20for=20= returning=20later.=20=20The=20scan=20must=20be=20performed=0A+=09=09=20*=20= in=20a=20memory=20context=20that=20will=20survive=20until=20after=20all=20= rows=20are=0A+=09=09=20*=20returned.=0A+=09=09=20*/=0A+=09=09funcctx=20=3D= =20SRF_FIRSTCALL_INIT();=0A+=09=09oldcontext=20=3D=20= MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);=0A+=09=09= funcctx->tuple_desc=20=3D=20heapcheck_relation_tupdesc();=0A+=09=09ctx=20= =3D=20(CheckRelCtx=20*)=20palloc0(sizeof(CheckRelCtx));=0A+=09=09= ctx->corruption=20=3D=20check_relation(relid);=0A+=09=09ctx->idx=20=3D=20= 0;=09=09=09/*=20start=20the=20iterator=20at=20the=20beginning=20*/=0A+=09= =09funcctx->user_fctx=20=3D=20(void=20*)=20ctx;=0A+=09=09= MemoryContextSwitchTo(oldcontext);=0A+=09}=0A+=0A+=09funcctx=20=3D=20= SRF_PERCALL_SETUP();=0A+=09ctx=20=3D=20(CheckRelCtx=20*)=20= funcctx->user_fctx;=0A+=0A+=09/*=0A+=09=20*=20Return=20the=20next=20= corruption=20message=20from=20the=20list,=20if=20any.=20=20Our=20= location=0A+=09=20*=20in=20the=20list=20is=20recorded=20in=20ctx->idx.=20= =20The=20special=20value=20-1=20is=20used=20in=0A+=09=20*=20the=20list=20= of=20corruptions=20to=20represent=20NULL;=20we=20check=20for=20negative=0A= +=09=20*=20numbers=20when=20setting=20the=20nulls[]=20values.=0A+=09=20= */=0A+=09if=20(ctx->idx=20<=20list_length(ctx->corruption))=0A+=09{=0A+=09= =09Datum=09=09values[8];=0A+=09=09bool=09=09nulls[8];=0A+=09=09HeapTuple=09= tuple;=0A+=09=09CorruptionInfo=20*info=20=3D=20list_nth(ctx->corruption,=20= ctx->idx);=0A+=0A+=09=09MemSet(values,=200,=20sizeof(nulls));=0A+=09=09= MemSet(nulls,=200,=20sizeof(nulls));=0A+=09=09values[0]=20=3D=20= Int64GetDatum(info->blkno);=0A+=09=09values[1]=20=3D=20= Int32GetDatum(info->offnum);=0A+=09=09nulls[1]=20=3D=20(info->offnum=20<=20= 0);=0A+=09=09values[2]=20=3D=20Int16GetDatum(info->lp_off);=0A+=09=09= nulls[2]=20=3D=20(info->lp_off=20<=200);=0A+=09=09values[3]=20=3D=20= Int16GetDatum(info->lp_flags);=0A+=09=09nulls[3]=20=3D=20(info->lp_flags=20= <=200);=0A+=09=09values[4]=20=3D=20Int16GetDatum(info->lp_len);=0A+=09=09= nulls[4]=20=3D=20(info->lp_len=20<=200);=0A+=09=09values[5]=20=3D=20= Int32GetDatum(info->attnum);=0A+=09=09nulls[5]=20=3D=20(info->attnum=20<=20= 0);=0A+=09=09values[6]=20=3D=20Int32GetDatum(info->chunk);=0A+=09=09= nulls[6]=20=3D=20(info->chunk=20<=200);=0A+=09=09values[7]=20=3D=20= CStringGetTextDatum(info->msg);=0A+=09=09ctx->idx++;=0A+=0A+=09=09tuple=20= =3D=20heap_form_tuple(funcctx->tuple_desc,=20values,=20nulls);=0A+=09=09= SRF_RETURN_NEXT(funcctx,=20HeapTupleGetDatum(tuple));=0A+=09}=0A+=0A+=09= SRF_RETURN_DONE(funcctx);=0A+}=0A+=0A+/*=0A+=20*=20= beginRelBlockIteration=0A+=20*=0A+=20*=20=20=20For=20the=20given=20heap=20= relation=20being=20checked,=20as=20recorded=20in=20ctx,=20sets=20up=0A+=20= *=20=20=20variables=20for=20iterating=20over=20the=20heap's=20pages.=0A+=20= *=0A+=20*=20=20=20The=20caller=20should=20have=20already=20opened=20the=20= heap=20relation,=20ctx->rel=0A+=20*/=0A+void=0A= +beginRelBlockIteration(HeapCheckContext=20*=20ctx)=0A+{=0A+=09= ctx->nblocks=20=3D=20RelationGetNumberOfBlocks(ctx->rel);=0A+=09= ctx->blkno=20=3D=20InvalidBlockNumber;=0A+=09ctx->bstrategy=20=3D=20= GetAccessStrategy(BAS_BULKREAD);=0A+=09ctx->buffer=20=3D=20= InvalidBuffer;=0A+=09ctx->page=20=3D=20NULL;=0A+}=0A+=0A+/*=0A+=20*=20= endRelBlockIteration=0A+=20*=0A+=20*=20=20=20Releases=20resources=20that=20= were=20reserved=20by=20either=20beginRelBlockIteration=20or=0A+=20*=20=20= =20relBlockIteration_next.=0A+=20*/=0A+void=0A= +endRelBlockIteration(HeapCheckContext=20*=20ctx)=0A+{=0A+=09/*=0A+=09=20= *=20Clean=20up.=20=20If=20the=20caller=20iterated=20to=20the=20end,=20= the=20final=20call=20to=0A+=09=20*=20relBlockIteration_next=20will=20= already=20have=20released=20the=20buffer,=20but=20if=0A+=09=20*=20the=20= caller=20is=20bailing=20out=20early,=20we=20have=20to=20release=20it=20= ourselves.=0A+=09=20*/=0A+=09if=20(InvalidBuffer=20!=3D=20ctx->buffer)=0A= +=09=09UnlockReleaseBuffer(ctx->buffer);=0A+}=0A+=0A+/*=0A+=20*=20= relBlockIteration_next=0A+=20*=0A+=20*=20=20=20Updates=20the=20state=20= in=20ctx=20to=20point=20to=20the=20next=20page=20in=20the=20relation.=0A= +=20*=20=20=20Returns=20true=20if=20there=20is=20any=20such=20page,=20= else=20false.=0A+=20*=0A+=20*=20=20=20The=20caller=20should=20have=20= already=20called=20beginRelBlockIteration,=20and=20should=0A+=20*=20=20=20= only=20continue=20calling=20until=20the=20false=20result.=0A+=20*/=0A= +bool=0A+relBlockIteration_next(HeapCheckContext=20*=20ctx)=0A+{=0A+=09= /*=20We=20must=20unlock=20the=20page=20from=20the=20prior=20iteration,=20= if=20any=20*/=0A+=09Assert(ctx->blkno=20=3D=3D=20InvalidBlockNumber=20||=20= ctx->buffer=20!=3D=20InvalidBuffer);=0A+=09if=20(InvalidBuffer=20!=3D=20= ctx->buffer)=0A+=09{=0A+=09=09UnlockReleaseBuffer(ctx->buffer);=0A+=09=09= ctx->buffer=20=3D=20InvalidBuffer;=0A+=09}=0A+=0A+=09/*=20We=20rely=20on=20= this=20math=20property=20for=20the=20first=20iteration=20*/=0A+=09= StaticAssertStmt(InvalidBlockNumber=20+=201=20=3D=3D=200,=0A+=09=09=09=09= =09=20"InvalidBlockNumber=20increments=20to=20zero");=0A+=09= ctx->blkno++;=0A+=09if=20(ctx->blkno=20>=3D=20ctx->nblocks)=0A+=09=09= return=20false;=0A+=0A+=09/*=20Read=20and=20lock=20the=20next=20page.=20= */=0A+=09ctx->buffer=20=3D=20ReadBufferExtended(ctx->rel,=20= MAIN_FORKNUM,=20ctx->blkno,=0A+=09=09=09=09=09=09=09=09=09=20RBM_NORMAL,=20= ctx->bstrategy);=0A+=09LockBuffer(ctx->buffer,=20BUFFER_LOCK_SHARE);=0A+=09= ctx->page=20=3D=20BufferGetPage(ctx->buffer);=0A+=0A+=09return=20true;=0A= +}=0A+=0A+/*=0A+=20*=20beginPageTupleIteration=0A+=20*=0A+=20*=20=20=20= For=20the=20given=20page=20begin=20visited,=20as=20stored=20in=20ctx,=20= sets=20up=20variables=20for=0A+=20*=20=20=20iterating=20over=20the=20= tuples=20on=20the=20page.=0A+=20*/=0A+void=0A= +beginPageTupleIteration(HeapCheckContext=20*=20ctx)=0A+{=0A+=09/*=20We=20= rely=20on=20this=20math=20property=20for=20the=20first=20iteration=20*/=0A= +=09StaticAssertStmt(InvalidOffsetNumber=20+=201=20=3D=3D=20= FirstOffsetNumber,=0A+=09=09=09=09=09=20"InvalidOffsetNumber=20= increments=20to=20FirstOffsetNumber");=0A+=0A+=09ctx->offnum=20=3D=20= InvalidOffsetNumber;=0A+=09ctx->maxoff=20=3D=20= PageGetMaxOffsetNumber(ctx->page);=0A+=09ctx->itemid=20=3D=20NULL;=0A+=09= ctx->lp_len=20=3D=200;=0A+=09ctx->tuphdr=20=3D=20NULL;=0A+=09ctx->xmin=20= =3D=20InvalidOid;=0A+=09ctx->xmax=20=3D=20InvalidOid;=0A+=09= ctx->infomask=20=3D=200;=0A+=09ctx->natts=20=3D=200;=0A+=09ctx->hasnulls=20= =3D=20false;=0A+}=0A+=0A+/*=0A+=20*=20endPageTupleIteration=0A+=20*=0A+=20= *=20=20=20Releases=20resources=20taken=20by=20beginPageTupleIteration=20= or=0A+=20*=20=20=20pageTupleIteration_next.=0A+=20*/=0A+void=0A= +endPageTupleIteration(HeapCheckContext=20*=20ctx)=0A+{=0A+=09/*=20Abuse=20= beginPageTupleIteration=20to=20reset=20the=20tuple=20iteration=20= variables=20*/=0A+=09beginPageTupleIteration(ctx);=0A+}=0A+=0A+/*=0A+=20= *=20pageTupleIteration_next=0A+=20*=0A+=20*=20=20=20Advances=20the=20= state=20tracked=20in=20ctx=20to=20the=20next=20tuple=20on=20the=20page.=0A= +=20*=0A+=20*=20=20=20Caller=20should=20have=20already=20set=20up=20the=20= iteration=20via=0A+=20*=20=20=20beginPageTupleIteration,=20and=20should=20= stop=20calling=20when=20this=20function=0A+=20*=20=20=20returns=20false.=0A= +=20*/=0A+bool=0A+pageTupleIteration_next(HeapCheckContext=20*=20ctx)=0A= +{=0A+=09/*=0A+=09=20*=20Iterate=20to=20the=20next=20interesting=20line=20= pointer,=20if=20any.=20Unused,=20dead=20and=0A+=09=20*=20redirect=20line=20= pointers=20are=20of=20no=20interest.=0A+=09=20*/=0A+=09do=0A+=09{=0A+=09=09= ctx->offnum=20=3D=20OffsetNumberNext(ctx->offnum);=0A+=09=09if=20= (ctx->offnum=20>=20ctx->maxoff)=0A+=09=09=09return=20false;=0A+=09=09= ctx->itemid=20=3D=20PageGetItemId(ctx->page,=20ctx->offnum);=0A+=09}=20= while=20(!ItemIdIsUsed(ctx->itemid)=20||=0A+=09=09=09=20= ItemIdIsDead(ctx->itemid)=20||=0A+=09=09=09=20= ItemIdIsRedirected(ctx->itemid));=0A+=0A+=09/*=20Set=20up=20context=20= information=20about=20this=20next=20tuple=20*/=0A+=09ctx->lp_len=20=3D=20= ItemIdGetLength(ctx->itemid);=0A+=09ctx->tuphdr=20=3D=20= (HeapTupleHeader)=20PageGetItem(ctx->page,=20ctx->itemid);=0A+=09= ctx->xmin=20=3D=20HeapTupleHeaderGetXmin(ctx->tuphdr);=0A+=09ctx->xmax=20= =3D=20HeapTupleHeaderGetRawXmax(ctx->tuphdr);=0A+=09ctx->infomask=20=3D=20= ctx->tuphdr->t_infomask;=0A+=09ctx->natts=20=3D=20= HeapTupleHeaderGetNatts(ctx->tuphdr);=0A+=09ctx->hasnulls=20=3D=20= ctx->infomask=20&=20HEAP_HASNULL;=0A+=0A+=09/*=0A+=09=20*=20Reset=20= information=20about=20individual=20attributes=20and=20related=20toast=20= values,=0A+=09=20*=20so=20they=20show=20as=20NULL=20in=20the=20= corruption=20report=20if=20we=20record=20a=20corruption=0A+=09=20*=20= before=20beginning=20to=20iterate=20over=20the=20attributes.=0A+=09=20*/=0A= +=09ctx->attnum=20=3D=20-1;=0A+=09ctx->chunkno=20=3D=20-1;=0A+=0A+=09= return=20true;=0A+}=0A+=0A+/*=0A+=20*=20beginTupleAttributeIteration=0A+=20= *=0A+=20*=20=20=20For=20the=20given=20tuple=20begin=20visited,=20as=20= stored=20in=20ctx,=20sets=20up=20variables=20for=0A+=20*=20=20=20= iterating=20over=20the=20attributes=20in=20the=20tuple.=0A+=20*/=0A+void=0A= +beginTupleAttributeIteration(HeapCheckContext=20*=20ctx)=0A+{=0A+=09= ctx->offset=20=3D=200;=0A+=09ctx->attnum=20=3D=20-1;=0A+=09ctx->tp=20=3D=20= (char=20*)=20ctx->tuphdr=20+=20ctx->tuphdr->t_hoff;=0A+=09ctx->bp=20=3D=20= ctx->tuphdr->t_bits;=0A+}=0A+=0A+/*=0A+=20*=20= tupleAttributeIteration_next=0A+=20*=0A+=20*=20=20=20Advances=20the=20= state=20tracked=20in=20ctx=20to=20the=20next=20attribute=20in=20the=20= tuple.=0A+=20*=0A+=20*=20=20=20Caller=20should=20have=20already=20set=20= up=20the=20iteration=20via=0A+=20*=20=20=20beginTupleAttributeIteration,=20= and=20should=20stop=20calling=20when=20this=20function=0A+=20*=20=20=20= returns=20false.=0A+=20*/=0A+bool=0A= +tupleAttributeIteration_next(HeapCheckContext=20*=20ctx)=0A+{=0A+=09= ctx->attnum++;=0A+=09if=20(ctx->attnum=20>=3D=20ctx->natts)=0A+=09=09= return=20false;=0A+=09ctx->thisatt=20=3D=20TupleDescAttr(ctx->relDesc,=20= ctx->attnum);=0A+=09return=20true;=0A+}=0A+=0A+/*=0A+=20*=20= endTupleAttributeIteration=0A+=20*=0A+=20*=20=20=20Resets=20state=20= tracked=20in=20ctx=20after=20iteration=20over=20attributes=20ends.=0A+=20= */=0A+void=0A+endTupleAttributeIteration(HeapCheckContext=20*=20ctx)=0A= +{=0A+=09ctx->offset=20=3D=20-1;=0A+=09ctx->attnum=20=3D=20-1;=0A+}=0A+=0A= +/*=0A+=20*=20beginToastTupleIteration=0A+=20*=0A+=20*=20=20=20For=20the=20= given=20attribute=20begin=20visited,=20as=20stored=20in=20ctx,=20sets=20= up=20variables=20for=0A+=20*=20=20=20iterating=20over=20the=20related=20= toast=20value.=0A+=20*/=0A+void=0A= +beginToastTupleIteration(HeapCheckContext=20*=20ctx,=0A+=09=09=09=09=09=09= =20struct=20varatt_external=20*toast_pointer)=0A+{=0A+=09= ctx->toasttupDesc=20=3D=20ctx->toastrel->rd_att;=0A+=09= ctx->found_toasttup=20=3D=20false;=0A+=0A+=09ctx->attrsize=20=3D=20= toast_pointer->va_extsize;=0A+=09ctx->endchunk=20=3D=20(ctx->attrsize=20= -=201)=20/=20TOAST_MAX_CHUNK_SIZE;=0A+=09ctx->totalchunks=20=3D=20= ctx->endchunk=20+=201;=0A+=0A+=09/*=0A+=09=20*=20Setup=20a=20scan=20key=20= to=20find=20chunks=20in=20toast=20table=20with=20matching=20va_valueid=0A= +=09=20*/=0A+=09ScanKeyInit(&ctx->toastkey,=0A+=09=09=09=09(AttrNumber)=20= 1,=0A+=09=09=09=09BTEqualStrategyNumber,=20F_OIDEQ,=0A+=09=09=09=09= ObjectIdGetDatum(toast_pointer->va_valueid));=0A+=0A+=09/*=0A+=09=20*=20= Check=20if=20any=20chunks=20for=20this=20toasted=20object=20exist=20in=20= the=20toast=20table,=0A+=09=20*=20accessible=20via=20the=20index.=0A+=09=20= */=0A+=09init_toast_snapshot(&ctx->SnapshotToast);=0A+=09ctx->toastscan=20= =3D=20systable_beginscan_ordered(ctx->toastrel,=0A+=09=09=09=09=09=09=09=09= =09=09=09=09ctx->valid_toast_index,=0A+=09=09=09=09=09=09=09=09=09=09=09=09= &ctx->SnapshotToast,=201,=0A+=09=09=09=09=09=09=09=09=09=09=09=09= &ctx->toastkey);=0A+=09ctx->chunkno=20=3D=200;=0A+}=0A+=0A+/*=0A+=20*=20= toastTupleIteration_next=0A+=20*=0A+=20*=20=20=20Advances=20the=20state=20= tracked=20in=20ctx=20to=20the=20next=20toast=20tuple=20for=20the=0A+=20*=20= =20=20attribute.=0A+=20*=0A+=20*=20=20=20Caller=20should=20have=20= already=20set=20up=20the=20iteration=20via=0A+=20*=20=20=20= beginToastTupleIteration,=20and=20should=20stop=20calling=20when=20this=20= function=0A+=20*=20=20=20returns=20false.=0A+=20*/=0A+bool=0A= +toastTupleIteration_next(HeapCheckContext=20*=20ctx)=0A+{=0A+=09= ctx->toasttup=20=3D=20systable_getnext_ordered(ctx->toastscan,=0A+=09=09=09= =09=09=09=09=09=09=09=09=20ForwardScanDirection);=0A+=09return=20= ctx->toasttup=20!=3D=20NULL;=0A+}=0A+=0A+/*=0A+=20*=20= endToastTupleIteration=0A+=20*=0A+=20*=20=20=20Releases=20resources=20= taken=20by=20beginToastTupleIteration=20or=0A+=20*=20=20=20= toastTupleIteration_next.=0A+=20*/=0A+void=0A= +endToastTupleIteration(HeapCheckContext=20*=20ctx)=0A+{=0A+=09= systable_endscan_ordered(ctx->toastscan);=0A+}=0A+=0A+/*=0A+=20*=20Given=20= a=20TransactionId,=20attempt=20to=20interpret=20it=20as=20a=20valid=0A+=20= *=20FullTransactionId,=20neither=20in=20the=20future=20nor=20overlong=20= in=0A+=20*=20the=20past.=20=20Stores=20the=20inferred=20= FullTransactionId=20in=20*fxid.=0A+=20*=0A+=20*=20Returns=20whether=20= the=20xid=20is=20newer=20than=20the=20oldest=20clog=20xid.=0A+=20*/=0A= +bool=0A+TransactionIdStillValid(TransactionId=20xid,=20= FullTransactionId=20*fxid)=0A+{=0A+=09FullTransactionId=20fnow;=0A+=09= uint32=09=09epoch;=0A+=0A+=09/*=20Initialize=20fxid;=20we'll=20overwrite=20= this=20later=20if=20needed=20*/=0A+=09*fxid=20=3D=20= FullTransactionIdFromEpochAndXid(0,=20xid);=0A+=0A+=09/*=20Special=20= xids=20can=20quickly=20be=20turned=20into=20invalid=20fxids=20*/=0A+=09= if=20(!TransactionIdIsValid(xid))=0A+=09=09return=20false;=0A+=09if=20= (!TransactionIdIsNormal(xid))=0A+=09=09return=20true;=0A+=0A+=09/*=0A+=09= =20*=20Charitably=20infer=20the=20full=20transaction=20id=20as=20being=20= within=20one=20epoch=20ago=0A+=09=20*/=0A+=09fnow=20=3D=20= ReadNextFullTransactionId();=0A+=09epoch=20=3D=20= EpochFromFullTransactionId(fnow);=0A+=09*fxid=20=3D=20= FullTransactionIdFromEpochAndXid(epoch,=20xid);=0A+=09if=20= (!FullTransactionIdPrecedes(*fxid,=20fnow))=0A+=09=09*fxid=20=3D=20= FullTransactionIdFromEpochAndXid(epoch=20-=201,=20xid);=0A+=09if=20= (!FullTransactionIdPrecedes(*fxid,=20fnow))=0A+=09=09return=20false;=0A+=0A= +=09/*=20The=20oldestClogXid=20is=20protected=20by=20CLogTruncationLock=20= */=0A+=09Assert(LWLockHeldByMe(CLogTruncationLock));=0A+=09if=20= (TransactionIdPrecedes(xid,=20ShmemVariableCache->oldestClogXid))=0A+=09=09= return=20false;=0A+=09return=20true;=0A+}=0A+=0A+/*=0A+=20*=20= HeapTupleIsVisible=0A+=20*=0A+=20*=09Determine=20whether=20tuples=20are=20= visible=20for=20heapcheck.=20=20Similar=20to=0A+=20*=20=20= HeapTupleSatisfiesVacuum,=20but=20with=20critical=20differences.=0A+=20*=0A= +=20*=20=201)=20Does=20not=20touch=20hint=20bits.=20=20It=20seems=20= imprudent=20to=20write=20hint=20bits=0A+=20*=20=20=20=20=20to=20a=20= table=20during=20a=20corruption=20check.=0A+=20*=20=202)=20Gracefully=20= handles=20xids=20that=20are=20too=20old=20by=20calling=0A+=20*=20=20=20=20= =20TransactionIdStillValid=20before=20TransactionLogFetch,=20thus=20= avoiding=0A+=20*=20=20=20=20=20a=20backend=20abort.=0A+=20*=20=203)=20= Only=20makes=20a=20boolean=20determination=20of=20whether=20heapcheck=20= should=0A+=20*=20=20=20=20=20see=20the=20tuple,=20rather=20than=20doing=20= extra=20work=20for=20vacuum-related=0A+=20*=20=20=20=20=20= categorization.=0A+=20*/=0A+bool=0A+HeapTupleIsVisible(HeapTupleHeader=20= tuphdr,=20HeapCheckContext=20*=20ctx)=0A+{=0A+=09FullTransactionId=20= fxmin,=0A+=09=09=09=09fxmax;=0A+=09uint16=09=09infomask=20=3D=20= tuphdr->t_infomask;=0A+=09TransactionId=20xmin=20=3D=20= HeapTupleHeaderGetXmin(tuphdr);=0A+=0A+=09if=20= (!HeapTupleHeaderXminCommitted(tuphdr))=0A+=09{=0A+=09=09if=20= (HeapTupleHeaderXminInvalid(tuphdr))=0A+=09=09{=0A+=09=09=09return=20= false;=09=09/*=20HEAPTUPLE_DEAD=20*/=0A+=09=09}=0A+=09=09/*=20Used=20by=20= pre-9.0=20binary=20upgrades=20*/=0A+=09=09else=20if=20(infomask=20&=20= HEAP_MOVED_OFF)=0A+=09=09{=0A+=09=09=09TransactionId=20xvac=20=3D=20= HeapTupleHeaderGetXvac(tuphdr);=0A+=0A+=09=09=09if=20= (TransactionIdIsCurrentTransactionId(xvac))=0A+=09=09=09=09return=20= false;=09/*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A+=09=09=09if=20= (TransactionIdIsInProgress(xvac))=0A+=09=09=09=09return=20false;=09/*=20= HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A+=09=09=09if=20= (TransactionIdDidCommit(xvac))=0A+=09=09=09=09return=20false;=09/*=20= HEAPTUPLE_DEAD=20*/=0A+=09=09}=0A+=09=09/*=20Used=20by=20pre-9.0=20= binary=20upgrades=20*/=0A+=09=09else=20if=20(infomask=20&=20= HEAP_MOVED_IN)=0A+=09=09{=0A+=09=09=09TransactionId=20xvac=20=3D=20= HeapTupleHeaderGetXvac(tuphdr);=0A+=0A+=09=09=09if=20= (TransactionIdIsCurrentTransactionId(xvac))=0A+=09=09=09=09return=20= false;=09/*=20HEAPTUPLE_INSERT_IN_PROGRESS=20*/=0A+=09=09=09if=20= (TransactionIdIsInProgress(xvac))=0A+=09=09=09=09return=20false;=09/*=20= HEAPTUPLE_INSERT_IN_PROGRESS=20*/=0A+=09=09=09if=20= (!TransactionIdDidCommit(xvac))=0A+=09=09=09=09return=20false;=09/*=20= HEAPTUPLE_DEAD=20*/=0A+=09=09}=0A+=09=09else=20if=20= (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetRawXmin(tuphdr)))=0A= +=09=09=09return=20false;=09=09/*=20insert=20or=20delete=20in=20progress=20= */=0A+=09=09else=20if=20= (TransactionIdIsInProgress(HeapTupleHeaderGetRawXmin(tuphdr)))=0A+=09=09=09= return=20false;=09=09/*=20HEAPTUPLE_INSERT_IN_PROGRESS=20*/=0A+=0A+=09=09= /*=0A+=09=09=20*=20The=20tuple=20appears=20to=20either=20be=20or=20to=20= have=20been=20visible=20to=20us,=20but=0A+=09=09=20*=20the=20xmin=20may=20= be=20too=20far=20in=20the=20past=20to=20be=20used.=20=20We=20have=20to=20= check=0A+=09=09=20*=20that=20before=20calling=20TransactionIdDidCommit=20= to=20avoid=20an=20Assertion.=0A+=09=09=20*/=0A+=09=09= LWLockAcquire(CLogTruncationLock,=20LW_SHARED);=0A+=09=09if=20= (!TransactionIdStillValid(xmin,=20&fxmin))=0A+=09=09{=0A+=09=09=09= LWLockRelease(CLogTruncationLock);=0A+=09=09=09record_corruption(ctx,=20= psprintf("tuple=20xmin=20=3D=20%u=20(interpreted=20as=20"=0A+=09=09=09=09= =09=09=09=09=09=09=09UINT64_FORMAT=0A+=09=09=09=09=09=09=09=09=09=09=09= ")=20not=20or=20no=20longer=20valid",=0A+=09=09=09=09=09=09=09=09=09=09=09= xmin,=20fxmin.value));=0A+=09=09=09return=20false;=0A+=09=09}=0A+=09=09= else=20if=20(!TransactionIdDidCommit(HeapTupleHeaderGetRawXmin(tuphdr)))=0A= +=09=09{=0A+=09=09=09LWLockRelease(CLogTruncationLock);=0A+=09=09=09= return=20false;=09=09/*=20HEAPTUPLE_DEAD=20*/=0A+=09=09}=0A+=09=09= LWLockRelease(CLogTruncationLock);=0A+=09}=0A+=0A+=09if=20(!(infomask=20= &=20HEAP_XMAX_INVALID)=20&&=20!HEAP_XMAX_IS_LOCKED_ONLY(infomask))=0A+=09= {=0A+=09=09if=20(infomask=20&=20HEAP_XMAX_IS_MULTI)=0A+=09=09{=0A+=09=09=09= TransactionId=20xmax=20=3D=20HeapTupleGetUpdateXid(tuphdr);=0A+=0A+=09=09= =09/*=20not=20LOCKED_ONLY,=20so=20it=20has=20to=20have=20an=20xmax=20*/=0A= +=09=09=09if=20(!TransactionIdIsValid(xmax))=0A+=09=09=09{=0A+=09=09=09=09= record_corruption(ctx,=20_("heap=20tuple=20with=20XMAX_IS_MULTI=20is=20"=0A= +=09=09=09=09=09=09=09=09=09=09=20"neither=20LOCKED_ONLY=20nor=20has=20a=20= "=0A+=09=09=09=09=09=09=09=09=09=09=20"valid=20xmax"));=0A+=09=09=09=09= return=20false;=0A+=09=09=09}=0A+=09=09=09if=20= (TransactionIdIsInProgress(xmax))=0A+=09=09=09=09return=20false;=09/*=20= HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A+=0A+=09=09=09= LWLockAcquire(CLogTruncationLock,=20LW_SHARED);=0A+=09=09=09if=20= (!TransactionIdStillValid(xmax,=20&fxmax))=0A+=09=09=09{=0A+=09=09=09=09= LWLockRelease(CLogTruncationLock);=0A+=09=09=09=09record_corruption(ctx,=20= psprintf("tuple=20xmax=20=3D=20%u=20(interpreted=20"=0A+=09=09=09=09=09=09= =09=09=09=09=09=09"as=20"=20UINT64_FORMAT=0A+=09=09=09=09=09=09=09=09=09=09= =09=09")=20not=20or=20no=20longer=20valid",=0A+=09=09=09=09=09=09=09=09=09= =09=09=09xmax,=20fxmax.value));=0A+=09=09=09=09return=20false;=0A+=09=09=09= }=0A+=09=09=09else=20if=20(TransactionIdDidCommit(xmax))=0A+=09=09=09{=0A= +=09=09=09=09LWLockRelease(CLogTruncationLock);=0A+=09=09=09=09return=20= false;=09/*=20HEAPTUPLE_RECENTLY_DEAD=20or=20HEAPTUPLE_DEAD=20*/=0A+=09=09= =09}=0A+=09=09=09LWLockRelease(CLogTruncationLock);=0A+=09=09=09/*=20Ok,=20= the=20tuple=20is=20live=20*/=0A+=09=09}=0A+=09=09else=20if=20(!(infomask=20= &=20HEAP_XMAX_COMMITTED))=0A+=09=09{=0A+=09=09=09if=20= (TransactionIdIsInProgress(HeapTupleHeaderGetRawXmax(tuphdr)))=0A+=09=09=09= =09return=20false;=09/*=20HEAPTUPLE_DELETE_IN_PROGRESS=20*/=0A+=09=09=09= /*=20Ok,=20the=20tuple=20is=20live=20*/=0A+=09=09}=0A+=09=09else=0A+=09=09= =09return=20false;=09=09/*=20HEAPTUPLE_RECENTLY_DEAD=20or=20= HEAPTUPLE_DEAD=20*/=0A+=09}=0A+=09return=20true;=0A+}=0A+=0A+/*=0A+=20*=20= check_toast_tuple=0A+=20*=0A+=20*=20=20=20Checks=20the=20current=20toast=20= tuple=20as=20tracked=20in=20ctx=20for=20corruption.=20=20Records=0A+=20*=20= =20=20any=20corruption=20found=20in=20ctx->corruption.=0A+=20*=0A+=20*=20= =20=20The=20caller=20should=20have=20iterated=20to=20a=20tuple=20via=20= toastTupleIteration_next.=0A+=20*/=0A+void=0A= +check_toast_tuple(HeapCheckContext=20*=20ctx)=0A+{=0A+=09int32=09=09= curchunk;=0A+=09Pointer=09=09chunk;=0A+=09bool=09=09isnull;=0A+=09char=09= =20=20=20*chunkdata;=0A+=09int32=09=09chunksize;=0A+=09int32=09=09= expected_size;=0A+=0A+=09ctx->found_toasttup=20=3D=20true;=0A+=0A+=09/*=0A= +=09=20*=20Have=20a=20chunk,=20extract=20the=20sequence=20number=20and=20= the=20data=0A+=09=20*/=0A+=09curchunk=20=3D=20= DatumGetInt32(fastgetattr(ctx->toasttup,=202,=0A+=09=09=09=09=09=09=09=09= =09=09=20ctx->toasttupDesc,=20&isnull));=0A+=09if=20(isnull)=0A+=09{=0A+=09= =09record_corruption(ctx,=20_("toast=20chunk=20sequencenumber=20is=20= null"));=0A+=09=09return;=0A+=09}=0A+=09chunk=20=3D=20= DatumGetPointer(fastgetattr(ctx->toasttup,=203,=0A+=09=09=09=09=09=09=09=09= =09=09ctx->toasttupDesc,=20&isnull));=0A+=09if=20(isnull)=0A+=09{=0A+=09=09= record_corruption(ctx,=20_("toast=20chunk=20data=20is=20null"));=0A+=09=09= return;=0A+=09}=0A+=09if=20(!VARATT_IS_EXTENDED(chunk))=0A+=09{=0A+=09=09= chunksize=20=3D=20VARSIZE(chunk)=20-=20VARHDRSZ;=0A+=09=09chunkdata=20=3D=20= VARDATA(chunk);=0A+=09}=0A+=09else=20if=20(VARATT_IS_SHORT(chunk))=0A+=09= {=0A+=09=09/*=0A+=09=09=20*=20could=20happen=20due=20to=20= heap_form_tuple=20doing=20its=20thing=0A+=09=09=20*/=0A+=09=09chunksize=20= =3D=20VARSIZE_SHORT(chunk)=20-=20VARHDRSZ_SHORT;=0A+=09=09chunkdata=20=3D=20= VARDATA_SHORT(chunk);=0A+=09}=0A+=09else=0A+=09{=0A+=09=09/*=20should=20= never=20happen=20*/=0A+=09=09record_corruption(ctx,=20_("toast=20chunk=20= is=20neither=20short=20nor=20extended"));=0A+=09=09return;=0A+=09}=0A+=0A= +=09/*=0A+=09=20*=20Some=20checks=20on=20the=20data=20we've=20found=0A+=09= =20*/=0A+=09if=20(curchunk=20!=3D=20ctx->chunkno)=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("toast=20chunk=20sequence=20number=20= %u=20"=0A+=09=09=09=09=09=09=09=09=09=09"not=20the=20expected=20sequence=20= number=20%u",=0A+=09=09=09=09=09=09=09=09=09=09curchunk,=20= ctx->chunkno));=0A+=09=09return;=0A+=09}=0A+=09if=20(curchunk=20>=20= ctx->endchunk)=0A+=09{=0A+=09=09record_corruption(ctx,=20psprintf("toast=20= chunk=20sequence=20number=20%u=20"=0A+=09=09=09=09=09=09=09=09=09=09= "exceeds=20the=20end=20chunk=20sequence=20"=0A+=09=09=09=09=09=09=09=09=09= =09"number=20%u",=0A+=09=09=09=09=09=09=09=09=09=09curchunk,=20= ctx->endchunk));=0A+=09=09return;=0A+=09}=0A+=0A+=09expected_size=20=3D=20= curchunk=20<=20ctx->totalchunks=20-=201=20?=20TOAST_MAX_CHUNK_SIZE=0A+=09= =09:=20ctx->attrsize=20-=20((ctx->totalchunks=20-=201)=20*=20= TOAST_MAX_CHUNK_SIZE);=0A+=09if=20(chunksize=20!=3D=20expected_size)=0A+=09= {=0A+=09=09record_corruption(ctx,=20psprintf("chunk=20size=20%u=20= differs=20from=20"=0A+=09=09=09=09=09=09=09=09=09=09"expected=20size=20= %u",=0A+=09=09=09=09=09=09=09=09=09=09chunksize,=20expected_size));=0A+=09= =09return;=0A+=09}=0A+=0A+=09ctx->chunkno++;=0A+}=0A+=0A+/*=0A+=20*=20= check_tuple_attribute=0A+=20*=0A+=20*=20=20=20Checks=20the=20current=20= attribute=20as=20tracked=20in=20ctx=20for=20corruption.=20=20Records=0A+=20= *=20=20=20any=20corruption=20found=20in=20ctx->corruption.=0A+=20*=0A+=20= *=20=20=20The=20caller=20should=20have=20iterated=20to=20a=20tuple=20via=0A= +=20*=20=20=20tupleAttributeIteration_next.=0A+=20*/=0A+bool=0A= +check_tuple_attribute(HeapCheckContext=20*=20ctx)=0A+{=0A+=09Datum=09=09= attdatum;=0A+=09struct=20varlena=20*attr;=0A+=0A+=09if=20= (ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20ctx->lp_len)=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("t_hoff=20+=20offset=20>=20lp_len=20= (%u=20+=20%u=20>=20%u)",=0A+=09=09=09=09=09=09=09=09=09=09= ctx->tuphdr->t_hoff,=20ctx->offset,=0A+=09=09=09=09=09=09=09=09=09=09= ctx->lp_len));=0A+=09=09return=20false;=0A+=09}=0A+=0A+=09/*=20Skip=20= null=20values=20*/=0A+=09if=20(ctx->hasnulls=20&&=20= att_isnull(ctx->attnum,=20ctx->bp))=0A+=09=09return=20true;=0A+=0A+=09/*=20= Skip=20non-varlena=20values,=20but=20update=20offset=20first=20*/=0A+=09= if=20(ctx->thisatt->attlen=20!=3D=20-1)=0A+=09{=0A+=09=09ctx->offset=20=3D= =20att_align_nominal(ctx->offset,=20ctx->thisatt->attalign);=0A+=09=09= ctx->offset=20=3D=20att_addlength_pointer(ctx->offset,=20= ctx->thisatt->attlen,=0A+=09=09=09=09=09=09=09=09=09=09=09ctx->tp=20+=20= ctx->offset);=0A+=09=09return=20true;=0A+=09}=0A+=0A+=09/*=20Ok,=20we're=20= looking=20at=20a=20varlena=20attribute.=20*/=0A+=09ctx->offset=20=3D=20= att_align_pointer(ctx->offset,=20ctx->thisatt->attalign,=20-1,=0A+=09=09=09= =09=09=09=09=09=09ctx->tp=20+=20ctx->offset);=0A+=0A+=09/*=20Get=20the=20= (possibly=20corrupt)=20varlena=20datum=20*/=0A+=09attdatum=20=3D=20= fetchatt(ctx->thisatt,=20ctx->tp=20+=20ctx->offset);=0A+=0A+=09/*=0A+=09=20= *=20We=20have=20the=20datum,=20but=20we=20cannot=20decode=20it=20= carelessly,=20as=20it=20may=20still=0A+=09=20*=20be=20corrupt.=0A+=09=20= */=0A+=0A+=09/*=0A+=09=20*=20Check=20that=20VARTAG_SIZE=20won't=20hit=20= a=20TrapMacro=20on=20a=20corrupt=20va_tag=20before=0A+=09=20*=20risking=20= a=20call=20into=20att_addlength_pointer=0A+=09=20*/=0A+=09if=20= (VARATT_IS_1B_E(ctx->tp=20+=20ctx->offset))=0A+=09{=0A+=09=09uint8=09=09= va_tag=20=3D=20va_tag=20=3D=20VARTAG_EXTERNAL(ctx->tp=20+=20= ctx->offset);=0A+=0A+=09=09if=20(va_tag=20!=3D=20VARTAG_ONDISK)=0A+=09=09= {=0A+=09=09=09record_corruption(ctx,=20psprintf("unexpected=20TOAST=20= vartag=20%u=20for=20"=0A+=09=09=09=09=09=09=09=09=09=09=09"attribute=20= #%u=20at=20t_hoff=20=3D=20%u,=20"=0A+=09=09=09=09=09=09=09=09=09=09=09= "offset=20=3D=20%u",=0A+=09=09=09=09=09=09=09=09=09=09=09va_tag,=20= ctx->attnum,=0A+=09=09=09=09=09=09=09=09=09=09=09ctx->tuphdr->t_hoff,=20= ctx->offset));=0A+=09=09=09return=20false;=09=09/*=20We=20can't=20know=20= where=20the=20next=20attribute=0A+=09=09=09=09=09=09=09=09=20*=20begins=20= */=0A+=09=09}=0A+=09}=0A+=0A+=09/*=20Ok,=20should=20be=20safe=20now=20*/=0A= +=09ctx->offset=20=3D=20att_addlength_pointer(ctx->offset,=20= ctx->thisatt->attlen,=0A+=09=09=09=09=09=09=09=09=09=09ctx->tp=20+=20= ctx->offset);=0A+=0A+=09/*=0A+=09=20*=20heap_deform_tuple=20would=20be=20= done=20with=20this=20attribute=20at=20this=20point,=0A+=09=20*=20having=20= stored=20it=20in=20values[],=20and=20would=20continue=20to=20the=20next=20= attribute.=0A+=09=20*=20We=20go=20further,=20because=20we=20need=20to=20= check=20if=20the=20toast=20datum=20is=20corrupt.=0A+=09=20*/=0A+=0A+=09= attr=20=3D=20(struct=20varlena=20*)=20DatumGetPointer(attdatum);=0A+=0A+=09= /*=0A+=09=20*=20Now=20we=20follow=20the=20logic=20of=20= detoast_external_attr(),=20with=20the=20same=0A+=09=20*=20caveats=20= about=20being=20paranoid=20about=20corruption.=0A+=09=20*/=0A+=0A+=09/*=20= Skip=20values=20that=20are=20not=20external=20*/=0A+=09if=20= (!VARATT_IS_EXTERNAL(attr))=0A+=09=09return=20true;=0A+=0A+=09/*=20It=20= is=20external,=20and=20we're=20looking=20at=20a=20page=20on=20disk=20*/=0A= +=09if=20(!VARATT_IS_EXTERNAL_ONDISK(attr))=0A+=09{=0A+=09=09= record_corruption(ctx,=0A+=09=09=09=09=09=09=20=20_("attribute=20is=20= external=20but=20not=20marked=20as=20on=20disk"));=0A+=09=09return=20= true;=0A+=09}=0A+=0A+=09/*=20The=20tuple=20header=20better=20claim=20to=20= contain=20toasted=20values=20*/=0A+=09if=20(!(ctx->infomask=20&=20= HEAP_HASEXTERNAL))=0A+=09{=0A+=09=09record_corruption(ctx,=20= _("attribute=20is=20external=20but=20tuple=20header=20"=0A+=09=09=09=09=09= =09=09=09=20"flag=20HEAP_HASEXTERNAL=20not=20set"));=0A+=09=09return=20= true;=0A+=09}=0A+=0A+=09/*=20The=20relation=20better=20have=20a=20toast=20= table=20*/=0A+=09if=20(!ctx->has_toastrel)=0A+=09{=0A+=09=09= record_corruption(ctx,=20_("attribute=20is=20external=20but=20relation=20= has=20"=0A+=09=09=09=09=09=09=09=09=20"no=20toast=20relation"));=0A+=09=09= return=20true;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20Must=20dereference=20= indirect=20toast=20pointers=20before=20we=20can=20check=20them=0A+=09=20= */=0A+=09if=20(VARATT_IS_EXTERNAL_INDIRECT(attr))=0A+=09{=0A+=09=09= struct=20varatt_indirect=20redirect;=0A+=0A+=09=09= VARATT_EXTERNAL_GET_POINTER(redirect,=20attr);=0A+=09=09attr=20=3D=20= (struct=20varlena=20*)=20redirect.pointer;=0A+=0A+=09=09/*=20nested=20= indirect=20Datums=20aren't=20allowed=20*/=0A+=09=09if=20= (VARATT_IS_EXTERNAL_INDIRECT(attr))=0A+=09=09{=0A+=09=09=09= record_corruption(ctx,=20_("attribute=20has=20nested=20external=20"=0A+=09= =09=09=09=09=09=09=09=09=20"indirect=20toast=20pointer"));=0A+=09=09=09= return=20true;=0A+=09=09}=0A+=09}=0A+=0A+=09if=20= (VARATT_IS_EXTERNAL_ONDISK(attr))=0A+=09{=0A+=09=09struct=20= varatt_external=20toast_pointer;=0A+=0A+=09=09/*=0A+=09=09=20*=20Must=20= copy=20attr=20into=20toast_pointer=20for=20alignment=20considerations=0A= +=09=09=20*/=0A+=09=09VARATT_EXTERNAL_GET_POINTER(toast_pointer,=20= attr);=0A+=09=09beginToastTupleIteration(ctx,=20&toast_pointer);=0A+=0A+=09= =09while=20(toastTupleIteration_next(ctx))=0A+=09=09=09= check_toast_tuple(ctx);=0A+=0A+=09=09if=20(ctx->chunkno=20!=3D=20= (ctx->endchunk=20+=201))=0A+=09=09=09record_corruption(ctx,=20= psprintf("final=20chunk=20number=20differs=20from=20"=0A+=09=09=09=09=09=09= =09=09=09=09=09"expected=20(%u=20vs.=20%u)",=0A+=09=09=09=09=09=09=09=09=09= =09=09ctx->chunkno,=20(ctx->endchunk=20+=201)));=0A+=09=09if=20= (!ctx->found_toasttup)=0A+=09=09=09record_corruption(ctx,=20_("toasted=20= value=20missing=20from=20"=0A+=09=09=09=09=09=09=09=09=09=20"toast=20= table"));=0A+=09=09endToastTupleIteration(ctx);=0A+=09}=0A+=09return=20= true;=0A+}=0A+=0A+/*=0A+=20*=20check_tuple=0A+=20*=0A+=20*=20=20=20= Checks=20the=20current=20tuple=20as=20tracked=20in=20ctx=20for=20= corruption.=20=20Records=20any=0A+=20*=20=20=20corruption=20found=20in=20= ctx->corruption.=0A+=20*=0A+=20*=20=20=20The=20caller=20should=20have=20= iterated=20to=20a=20tuple=20via=20pageTupleIteration_next.=0A+=20*/=0A= +void=0A+check_tuple(HeapCheckContext=20*=20ctx)=0A+{=0A+=09bool=09=09= fatal=20=3D=20false;=0A+=0A+=09/*=20Check=20relminmxid=20against=20mxid,=20= if=20any=20*/=0A+=09if=20(ctx->infomask=20&=20HEAP_XMAX_IS_MULTI=20&&=0A= +=09=09MultiXactIdPrecedes(ctx->xmax,=20ctx->relminmxid))=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("tuple=20xmax=20=3D=20%u=20precedes=20= relation=20"=0A+=09=09=09=09=09=09=09=09=09=09"relminmxid=20=3D=20%u",=0A= +=09=09=09=09=09=09=09=09=09=09ctx->xmax,=20ctx->relminmxid));=0A+=09}=0A= +=0A+=09/*=20Check=20xmin=20against=20relfrozenxid=20*/=0A+=09if=20= (TransactionIdIsNormal(ctx->relfrozenxid)=20&&=0A+=09=09= TransactionIdIsNormal(ctx->xmin)=20&&=0A+=09=09= TransactionIdPrecedes(ctx->xmin,=20ctx->relfrozenxid))=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("tuple=20xmin=20=3D=20%u=20precedes=20= relation=20"=0A+=09=09=09=09=09=09=09=09=09=09"relfrozenxid=20=3D=20%u",=0A= +=09=09=09=09=09=09=09=09=09=09ctx->xmin,=20ctx->relfrozenxid));=0A+=09}=0A= +=0A+=09/*=20Check=20xmax=20against=20relfrozenxid=20*/=0A+=09if=20= (TransactionIdIsNormal(ctx->relfrozenxid)=20&&=0A+=09=09= TransactionIdIsNormal(ctx->xmax)=20&&=0A+=09=09= TransactionIdPrecedes(ctx->xmax,=20ctx->relfrozenxid))=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("tuple=20xmax=20=3D=20%u=20precedes=20= relation=20"=0A+=09=09=09=09=09=09=09=09=09=09"relfrozenxid=20=3D=20%u",=0A= +=09=09=09=09=09=09=09=09=09=09ctx->xmax,=20ctx->relfrozenxid));=0A+=09}=0A= +=0A+=09/*=20Check=20for=20tuple=20header=20corruption=20*/=0A+=09if=20= (ctx->tuphdr->t_hoff=20<=20SizeofHeapTupleHeader)=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("t_hoff=20<=20SizeofHeapTupleHeader=20= (%u=20<=20%u)",=0A+=09=09=09=09=09=09=09=09=09=09ctx->tuphdr->t_hoff,=0A= +=09=09=09=09=09=09=09=09=09=09(unsigned)=20SizeofHeapTupleHeader));=0A+=09= =09fatal=20=3D=20true;=0A+=09}=0A+=09if=20(ctx->tuphdr->t_hoff=20>=20= ctx->lp_len)=0A+=09{=0A+=09=09record_corruption(ctx,=20psprintf("t_hoff=20= >=20lp_len=20(%u=20>=20%u)",=0A+=09=09=09=09=09=09=09=09=09=09= ctx->tuphdr->t_hoff,=20ctx->lp_len));=0A+=09=09fatal=20=3D=20true;=0A+=09= }=0A+=09if=20(ctx->tuphdr->t_hoff=20!=3D=20= MAXALIGN(ctx->tuphdr->t_hoff))=0A+=09{=0A+=09=09record_corruption(ctx,=20= psprintf("t_hoff=20not=20max-aligned=20(%u)",=0A+=09=09=09=09=09=09=09=09= =09=09ctx->tuphdr->t_hoff));=0A+=09=09fatal=20=3D=20true;=0A+=09}=0A+=0A= +=09/*=0A+=09=20*=20If=20the=20tuple=20has=20nulls,=20check=20that=20the=20= implied=20length=20of=20the=20variable=0A+=09=20*=20length=20nulls=20= bitmap=20field=20t_bits=20does=20not=20overflow=20the=20allowed=20space.=0A= +=09=20*=20We=20don't=20know=20if=20the=20corruption=20is=20in=20the=20= natts=20field=20or=20the=20infomask=0A+=09=20*=20bit=20HEAP_HASNULL.=0A+=09= =20*/=0A+=09if=20(ctx->hasnulls=20&&=0A+=09=09SizeofHeapTupleHeader=20+=20= BITMAPLEN(ctx->natts)=20>=20ctx->tuphdr->t_hoff)=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("SizeofHeapTupleHeader=20+=20"=0A+=09=09= =09=09=09=09=09=09=09=09"BITMAPLEN(natts)=20>=20t_hoff=20"=0A+=09=09=09=09= =09=09=09=09=09=09"(%u=20+=20%u=20>=20%u)",=0A+=09=09=09=09=09=09=09=09=09= =09(unsigned)=20SizeofHeapTupleHeader,=0A+=09=09=09=09=09=09=09=09=09=09= BITMAPLEN(ctx->natts),=0A+=09=09=09=09=09=09=09=09=09=09= ctx->tuphdr->t_hoff));=0A+=09=09fatal=20=3D=20true;=0A+=09}=0A+=0A+=09/*=20= Cannot=20process=20tuple=20data=20if=20tuple=20header=20was=20corrupt=20= */=0A+=09if=20(fatal)=0A+=09=09return;=0A+=0A+=09/*=0A+=09=20*=20Skip=20= tuples=20that=20are=20invisible,=20as=20we=20cannot=20assume=20the=20= TupleDesc=20we=0A+=09=20*=20are=20using=20is=20appropriate.=0A+=09=20*/=0A= +=09if=20(!HeapTupleIsVisible(ctx->tuphdr,=20ctx))=0A+=09=09return;=0A+=0A= +=09/*=0A+=09=20*=20If=20we=20get=20this=20far,=20the=20tuple=20is=20= visible=20to=20us,=20so=20it=20must=20not=20be=0A+=09=20*=20incompatible=20= with=20our=20relDesc.=20=20The=20natts=20field=20could=20be=20= legitimately=0A+=09=20*=20shorter=20than=20rel_natts,=20but=20it=20= cannot=20be=20longer=20than=20rel_natts.=0A+=09=20*/=0A+=09if=20= (ctx->rel_natts=20<=20ctx->natts)=0A+=09{=0A+=09=09= record_corruption(ctx,=20psprintf("relation=20natts=20<=20tuple=20natts=20= (%u=20<=20%u)",=0A+=09=09=09=09=09=09=09=09=09=09ctx->rel_natts,=20= ctx->natts));=0A+=09=09return;=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20Iterate=20= over=20the=20attributes=20looking=20for=20broken=20toast=20values.=20= This=0A+=09=20*=20roughly=20follows=20the=20logic=20of=20= heap_deform_tuple,=20except=20that=20it=20doesn't=0A+=09=20*=20bother=20= building=20up=20isnull[]=20and=20values[]=20arrays,=20since=20nobody=20= wants=0A+=09=20*=20them,=20and=20it=20unrolls=20anything=20that=20might=20= trip=20over=20an=20Assert=20when=0A+=09=20*=20processing=20corrupt=20= data.=0A+=09=20*/=0A+=09beginTupleAttributeIteration(ctx);=0A+=09while=20= (tupleAttributeIteration_next(ctx)=20&&=0A+=09=09=20=20=20= check_tuple_attribute(ctx))=0A+=09=09;=0A+=09= endTupleAttributeIteration(ctx);=0A+}=0A+=0A+/*=0A+=20*=20check_relation=0A= +=20*=0A+=20*=20=20=20Checks=20the=20relation=20given=20by=20relid=20for=20= corruption,=20returning=20a=20list=20of=20all=0A+=20*=20=20=20it=20= finds.=0A+=20*=0A+=20*=20=20=20The=20caller=20should=20set=20up=20the=20= memory=20context=20as=20desired=20before=20calling.=0A+=20*=20=20=20The=20= returned=20list=20belongs=20to=20the=20caller.=0A+=20*/=0A+List=20*=0A= +check_relation(Oid=20relid)=0A+{=0A+=09HeapCheckContext=20ctx;=0A+=0A+=09= memset(&ctx,=200,=20sizeof(HeapCheckContext));=0A+=0A+=09/*=20Open=20the=20= relation=20*/=0A+=09ctx.relid=20=3D=20relid;=0A+=09ctx.corruption=20=3D=20= NIL;=0A+=09ctx.rel=20=3D=20relation_open(relid,=20AccessShareLock);=0A+=09= check_relation_relkind(ctx.rel);=0A+=0A+=09ctx.relDesc=20=3D=20= RelationGetDescr(ctx.rel);=0A+=09ctx.rel_natts=20=3D=20= RelationGetDescr(ctx.rel)->natts;=0A+=09ctx.relfrozenxid=20=3D=20= ctx.rel->rd_rel->relfrozenxid;=0A+=09ctx.relminmxid=20=3D=20= ctx.rel->rd_rel->relminmxid;=0A+=0A+=09/*=20Open=20the=20toast=20= relation,=20if=20any=20*/=0A+=09if=20(ctx.rel->rd_rel->reltoastrelid)=0A= +=09{=0A+=09=09int=09=09=09offset;=0A+=0A+=09=09/*=20Main=20relation=20= has=20associated=20toast=20relation=20*/=0A+=09=09ctx.has_toastrel=20=3D=20= true;=0A+=09=09ctx.toastrel=20=3D=20= table_open(ctx.rel->rd_rel->reltoastrelid,=0A+=09=09=09=09=09=09=09=09=20= =20AccessShareLock);=0A+=09=09offset=20=3D=20= toast_open_indexes(ctx.toastrel,=0A+=09=09=09=09=09=09=09=09=09= AccessShareLock,=0A+=09=09=09=09=09=09=09=09=09&(ctx.toast_indexes),=0A+=09= =09=09=09=09=09=09=09=09&(ctx.num_toast_indexes));=0A+=09=09= ctx.valid_toast_index=20=3D=20ctx.toast_indexes[offset];=0A+=09}=0A+=09= else=0A+=09{=0A+=09=09/*=20Main=20relation=20has=20no=20associated=20= toast=20relation=20*/=0A+=09=09ctx.has_toastrel=20=3D=20false;=0A+=09=09= ctx.toast_indexes=20=3D=20NULL;=0A+=09=09ctx.num_toast_indexes=20=3D=20= 0;=0A+=09}=0A+=0A+=09/*=20check=20all=20blocks=20of=20the=20relation=20= */=0A+=09beginRelBlockIteration(&ctx);=0A+=09while=20= (relBlockIteration_next(&ctx))=0A+=09{=0A+=09=09/*=20Perform=20tuple=20= checks=20*/=0A+=09=09beginPageTupleIteration(&ctx);=0A+=09=09while=20= (pageTupleIteration_next(&ctx))=0A+=09=09=09check_tuple(&ctx);=0A+=09=09= endPageTupleIteration(&ctx);=0A+=09}=0A+=09endRelBlockIteration(&ctx);=0A= +=0A+=09/*=20Close=20the=20associated=20toast=20table=20and=20indexes,=20= if=20any.=20*/=0A+=09if=20(ctx.has_toastrel)=0A+=09{=0A+=09=09= toast_close_indexes(ctx.toast_indexes,=20ctx.num_toast_indexes,=0A+=09=09= =09=09=09=09=09AccessShareLock);=0A+=09=09table_close(ctx.toastrel,=20= AccessShareLock);=0A+=09}=0A+=0A+=09/*=20Close=20the=20main=20relation=20= */=0A+=09relation_close(ctx.rel,=20AccessShareLock);=0A+=0A+=09return=20= ctx.corruption;=0A+}=0A+=0A+/*=0A+=20*=20check_relation_relkind=0A+=20*=0A= +=20*=20=20=20convenience=20routine=20to=20check=20that=20relation=20is=20= of=20a=20supported=20relkind.=0A+=20*/=0A+void=0A= +check_relation_relkind(Relation=20rel)=0A+{=0A+=09if=20= (rel->rd_rel->relkind=20!=3D=20RELKIND_RELATION=20&&=0A+=09=09= rel->rd_rel->relkind=20!=3D=20RELKIND_MATVIEW=20&&=0A+=09=09= rel->rd_rel->relkind=20!=3D=20RELKIND_TOASTVALUE)=0A+=09=09= ereport(ERROR,=0A+=09=09=09=09(errcode(ERRCODE_WRONG_OBJECT_TYPE),=0A+=09= =09=09=09=20errmsg("\"%s\"=20is=20not=20a=20table,=20materialized=20= view,=20"=0A+=09=09=09=09=09=09"or=20TOAST=20table",=0A+=09=09=09=09=09=09= RelationGetRelationName(rel))));=0A+=09if=20(rel->rd_rel->relam=20!=3D=20= HEAP_TABLE_AM_OID)=0A+=09=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_WRONG_OBJECT_TYPE),=0A+=09=09=09=09=20errmsg("\"%s\"=20= is=20not=20a=20heap=20AM",=0A+=09=09=09=09=09=09= RelationGetRelationName(rel))));=0A+}=0Adiff=20--git=20= a/contrib/heapcheck/heapcheck.control=20= b/contrib/heapcheck/heapcheck.control=0Anew=20file=20mode=20100644=0A= index=200000000000..23b076169e=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/heapcheck.control=0A@@=20-0,0=20+1,5=20@@=0A+#=20= heapcheck=20extension=0A+comment=20=3D=20'examine=20relations=20for=20= corruption'=0A+default_version=20=3D=20'1.0'=0A+module_pathname=20=3D=20= '$libdir/heapcheck'=0A+relocatable=20=3D=20true=0Adiff=20--git=20= a/contrib/heapcheck/sql/001_create_extension.sql=20= b/contrib/heapcheck/sql/001_create_extension.sql=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..0ca79c22be=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/sql/001_create_extension.sql=0A@@=20-0,0=20+1=20@@=0A= +create=20extension=20heapcheck;=0Adiff=20--git=20= a/contrib/heapcheck/sql/002_disallowed_reltypes.sql=20= b/contrib/heapcheck/sql/002_disallowed_reltypes.sql=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..782e2c7039=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/sql/002_disallowed_reltypes.sql=0A@@=20-0,0=20+1,29=20= @@=0A+--=0A+--=20check=20that=20using=20the=20module's=20functions=20= with=20unsupported=20relations=20will=20fail=0A+--=0A+=0A+--=20= partitioned=20tables=20(the=20parent=20ones)=20don't=20have=20visibility=20= maps=0A+create=20table=20test_partitioned=20(a=20int,=20b=20text=20= default=20repeat('x',=205000))=20partition=20by=20list=20(a);=0A+--=20= these=20should=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_partitioned');=0A+=0A+create=20table=20= test_partition=20partition=20of=20test_partitioned=20for=20values=20in=20= (1);=0A+create=20index=20test_index=20on=20test_partition=20(a);=0A+--=20= indexes=20do=20not,=20so=20these=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_index');=0A+=0A+create=20view=20test_view=20as=20= select=201;=0A+--=20views=20do=20not=20have=20vms,=20so=20these=20all=20= fail=0A+select=20*=20from=20heapcheck_relation('test_view');=0A+=0A= +create=20sequence=20test_sequence;=0A+--=20sequences=20do=20not=20have=20= vms,=20so=20these=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_sequence');=0A+=0A+create=20foreign=20data=20= wrapper=20dummy;=0A+create=20server=20dummy_server=20foreign=20data=20= wrapper=20dummy;=0A+create=20foreign=20table=20test_foreign_table=20()=20= server=20dummy_server;=0A+--=20foreign=20tables=20do=20not=20have=20vms,=20= so=20these=20all=20fail=0A+select=20*=20from=20= heapcheck_relation('test_foreign_table');=0A+=0A+=0Adiff=20--git=20= a/contrib/heapcheck/t/003_heapcheck_relation.pl=20= b/contrib/heapcheck/t/003_heapcheck_relation.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..8630ac798b=0A---=20/dev/null=0A+++=20= b/contrib/heapcheck/t/003_heapcheck_relation.pl=0A@@=20-0,0=20+1,361=20= @@=0A+use=20strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20= TestLib;=0A+=0A+use=20Test::More=20tests=20=3D>=201;=0A+=0A+#=20This=20= regression=20test=20demonstrates=20that=20the=20heapcheck_relation()=20= function=0A+#=20supplied=20with=20this=20contrib=20module=20correctly=20= identifies=20specific=20kinds=20of=0A+#=20corruption=20within=20pages.=20= =20To=20test=20this,=20we=20need=20a=20mechanism=20to=20create=20corrupt=0A= +#=20pages=20with=20predictable,=20repeatable=20corruption.=20=20The=20= postgres=20backend=20cannot=20be=0A+#=20expected=20to=20help=20us=20with=20= this,=20as=20its=20design=20is=20not=20consistent=20with=20the=20goal=0A= +#=20of=20intentionally=20corrupting=20pages.=0A+#=0A+#=20Instead,=20we=20= create=20a=20table=20to=20corrupt,=20and=20with=20careful=20= consideration=20of=20how=0A+#=20postgresql=20lays=20out=20heap=20pages,=20= we=20seek=20to=20offsets=20within=20the=20page=20and=0A+#=20overwrite=20= deliberately=20chosen=20bytes=20with=20specific=20values=20calculated=20= to=0A+#=20corrupt=20the=20page=20in=20expected=20ways.=20=20We=20then=20= verify=20that=20heapcheck_relation=0A+#=20reports=20the=20corruption,=20= and=20that=20it=20runs=20without=20crashing.=20=20Note=20that=20the=0A+#=20= backend=20cannot=20simply=20be=20started=20to=20run=20queries=20against=20= the=20corrupt=20table,=20as=0A+#=20the=20backend=20will=20crash,=20at=20= least=20for=20some=20of=20the=20corruption=20types=20we=0A+#=20generate.=0A= +#=0A+#=20Autovacuum=20potentially=20touching=20the=20table=20in=20the=20= background=20makes=20the=20exact=0A+#=20behavior=20of=20this=20test=20= harder=20to=20reason=20about.=20=20We=20turn=20it=20off=20to=20keep=20= things=0A+#=20simpler.=20=20We=20use=20a=20"belt=20and=20suspenders"=20= approach,=20turning=20it=20off=20for=20the=0A+#=20system=20generally=20= in=20postgresql.conf,=20and=20turning=20it=20off=20specifically=20for=20= the=0A+#=20test=20table.=0A+#=0A+#=20This=20test=20depends=20on=20the=20= table=20being=20written=20to=20the=20heap=20file=20exactly=20as=20we=0A= +#=20expect=20it=20to=20be,=20so=20we=20take=20care=20to=20arrange=20the=20= columns=20of=20the=20table,=20and=0A+#=20insert=20rows=20of=20the=20= table,=20that=20give=20predictable=20sizes=20and=20locations=20within=0A= +#=20the=20table=20page.=0A+#=0A+#=20The=20HeapTupleHeaderData=20has=20= 23=20bytes=20of=20fixed=20size=20fields=20before=20the=20variable=0A+#=20= length=20t_bits[]=20array.=20=20We=20have=20exactly=203=20columns=20in=20= the=20table,=20so=20natts=20=3D=203,=0A+#=20t_bits=20is=201=20byte=20= long,=20and=20t_hoff=20=3D=20MAXALIGN(23=20+=201)=20=3D=2024.=0A+#=0A+#=20= We're=20not=20too=20fussy=20about=20which=20datatypes=20we=20use=20for=20= the=20test,=20but=20we=20do=20care=0A+#=20about=20some=20specific=20= properties.=20=20We'd=20like=20to=20test=20both=20fixed=20size=20and=0A= +#=20varlena=20types.=20=20We'd=20like=20some=20varlena=20data=20inline=20= and=20some=20toasted.=20=20And=0A+#=20we'd=20like=20the=20layout=20of=20= the=20table=20such=20that=20the=20datums=20land=20at=20predictable=0A+#=20= offsets=20within=20the=20tuple.=20=20We=20choose=20a=20structure=20= without=20padding=20on=20all=0A+#=20supported=20architectures:=0A+#=0A+#=20= =09a=20BIGINT=0A+#=09b=20TEXT=0A+#=09c=20TEXT=0A+#=0A+#=20We=20always=20= insert=20a=207-ascii=20character=20string=20into=20field=20'b',=20which=20= with=20a=0A+#=201-byte=20varlena=20header=20gives=20an=208=20byte=20= inline=20value.=20=20We=20always=20insert=20a=20long=0A+#=20text=20= string=20in=20field=20'c',=20long=20enough=20to=20force=20toast=20= storage.=0A+#=0A+#=20This=20formatting=20produces=20heap=20pages=20where=20= each=20tuple=20is=2058=20bytes=20long,=20padded=0A+#=20out=20to=2064=20= bytes=20for=20alignment,=20with=20the=20first=20one=20on=20the=20page=20= starting=20at=0A+#=20offset=208128,=20as=20follows:=0A+#=0A+#=20=20=20=20= [=20lp_off:=208128=20lp_len:=20=20=2058]=0A+#=20=20=20=20[=20lp_off:=20= 8064=20lp_len:=20=20=2058]=0A+#=20=20=20=20[=20lp_off:=208000=20lp_len:=20= =20=2058]=0A+#=20=20=20=20[=20lp_off:=207936=20lp_len:=20=20=2058]=0A+#=20= =20=20=20[=20lp_off:=207872=20lp_len:=20=20=2058]=0A+#=20=20=20=20[=20= lp_off:=207808=20lp_len:=20=20=2058]=0A+#=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20...=0A+#=0A+=0A+use=20constant=20LP_OFF_BEGIN=20=3D>=208128;=0A= +use=20constant=20LP_OFF_DELTA=20=3D>=2064;=0A+=0A+#=20We=20choose=20to=20= read=20and=20write=20binary=20copies=20of=20our=20table's=20tuples,=20= using=20perl's=0A+#=20pack()=20and=20unpack()=20functions.=20=20Perl=20= uses=20a=20packing=20code=20system=20in=20which:=0A+#=0A+#=09L=20=3D=20= "Unsigned=2032-bit=20Long",=0A+#=09S=20=3D=20"Unsigned=2016-bit=20= Short",=0A+#=09C=20=3D=20"Unsigned=208-bit=20Octet",=0A+#=09c=20=3D=20= "signed=208-bit=20octet",=0A+#=09q=20=3D=20"signed=2064-bit=20quadword"=0A= +#=09=0A+#=20Each=20tuple=20in=20our=20table=20has=20a=20layout=20as=20= follows:=0A+#=0A+#=20=20=20=20xx=20xx=20xx=20xx=20=20=20=20=20=20=20=20=20= =20=20=20t_xmin:=20xxxx=09=09offset=20=3D=200=09=09L=0A+#=20=20=20=20xx=20= xx=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20t_xmax:=20xxxx=09=09= offset=20=3D=204=09=09L=0A+#=20=20=20=20xx=20xx=20xx=20xx=20=20=20=20=20=20= =20=20=20=20t_field3:=20xxxx=09=09offset=20=3D=208=09=09L=0A+#=20=20=20=20= xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20bi_hi:=20= xx=09=09=09offset=20=3D=2012=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20bi_lo:=20xx=09=09=09offset=20=3D= =2014=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20ip_posid:=20xx=09=09=09offset=20=3D=2016=09=09S=0A+#=20=20=20=20= xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20t_infomask2:=20xx=09=09=09= offset=20=3D=2018=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20= =20=20=20=20=20t_infomask:=20xx=09=09=09offset=20=3D=2020=09=09S=0A+#=20=20= =20=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= t_hoff:=20x=09=09=09offset=20=3D=2022=09=09C=0A+#=20=20=20=20xx=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20t_bits:=20x=09=09=09= offset=20=3D=2023=09=09C=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20= xx=20xx=20=20=20'a':=20xxxxxxxx=09offset=20=3D=2024=09=09q=0A+#=20=20=20=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'b':=20xxxxxxxx=09offset=20= =3D=2032=09=09Cccccccc=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20= xx=20=20=20'c':=20xxxxxxxx=09offset=20=3D=2040=09=09SSSS=0A+#=20=20=20=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20=20=20=20:=20xxxxxxxx=09=20= ...continued=09SSSS=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20:=20xx=20=20=20=20=20=20=09=20= ...continued=09S=0A+#=09=0A+#=20We=20could=20choose=20to=20read=20and=20= write=20columns=20'b'=20and=20'c'=20in=20other=20ways,=20but=0A+#=20it=20= is=20convenient=20enough=20to=20do=20it=20this=20way.=20=20We=20define=20= packing=20code=0A+#=20constants=20here,=20where=20they=20can=20be=20= compared=20easily=20against=20the=20layout.=0A+=0A+use=20constant=20= HEAPTUPLE_PACK_CODE=20=3D>=20'LLLSSSSSCCqCcccccccSSSSSSSSS';=0A+use=20= constant=20HEAPTUPLE_PACK_LENGTH=20=3D>=2058;=20=20=20=20=20#=20Total=20= size=0A+=0A+#=20Read=20a=20tuple=20of=20our=20table=20from=20a=20heap=20= page.=0A+#=0A+#=20Takes=20an=20open=20filehandle=20to=20the=20heap=20= file,=20and=20the=20offset=20of=20the=20tuple.=0A+#=0A+#=20Rather=20than=20= returning=20the=20binary=20data=20from=20the=20file,=20unpacks=20the=20= data=20into=20a=0A+#=20perl=20hash=20with=20named=20fields.=20=20These=20= fields=20exactly=20match=20the=20ones=20understood=0A+#=20by=20= write_tuple(),=20below.=20=20Returns=20a=20reference=20to=20this=20hash.=0A= +#=0A+sub=20read_tuple=20($$)=0A+{=0A+=09my=20($fh,=20$offset)=20=3D=20= @_;=0A+=09my=20($buffer,=20%tup);=0A+=09seek($fh,=20$offset,=200);=0A+=09= sysread($fh,=20$buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=09=0A+=09@_=20=3D=20= unpack(HEAPTUPLE_PACK_CODE,=20$buffer);=0A+=09%tup=20=3D=20(t_xmin=20=3D>=20= shift,=0A+=09=09=09t_xmax=20=3D>=20shift,=0A+=09=09=09t_field3=20=3D>=20= shift,=0A+=09=09=09bi_hi=20=3D>=20shift,=0A+=09=09=09bi_lo=20=3D>=20= shift,=0A+=09=09=09ip_posid=20=3D>=20shift,=0A+=09=09=09t_infomask2=20=3D>= =20shift,=0A+=09=09=09t_infomask=20=3D>=20shift,=0A+=09=09=09t_hoff=20=3D>= =20shift,=0A+=09=09=09t_bits=20=3D>=20shift,=0A+=09=09=09a=20=3D>=20= shift,=0A+=09=09=09b_header=20=3D>=20shift,=0A+=09=09=09b_body1=20=3D>=20= shift,=0A+=09=09=09b_body2=20=3D>=20shift,=0A+=09=09=09b_body3=20=3D>=20= shift,=0A+=09=09=09b_body4=20=3D>=20shift,=0A+=09=09=09b_body5=20=3D>=20= shift,=0A+=09=09=09b_body6=20=3D>=20shift,=0A+=09=09=09b_body7=20=3D>=20= shift,=0A+=09=09=09c1=20=3D>=20shift,=0A+=09=09=09c2=20=3D>=20shift,=0A+=09= =09=09c3=20=3D>=20shift,=0A+=09=09=09c4=20=3D>=20shift,=0A+=09=09=09c5=20= =3D>=20shift,=0A+=09=09=09c6=20=3D>=20shift,=0A+=09=09=09c7=20=3D>=20= shift,=0A+=09=09=09c8=20=3D>=20shift,=0A+=09=09=09c9=20=3D>=20shift);=0A= +=09#=20Stitch=20together=20the=20text=20for=20column=20'b'=0A+=09= $tup{b}=20=3D=20join('',=20map=20{=20chr($tup{"b_body$_"})=20}=20= (1..7));=0A+=09return=20\%tup;=0A+}=0A+=0A+#=20Write=20a=20tuple=20of=20= our=20table=20to=20a=20heap=20page.=0A+#=0A+#=20Takes=20an=20open=20= filehandle=20to=20the=20heap=20file,=20the=20offset=20of=20the=20tuple,=20= and=20a=0A+#=20reference=20to=20a=20hash=20with=20the=20tuple=20values,=20= as=20returned=20by=20read_tuple().=0A+#=20Writes=20the=20tuple=20fields=20= from=20the=20hash=20into=20the=20heap=20file.=0A+#=0A+#=20The=20purpose=20= of=20this=20function=20is=20to=20write=20a=20tuple=20back=20to=20disk=20= with=20some=0A+#=20subset=20of=20fields=20modified.=20=20The=20function=20= does=20no=20error=20checking.=20=20Use=0A+#=20cautiously.=0A+#=0A+sub=20= write_tuple($$$)=0A+{=0A+=09my=20($fh,=20$offset,=20$tup)=20=3D=20@_;=0A= +=09my=20$buffer=20=3D=20pack(HEAPTUPLE_PACK_CODE,=0A+=09=09=09=09=09= $tup->{t_xmin},=0A+=09=09=09=09=09$tup->{t_xmax},=0A+=09=09=09=09=09= $tup->{t_field3},=0A+=09=09=09=09=09$tup->{bi_hi},=0A+=09=09=09=09=09= $tup->{bi_lo},=0A+=09=09=09=09=09$tup->{ip_posid},=0A+=09=09=09=09=09= $tup->{t_infomask2},=0A+=09=09=09=09=09$tup->{t_infomask},=0A+=09=09=09=09= =09$tup->{t_hoff},=0A+=09=09=09=09=09$tup->{t_bits},=0A+=09=09=09=09=09= $tup->{a},=0A+=09=09=09=09=09$tup->{b_header},=0A+=09=09=09=09=09= $tup->{b_body1},=0A+=09=09=09=09=09$tup->{b_body2},=0A+=09=09=09=09=09= $tup->{b_body3},=0A+=09=09=09=09=09$tup->{b_body4},=0A+=09=09=09=09=09= $tup->{b_body5},=0A+=09=09=09=09=09$tup->{b_body6},=0A+=09=09=09=09=09= $tup->{b_body7},=0A+=09=09=09=09=09$tup->{c1},=0A+=09=09=09=09=09= $tup->{c2},=0A+=09=09=09=09=09$tup->{c3},=0A+=09=09=09=09=09$tup->{c4},=0A= +=09=09=09=09=09$tup->{c5},=0A+=09=09=09=09=09$tup->{c6},=0A+=09=09=09=09= =09$tup->{c7},=0A+=09=09=09=09=09$tup->{c8},=0A+=09=09=09=09=09= $tup->{c9});=0A+=09seek($fh,=20$offset,=200);=0A+=09syswrite($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=09return;=0A+}=0A+=0A+#=20Set=20= umask=20so=20test=20directories=20and=20files=20are=20created=20with=20= default=20permissions=0A+umask(0077);=0A+=0A+my=20($result,=20$node);=0A= +=0A+#=20Set=20up=20the=20node=20and=20test=20table.=0A+$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A= +$node->append_conf('postgresql.conf',=20'autovacuum=3Doff');=0A= +$node->start;=0A+my=20$pgdata=20=3D=20$node->data_dir;=0A= +$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20heapcheck");=0A+=0A= +$node->safe_psql(=0A+=09'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20= public.test=20(a=20BIGINT,=20b=20TEXT,=20c=20TEXT);=0A+=09=09ALTER=20= TABLE=20public.test=20SET=20(autovacuum_enabled=3Dfalse);=0A+=09=09ALTER=20= TABLE=20public.test=20ALTER=20COLUMN=20c=20SET=20STORAGE=20EXTERNAL;=0A+=09= ));=0A+=0A+my=20$rel=20=3D=20$node->safe_psql('postgres',=20qq(SELECT=20= pg_relation_filepath('public.test')));=0A+my=20$relpath=20=3D=20= "$pgdata/$rel";=0A+=0A+use=20constant=20ROWCOUNT=20=3D>=2012;=0A= +$node->safe_psql('postgres',=20qq(=0A+=09INSERT=20INTO=20public.test=20= (a,=20b,=20c)=0A+=09=09VALUES=20(=0A+=09=09=0912345678,=0A+=09=09=09= repeat('f',=207),=0A+=09=09=09repeat('w',=2010000)=0A+=09=09);=0A+=09= VACUUM=20FREEZE=20public.test=0A+=09))=20for=20(1..ROWCOUNT);=0A+=0A+my=20= $relfrozenxid=20=3D=20$node->safe_psql('postgres',=0A+=09q(select=20= relfrozenxid=20from=20pg_class=20where=20relname=20=3D=20'test'));=0A+=0A= +$node->stop;=0A+=0A+#=20Some=20#define=20constants=20from=20= access/htup_details.h=20for=20use=20while=20corrupting.=0A+use=20= constant=20HEAP_HASNULL=20=20=20=20=20=20=20=20=20=20=20=20=3D>=20= 0x0001;=0A+use=20constant=20HEAP_XMIN_COMMITTED=20=20=20=20=20=3D>=20= 0x0100;=0A+use=20constant=20HEAP_XMIN_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0200;=0A+use=20constant=20HEAP_XMAX_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0800;=0A+use=20constant=20HEAP_NATTS_MASK=20=20=20=20=20=20=20=20=20=3D>= =200x07FF;=0A+=0A+#=20Corrupt=20the=20tuples,=20one=20type=20of=20= corruption=20per=20tuple.=20=20Some=20types=20of=0A+#=20corruption=20= cause=20heapcheck_relation=20to=20skip=20to=20the=20next=20tuple=20= without=0A+#=20performing=20any=20remaining=20checks,=20so=20we=20can't=20= exercise=20the=20system=20properly=20if=0A+#=20we=20focus=20all=20our=20= corruption=20on=20a=20single=20tuple.=0A+#=0A+my=20$file;=0A+open($file,=20= '+<',=20$relpath);=0A+binmode=20$file;=0A+for=20(my=20$offset=20=3D=20= LP_OFF_BEGIN,=20my=20$tupidx=20=3D=200;=20$tupidx=20<=20ROWCOUNT;=20= $tupidx++,=20$offset=20-=3D=20LP_OFF_DELTA)=0A+{=0A+=09my=20$tup=20=3D=20= read_tuple($file,=20$offset);=0A+=0A+=09if=20($tupidx=20=3D=3D=200)=0A+=09= {=0A+=09=09#=20Corruptly=20set=20xmin=20<=20relfrozenxid=0A+=09=09= $tup->{t_xmin}=20=3D=203;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=09}=0A+=09elsif=20($tupidx=20=3D=3D=201)=0A+=09{=0A= +=09=09#=20Corruptly=20set=20xmin=20<=20relfrozenxid,=20further=20back=0A= +=09=09$tup->{t_xmin}=20=3D=204026531839;=09=09#=20Note=20circularity=20= of=20xid=20comparison=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=09}=0A+=09elsif=20($tupidx=20=3D=3D=202)=0A+=09{=0A= +=09=09#=20Corruptly=20set=20xmax=20<=20relminmxid;=0A+=09=09= $tup->{t_xmax}=20=3D=204026531839;=09=09#=20Note=20circularity=20of=20= xid=20comparison=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMAX_INVALID;=0A+=09}=0A+=09elsif=20($tupidx=20=3D=3D=203)=0A+=09{=0A= +=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20but=20keep=20it=20aligned=20= properly=0A+=09=09$tup->{t_hoff}=20+=3D=20128;=0A+=09}=0A+=09elsif=20= ($tupidx=20=3D=3D=204)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20= t_hoff,=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20+=3D=203;=0A+=09}=0A= +=09elsif=20($tupidx=20=3D=3D=205)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= tuple=20t_hoff,=20underflow=20but=20correct=20alignment=0A+=09=09= $tup->{t_hoff}=20-=3D=208;=0A+=09}=0A+=09elsif=20($tupidx=20=3D=3D=206)=0A= +=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20underflow=20and=20= wrong=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=203;=0A+=09}=0A+=09= elsif=20($tupidx=20=3D=3D=207)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= tuple=20to=20look=20like=20it=20has=20lots=20of=20attributes,=20not=20= just=203=0A+=09=09$tup->{t_infomask2}=20|=3D=20HEAP_NATTS_MASK;=0A+=09}=0A= +=09elsif=20($tupidx=20=3D=3D=208)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= tuple=20to=20look=20like=20it=20has=20lots=20of=20attributes,=20some=20= of=0A+=09=09#=20them=20null.=20=20This=20falsely=20creates=20the=20= impression=20that=20the=20t_bits=0A+=09=09#=20array=20is=20longer=20than=20= just=20one=20byte,=20but=20t_hoff=20still=20says=20otherwise.=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09$tup->{t_infomask2}=20= |=3D=20HEAP_NATTS_MASK;=0A+=09=09$tup->{t_bits}=20=3D=200xAA;=0A+=09}=0A= +=09elsif=20($tupidx=20=3D=3D=209)=0A+=09{=0A+=09=09#=20Same=20as=20= above,=20but=20this=20time=20t_hoff=20plays=20along=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09$tup->{t_infomask2}=20= |=3D=20(HEAP_NATTS_MASK=20&=200x40);=0A+=09=09$tup->{t_bits}=20=3D=20= 0xAA;=0A+=09=09$tup->{t_hoff}=20=3D=2032;=0A+=09}=0A+=09elsif=20($tupidx=20= =3D=3D=2010)=0A+=09{=0A+=09=09#=20Corrupt=20the=20bits=20in=20column=20= 'b'=201-byte=20varlena=20header=0A+=09=09$tup->{b_header}=20=3D=200x80;=0A= +=09}=0A+=09elsif=20($tupidx=20=3D=3D=2011)=0A+=09{=0A+=09=09#=20Corrupt=20= the=20bits=20in=20column=20'c'=20toast=20pointer=0A+=09=09$tup->{c6}=20=3D= =2041;=0A+=09=09$tup->{c7}=20=3D=2041;=0A+=09}=0A+=09write_tuple($file,=20= $offset,=20$tup);=0A+}=0A+close($file);=0A+=0A+#=20Run=20= heapcheck_relation=20on=20the=20corrupted=20file=0A+$node->start;=0A+=0A= +$result=20=3D=20$node->safe_psql('postgres',=20q(SELECT=20*=20FROM=20= heapcheck_relation('test')));=0A+is=20($result,=0A= +"0|1|8128|1|58|||tuple=20xmin=20=3D=203=20precedes=20relation=20= relfrozenxid=20=3D=20$relfrozenxid=0A+0|1|8128|1|58|||tuple=20xmin=20=3D=20= 3=20(interpreted=20as=203)=20not=20or=20no=20longer=20valid=0A= +0|2|8064|1|58|||tuple=20xmin=20=3D=204026531839=20precedes=20relation=20= relfrozenxid=20=3D=20$relfrozenxid=0A+0|2|8064|1|58|||tuple=20xmin=20=3D=20= 4026531839=20(interpreted=20as=2018446744073441116159)=20not=20or=20no=20= longer=20valid=0A+0|3|8000|1|58|||tuple=20xmax=20=3D=204026531839=20= precedes=20relation=20relfrozenxid=20=3D=20$relfrozenxid=0A= +0|4|7936|1|58|||t_hoff=20>=20lp_len=20(152=20>=2058)=0A= +0|5|7872|1|58|||t_hoff=20not=20max-aligned=20(27)=0A= +0|6|7808|1|58|||t_hoff=20<=20SizeofHeapTupleHeader=20(16=20<=2023)=0A= +0|7|7744|1|58|||t_hoff=20<=20SizeofHeapTupleHeader=20(21=20<=2023)=0A= +0|7|7744|1|58|||t_hoff=20not=20max-aligned=20(21)=0A= +0|8|7680|1|58|||relation=20natts=20<=20tuple=20natts=20(3=20<=202047)=0A= +0|9|7616|1|58|||SizeofHeapTupleHeader=20+=20BITMAPLEN(natts)=20>=20= t_hoff=20(23=20+=20256=20>=2024)=0A+0|10|7552|1|58|||relation=20natts=20= <=20tuple=20natts=20(3=20<=2067)=0A+0|11|7488|1|58|2||t_hoff=20+=20= offset=20>=20lp_len=20(24=20+=20429496744=20>=2058)=0A= +0|12|7424|1|58|2|0|final=20chunk=20number=20differs=20from=20expected=20= (0=20vs.=206)=0A+0|12|7424|1|58|2|0|toasted=20value=20missing=20from=20= toast=20table",=0A+"Expected=20heapcheck_relation=20output");=0A+=0A= +$node->teardown_node;=0A+$node->clean_node;=0A+=0Adiff=20--git=20= a/doc/src/sgml/contrib.sgml=20b/doc/src/sgml/contrib.sgml=0Aindex=20= 261a559e81..f32b8ac5ef=20100644=0A---=20a/doc/src/sgml/contrib.sgml=0A= +++=20b/doc/src/sgml/contrib.sgml=0A@@=20-110,6=20+110,7=20@@=20CREATE=20= EXTENSION=20module_name;=0A=20=20= &earthdistance;=0A=20=20&file-fdw;=0A=20=20&fuzzystrmatch;=0A+=20= &heapcheck;=0A=20=20&hstore;=0A=20=20&intagg;=0A=20=20&intarray;=0Adiff=20= --git=20a/doc/src/sgml/filelist.sgml=20b/doc/src/sgml/filelist.sgml=0A= index=2068179f71cd..b43d72b8bb=20100644=0A---=20= a/doc/src/sgml/filelist.sgml=0A+++=20b/doc/src/sgml/filelist.sgml=0A@@=20= -122,6=20+122,7=20@@=0A=20=0A=20=0A=20=0A+=0A=20=0A=20=0A=20=0Adiff=20--git=20a/doc/src/sgml/heapcheck.sgml=20= b/doc/src/sgml/heapcheck.sgml=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..0a9942a452=0A---=20/dev/null=0A+++=20= b/doc/src/sgml/heapcheck.sgml=0A@@=20-0,0=20+1,133=20@@=0A+=0A+=0A+=0A+=20heapcheck=0A+=0A+=20= =0A+=20=20heapcheck=0A= +=20=0A+=0A+=20=0A+=20=20The=20= heapcheck=20module=20provides=20a=20means=20for=20= examining=20the=0A+=20=20integrity=20of=20a=20table=20relation.=0A+=20= =0A+=0A+=20=0A+=20=20Functions=0A+=0A+=20=20= =0A+=20=20=20=0A+=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20=20heapcheck_relation(relation=20= regclass,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20blkno=20OUT=20bigint,=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20offnum=20OUT=20integer,=0A+=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= lp_off=20OUT=20smallint,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20lp_flags=20OUT=20smallint,=0A+=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20lp_len=20OUT=20= smallint,=0A+=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20attnum=20OUT=20integer,=0A+=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20chunk=20OUT=20integer,=0A+=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20msg=20= OUT=20text)=0A+=20=20=20=20=20=20returns=20record=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=20=20Checks=20for=20"logical"=20corruption,=20= where=20the=20page=20is=20valid=20but=20inconsistent=0A+=20=20=20=20=20=20= with=20the=20rest=20of=20the=20database=20cluster.=20This=20can=20happen=20= due=20to=20faulty=20or=0A+=20=20=20=20=20=20ill-conceived=20backup=20and=20= restore=20tools,=20or=20bad=20storage,=20or=20user=20error,=20or=0A+=20=20= =20=20=20=20bugs=20in=20the=20server=20itself.=20=20It=20checks=20xmin=20= and=20xmax=20values=20against=0A+=20=20=20=20=20=20relfrozenxid=20and=20= relminmxid,=20and=20also=20validates=20TOAST=20pointers.=0A+=20=20=20=20=20= =0A+=0A+=20=20=20=20=20=0A+=20=20=20=20=20=20For=20each=20= block=20in=20the=20relation=20where=20corruption=20is=20detected,=20for=20= each=0A+=20=20=20=20=20=20corruption=20detected,=20returns=20one=20row=20= containing=20the=20following=20fields:=0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20blkno=0A+=20=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20number=20of=20= the=20block=20containing=20the=20corrupt=20page.=0A+=20=20=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20offnum=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20OffsetNumber=20of=20= the=20corrupt=20tuple.=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20lp_off=0A+=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=20The=20offset=20into=20the=20page=20of=20the=20line=20= pointer=20for=20the=20corrupt=20tuple.=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20lp_flags=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20flags=20in=20the=20= line=20pointer=20for=20the=20corrupt=20tuple.=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20lp_len=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20length=20of=20the=20= corrupt=20tuple=20as=20recorded=20in=20the=20line=20pointer.=0A+=20=20=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20attnum=0A+=20=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20attribute=20= number=20of=20the=20corrupt=20column=20in=20the=20tuple,=20if=20the=20= corruption=0A+=20=20=20=20=20=20=20=20=20is=20specific=20to=20a=20column=20= and=20not=20the=20tuple=20as=20a=20whole.=0A+=20=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20chunk=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=20=20=20The=20chunk=20number=20of=20= the=20corrupt=20toasted=20attribute,=20if=20the=20corruption=0A+=20=20=20= =20=20=20=20=20=20is=20specific=20to=20a=20toasted=20value.=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20msg=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=20=20A=20human=20readable=20= message=20describing=20the=20corruption=20in=20the=20page.=0A+=20=20=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=20=20=20=0A+=20=20=0A+=0A+=20= =20=0A+=20=20=20By=20default,=20this=20function=20is=20executable=20= only=20by=20superusers=20and=20members=0A+=20=20=20of=20the=20= pg_stat_scan_tables=20role.=0A+=20=20=0A+=20= =0A+=0A+=20=0A+=20=20Author=0A+=0A+=20=20= =0A+=20=20=20Mark=20Dilger=20= mark.dilger@enterprisedb.com=0A+=20=20=0A+=20= =0A+=0A+=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_4BC3E30B-3A68-4D41-9C2D-C21A8D3F28D4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 =E2=80=94 Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --Apple-Mail=_4BC3E30B-3A68-4D41-9C2D-C21A8D3F28D4--