Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lHTKi-0007HB-6v for pgsql-hackers@arkaria.postgresql.org; Wed, 03 Mar 2021 15:22:41 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lHTKh-0004Ws-40 for pgsql-hackers@arkaria.postgresql.org; Wed, 03 Mar 2021 15:22:39 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lHTKg-0004Wl-I0 for pgsql-hackers@lists.postgresql.org; Wed, 03 Mar 2021 15:22:38 +0000 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lHTKb-0007IC-C8 for pgsql-hackers@postgresql.org; Wed, 03 Mar 2021 15:22:37 +0000 Received: by mail-pl1-x633.google.com with SMTP id j6so645661plx.6 for ; Wed, 03 Mar 2021 07:22:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=from:mime-version:subject:message-id:date:cc:to; bh=Ez7i6j5/qSa/pQrGnBsRivIeWrOGFbuiMEzXqfoJvLY=; b=oSxc9aMQe0fzS6GAIDBiAFmrcXNFry20nfwM/YWJQUla4+G22vXxDQFW1WsRXTtPjG GXnqTpREqyn96t2VphC+2yM3XQRGQejQ5Ol0XP7wCmVp9SymtauN86hsnZ8d9BbEJBSz NLgPPU/zL4+b9+MA33G+aSMahqgWrIrGMqZO8tJMASpUf+L3+jyqpMlcBEnTd67AAYL5 iGFBlDHzmVDKPPBcsTEq0k6TFMmJhJGkvYPPZk/3QA2HXMpnxzvj6rAnG/ZHgiSZIhqO OsLf3xrghWgNrfYOHWz9xPrUHhZ5/PiKToHHx8MwMY5g036vaA4azsgsJiCe4rth1X4O buaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:cc:to; bh=Ez7i6j5/qSa/pQrGnBsRivIeWrOGFbuiMEzXqfoJvLY=; b=R8sBIJPTl+bAlBmQ6LHzbM9HIXTgQr3zg+yxvPQFxZqR+2u9bCR5Bp/e1vCbN2NbB6 +RmBbc7Er5NSXS2tSEZynxvZ7ljh067UCKLwvNJETITiDizqdp05n4lu/5UpsYCf+yda zD3Tl/lIc4u51lNmnQ2Lzp4XJiEgGWIZhARCrBUmwvDang32VoTGHHE3yYlZhVekQEX0 ChJuu9VxQ+3rMPi+aHSSgW+8UGXq1TkBqXI2uMbaq5ZdYURwFjT5XCZbTtSzy+I0dL/B pMYQzq+LklJBsl+ziLfQmk+Vl8vsGZY+UZE1Hl1Q1t67pwdymNCUBEh72wrmQJE1bglE IFog== X-Gm-Message-State: AOAM532CjQSC3U/RP05kCnvQKc4yPywf4nD5igePnlSiVuXtuAkY0eWL ZNxDQCKHQQo6DFHqibtr9700A0o7xJIwu5FgYv8PKPtFIuPJzkr8dDRiGr9bpHLKg/B+d3X0C4q qQgzmCBSGN6riko7HeDsfdn6ue9YkHSaxGnifV11l0FDCdDnZdx9PxXpvVxUDe/kSTfDmtSEdp9 KQro/fgsV/S+a4NPgidMrpawBeQZtNkeu1n/hP X-Google-Smtp-Source: ABdhPJzkYO1tH2Z0sAg+6D0kdm48AeEfsciMUM8+qdURxMZb/e+q+syiroPSndiQU7h1brls6QuzXw== X-Received: by 2002:a17:90b:1c0e:: with SMTP id oc14mr9976714pjb.188.1614784950515; Wed, 03 Mar 2021 07:22:30 -0800 (PST) Received: from laptop280-ma-us.hitronhub.home (24-113-193-150.wavecable.com. [24.113.193.150]) by smtp.gmail.com with ESMTPSA id j5sm23318805pgl.55.2021.03.03.07.22.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Mar 2021 07:22:29 -0800 (PST) From: Mark Dilger Content-Type: multipart/mixed; boundary="Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: pg_amcheck contrib application Message-Id: Date: Wed, 3 Mar 2021 07:22:28 -0800 Cc: Thomas Munro , Tom Lane , Peter Geoghegan , "Andrey M. Borodin" , Stephen Frost , Michael Paquier , Amul Sul , Dilip Kumar , Andres Freund , PostgreSQL-development To: Robert Haas X-Mailer: Apple Mail (2.3608.120.23.2.1) X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii New thread, was "Re: new heapcheck contrib module" > On Mar 2, 2021, at 10:24 AM, Robert Haas = wrote: >=20 > On Tue, Mar 2, 2021 at 12:10 PM Mark Dilger > wrote: >> On further reflection, I decided to implement these changes and not = worry about the behavioral change. >=20 > Thanks. >=20 >> I skipped this part. The initcmd argument is only handed to = ParallelSlotsGetIdle(). Doing as you suggest would not really be = simpler, it would just move that argument to ParallelSlotsSetup(). But = I don't feel strongly about it, so I can move this, too, if you like. >>=20 >> I didn't do this either, and for the same reason. It's just a = parameter to ParallelSlotsGetIdle(), so nothing is really gained by = moving it to ParallelSlotsSetup(). >=20 > OK. I thought it was more natural to pass a bunch of arguments at > setup time rather than passing a bunch of arguments at get-idle time, > but I don't feel strongly enough about it to insist, and somebody else > can always change it later if they decide I had the right idea. When you originally proposed the idea, I thought that it would work out = as a simpler interface to have it your way, but in terms of the = interface it came out about the same. Internally it is still simpler to = do it your way, so since you seem to still like your way better, this = next version has it that way. >> Rather than the slots user tweak the slot's ConnParams, = ParallelSlotsGetIdle() takes a dbname argument, and uses it as = ConnParams->override_dbname. >=20 > OK, but you forgot to update the comments. ParallelSlotsGetIdle() > still talks about a cparams argument that it no longer has. Fixed. > The usual idiom for sizing a memory allocation involving > FLEXIBLE_ARRAY_MEMBER is something like offsetof(ParallelSlotArray, > slots) + numslots * sizeof(ParallelSlot). Your version uses sizeof(); > don't. Fixed. > Other than that 0001 looks to me to be in pretty good shape now. And your other review email, also moved to this new thread.... > On Mar 2, 2021, at 12:39 PM, Robert Haas = wrote: >=20 > On Tue, Mar 2, 2021 at 1:24 PM Robert Haas = wrote: >> Other than that 0001 looks to me to be in pretty good shape now. >=20 > Incidentally, we might want to move this to a new thread with a better > subject line, since the current subject line really doesn't describe > the uncommitted portion of the work. And create a new CF entry, too. Moved here. > Moving onto 0002: >=20 > The index checking options should really be called btree index > checking options. I think I'd put the table options first, and the > btree options second. Other kinds of indexes could follow some day. I > would personally omit the short forms of --heapallindexed and > --parent-check; I think we'll run out of option names too quickly if > people add more kinds of checks. Done. While doing this, I also renamed some of the variables to more closely = match the option name. I think the code is clearer now. >=20 > Perhaps VerifyBtreeSlotHandler should emit a warning of some kind if > PQntuples(res) !=3D 0. The functions bt_index_check and bt_index_parent_check are defined to = return VOID, which results in PQntuples(res) =3D=3D 1. I added code to = verify this condition, but it only serves to alert the user if the = amcheck version is behaving in an unexpected way, perhaps due to a = amcheck/pg_amcheck version mismatch. > + /* > + * Test that this function works, but for now we're > not using the list > + * 'relations' that it builds. > + */ > + conn =3D connectDatabase(&cparams, progname, = opts.echo, > false, true); >=20 > This comment appears to have nothing to do with the code, since > connectDatabase() does not build a list of 'relations'. True. Removed. > amcheck_sql seems to include paranoia, but do we need that if we're > using a secure search path? Similarly for other SQL queries, e.g. in > prepare_table_command. I removed the OPERATOR(pg_catalog.=3D) paranoia. > It might not be strictly necessary for the static functions in > pg_amcheck.c to use_three completelyDifferent NamingConventions for > its static functions. The idea is that the functions that interoperate with parallel slots = would follow its NamingConvention; those interoperating with = patternToSQLRegex and PQExpBuffers would follow their namingConvention; = and those not so interoperating would follow a less obnoxious = naming_convention. To my eye, that color codes the function names in a = useful way. To your eye, it just looks awful. I've changed it to use = just one naming_convention. > should_processing_continue() is one semicolon over budget. That's not the first time I've done that recently. Removed. > The initializer for opts puts a comma even after the last member > initializer. Is that going to be portable to all compilers? I don't know. I learned to put commas at the end of lists back when I = did mostly perl programming, as you get cleaner diffs when you add more = stuff to the list later. Whether I can get away with that in C using = initializers I don't know. I don't have a multiplicity of compilers to = check. I have removed the extra comma. > + for (failed =3D false, cell =3D opts.include.head; cell; cell =3D = cell->next) >=20 > I think failed has to be false here, because it gets initialized at > the top of the function. If we need to reinitialize it for some > reason, I would prefer you do that on the previous line, separate from > the for loop stuff. It does have to be false there. There is no need to reinitialize it. > + char *dbrgx; /* Database regexp parsed from = pattern, or > + * NULL */ > + char *nsprgx; /* Schema regexp parsed from pattern, = or NULL */ > + char *relrgx; /* Relation regexp parsed from = pattern, or > + * NULL */ > + bool tblonly; /* true if relrgx should only match = tables */ > + bool idxonly; /* true if relrgx should only match = indexes */ >=20 > Maybe: db_regex, nsp_regex, rel_regex, table_only, index_only? >=20 > Just because it seems theoretically possible that someone will see > nsprgx and not immediately understand what it's supposed to mean, even > if they know that nsp is a common abbreviation for namespace in > PostgreSQL code, and even if they also know what a regular expression > is. Changed. Along the way, I noticed that "tbl" and "idx" were being used = in C/SQL both to mean ("table_only", "index_only") in some contexts and = ("is_table", "is_index') in others, so I replaced all instances of "tbl" = and "idx" with the unambiguous labels. > Your four messages about there being nothing to check seem like they > could be consolidated down to one: "nothing to check for pattern > \"%s\"". I anticipated your review comment, but I'm worried about the case that = somebody runs pg_amcheck -t "foo" -i "foo" and one of those matches and the other does not. The message 'nothing = to check for pattern "foo"' will be wrong (because there was something = to check for it) and unhelpful (because it doesn't say which failed to = match.) > I would favor changing things so that once argument parsing is > complete, we switch to reporting all errors that way. So in other > words here, and everything that follows: >=20 > + fprintf(stderr, "%s: no databases to check\n", = progname); Same concern about the output for pg_amcheck -t "foo" -i "foo" -d "foo" You might think I'm being silly here, as database names, table names, = and index names should in normal usage not be hard for the user to = distinguish. But consider pg_amcheck "mydb.myschema.mytable" If it says, 'nothing to check for pattern "mydb.myschema.mytable"', you = don't know if the database doesn't exist or if the table doesn't exist. >=20 > + * ParallelSlots based event loop follows. >=20 > "Main event loop." Changed. > To me it would read slightly better to change each reference to > "relations list" to "list of relations", but perhaps that is too > nitpicky. No harm picking those nits. Changed. > I think the two instances of goto finish could be avoided with not > much work. At most a few things need to happen only if !failed, and > maybe not even that, if you just said "break;" instead. Good point. Changed. > + * Note: Heap relation corruption is returned by verify_heapam() = without the > + * use of raising errors, but running verify_heapam() on a corrupted = table may >=20 > How about "Heap relation corruption() is reported by verify_heapam() > via the result set, rather than an ERROR, ..." Changed, though I assumed your parens for corruption() were not = intended. Ok, so now you've moved on to reviewing the regression tests.... > It seems mighty inefficient to have a whole bunch of consecutive calls > to remove_relation_file() or corrupt_first_page() when every such call > stops and restarts the database. I would guess these tests will run > noticeably faster if you don't do that. Either the functions need to > take a list of arguments, or the stop/start needs to be pulled up and > done in the caller. Changed. > corrupt_first_page() could use a comment explaining what exactly we're > overwriting, and in particular noting that we don't want to just > clobber the LSN, but rather something where we can detect a wrong > value. Added comments that we're skipping past the PageHeader and overwriting = garbage starting in the line pointers. > There's a long list of calls to command_checks_all() in 003_check.pl > that don't actually check anything but that the command failed, but > run it with a bunch of different options. I don't understand the value > of that, and suggest reducing the number of cases tested. If you want, > you can have tests elsewhere that focus -- perhaps by using verbose > mode -- on checking that the right tables are being checked. This should be better in this next patch series. >=20 > This is not yet a full review of everything in this patch -- I haven't > sorted through all of the tests yet, or all of the new query > construction logic -- but to me this looks pretty close to > committable. Thanks for the review! --Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A Content-Disposition: attachment; filename=v42-0001-Reworking-ParallelSlots-for-mutliple-DB-use.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v42-0001-Reworking-ParallelSlots-for-mutliple-DB-use.patch" Content-Transfer-Encoding: quoted-printable =46rom=2079c357606f06c6422ed2039b6395a59050a4cff1=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Wed,=203=20Mar=202021=2007:16:55=20-0800=0ASubject:=20[PATCH=20= v42=201/3]=20Reworking=20ParallelSlots=20for=20mutliple=20DB=20use=0A=0A= The=20existing=20implementation=20of=20ParallelSlots=20is=20used=20by=20= reindexdb=0Aand=20vacuumdb=20to=20process=20tables=20in=20parallel=20in=20= only=20one=20database=20at=0Aa=20time.=20=20The=20ParallelSlots=20= interface=20reflects=20this=20usage=20pattern.=0AThe=20function=20to=20= set=20up=20the=20slots=20assumes=20all=20slots=20should=20be=0Aconnected=20= to=20the=20same=20database,=20and=20the=20function=20for=20getting=20the=0A= next=20idle=20slot=20pays=20no=20attention=20to=20which=20database=20the=20= slot=20may=20be=0Aconnected=20to.=0A=0AIn=20anticipation=20of=20= pg_amcheck=20using=20parallel=20slots=20to=20process=0Amultiple=20= databases=20in=20parallel,=20reworking=20the=20interface=20while=0A= trying=20to=20remain=20reasonably=20simple=20for=20reindexdb=20and=20= vacuumdb=20to=0Ause:=0A=0AParallelSlotsSetup()=20no=20longer=20creates=20= or=20receives=20database=0Aconnections.=20=20It=20takes=20arguments=20= that=20it=20stores=20for=20use=20in=0Asubsequent=20operations=20when=20a=20= connection=20needs=20to=20be=20formed.=0A=0ACallers=20who=20already=20= have=20a=20connection=20and=20want=20to=20reuse=20it=20can=20give=0Ait=20= to=20the=20parallel=20slots=20using=20a=20new=20function,=0A= ParallelSlotsAdoptConn().=20=20Both=20reindexdb=20and=20vacuumdb=20use=20= this.=0A=0AParallelSlotsGetIdle()=20is=20extended=20to=20take=20a=20= dbname=20argument=0Aindicating=20the=20database=20to=20which=20a=20= connection=20is=20desired,=20and=20to=0Amanage=20a=20heterogeneous=20set=20= of=20slots=20potentially=20connected=20to=20varying=0Adatabases=20and=20= some=20perhaps=20not=20yet=20connected.=20=20The=20function=20will=0A= reuse=20an=20existing=20connection=20or=20form=20a=20new=20connection=20= as=20necessary.=0A=0AThe=20logic=20for=20determining=20whether=20a=20= slot's=20connection=20is=20suitable=0Afor=20reuse=20is=20based=20on=20= the=20database=20the=20slot's=20connection=20is=0Aconnected=20to,=20and=20= whether=20that=20matches=20the=20database=20desired.=20=20Other=0A= connection=20parameters=20(user,=20host,=20port,=20etc.)=20are=20assumed=20= not=20to=0Achange=20from=20slot=20to=20slot.=0A---=0A=20= src/bin/scripts/reindexdb.c=20=20=20=20=20=20=20=20=20=20|=20=2017=20+-=0A= =20src/bin/scripts/vacuumdb.c=20=20=20=20=20=20=20=20=20=20=20|=20=2046=20= +--=0A=20src/fe_utils/parallel_slot.c=20=20=20=20=20=20=20=20=20|=20411=20= +++++++++++++++++++--------=0A=20src/include/fe_utils/parallel_slot.h=20= |=20=2027=20+-=0A=20src/tools/pgindent/typedefs.list=20=20=20=20=20|=20=20= =202=20+=0A=205=20files=20changed,=20342=20insertions(+),=20161=20= deletions(-)=0A=0Adiff=20--git=20a/src/bin/scripts/reindexdb.c=20= b/src/bin/scripts/reindexdb.c=0Aindex=20cf28176243..fc0681538a=20100644=0A= ---=20a/src/bin/scripts/reindexdb.c=0A+++=20= b/src/bin/scripts/reindexdb.c=0A@@=20-36,7=20+36,7=20@@=20static=20= SimpleStringList=20*get_parallel_object_list(PGconn=20*conn,=0A=20=09=09=09= =09=09=09=09=09=09=09=09=09=20=20ReindexType=20type,=0A=20=09=09=09=09=09= =09=09=09=09=09=09=09=20=20SimpleStringList=20*user_list,=0A=20=09=09=09=09= =09=09=09=09=09=09=09=09=20=20bool=20echo);=0A-static=20void=20= reindex_one_database(const=20ConnParams=20*cparams,=20ReindexType=20= type,=0A+static=20void=20reindex_one_database(ConnParams=20*cparams,=20= ReindexType=20type,=0A=20=09=09=09=09=09=09=09=09=20SimpleStringList=20= *user_list,=0A=20=09=09=09=09=09=09=09=09=20const=20char=20*progname,=0A=20= =09=09=09=09=09=09=09=09=20bool=20echo,=20bool=20verbose,=20bool=20= concurrently,=0A@@=20-330,7=20+330,7=20@@=20main(int=20argc,=20char=20= *argv[])=0A=20}=0A=20=0A=20static=20void=0A-reindex_one_database(const=20= ConnParams=20*cparams,=20ReindexType=20type,=0A= +reindex_one_database(ConnParams=20*cparams,=20ReindexType=20type,=0A=20=09= =09=09=09=09=20SimpleStringList=20*user_list,=0A=20=09=09=09=09=09=20= const=20char=20*progname,=20bool=20echo,=0A=20=09=09=09=09=09=20bool=20= verbose,=20bool=20concurrently,=20int=20concurrentCons,=0A@@=20-341,7=20= +341,7=20@@=20reindex_one_database(const=20ConnParams=20*cparams,=20= ReindexType=20type,=0A=20=09bool=09=09parallel=20=3D=20concurrentCons=20= >=201;=0A=20=09SimpleStringList=20*process_list=20=3D=20user_list;=0A=20=09= ReindexType=20process_type=20=3D=20type;=0A-=09ParallelSlot=20*slots;=0A= +=09ParallelSlotArray=20*sa;=0A=20=09bool=09=09failed=20=3D=20false;=0A=20= =09int=09=09=09items_count=20=3D=200;=0A=20=0A@@=20-461,7=20+461,8=20@@=20= reindex_one_database(const=20ConnParams=20*cparams,=20ReindexType=20= type,=0A=20=0A=20=09Assert(process_list=20!=3D=20NULL);=0A=20=0A-=09= slots=20=3D=20ParallelSlotsSetup(cparams,=20progname,=20echo,=20conn,=20= concurrentCons);=0A+=09sa=20=3D=20ParallelSlotsSetup(concurrentCons,=20= cparams,=20progname,=20echo,=20NULL);=0A+=09ParallelSlotsAdoptConn(sa,=20= conn);=0A=20=0A=20=09cell=20=3D=20process_list->head;=0A=20=09do=0A@@=20= -475,7=20+476,7=20@@=20reindex_one_database(const=20ConnParams=20= *cparams,=20ReindexType=20type,=0A=20=09=09=09goto=20finish;=0A=20=09=09= }=0A=20=0A-=09=09free_slot=20=3D=20ParallelSlotsGetIdle(slots,=20= concurrentCons);=0A+=09=09free_slot=20=3D=20ParallelSlotsGetIdle(sa,=20= NULL);=0A=20=09=09if=20(!free_slot)=0A=20=09=09{=0A=20=09=09=09failed=20= =3D=20true;=0A@@=20-489,7=20+490,7=20@@=20reindex_one_database(const=20= ConnParams=20*cparams,=20ReindexType=20type,=0A=20=09=09cell=20=3D=20= cell->next;=0A=20=09}=20while=20(cell=20!=3D=20NULL);=0A=20=0A-=09if=20= (!ParallelSlotsWaitCompletion(slots,=20concurrentCons))=0A+=09if=20= (!ParallelSlotsWaitCompletion(sa))=0A=20=09=09failed=20=3D=20true;=0A=20=0A= =20finish:=0A@@=20-499,8=20+500,8=20@@=20finish:=0A=20=09=09= pg_free(process_list);=0A=20=09}=0A=20=0A-=09= ParallelSlotsTerminate(slots,=20concurrentCons);=0A-=09pfree(slots);=0A+=09= ParallelSlotsTerminate(sa);=0A+=09pfree(sa);=0A=20=0A=20=09if=20(failed)=0A= =20=09=09exit(1);=0Adiff=20--git=20a/src/bin/scripts/vacuumdb.c=20= b/src/bin/scripts/vacuumdb.c=0Aindex=20602fd45c42..7901c41f16=20100644=0A= ---=20a/src/bin/scripts/vacuumdb.c=0A+++=20b/src/bin/scripts/vacuumdb.c=0A= @@=20-45,7=20+45,7=20@@=20typedef=20struct=20vacuumingOptions=0A=20}=20= vacuumingOptions;=0A=20=0A=20=0A-static=20void=20= vacuum_one_database(const=20ConnParams=20*cparams,=0A+static=20void=20= vacuum_one_database(ConnParams=20*cparams,=0A=20=09=09=09=09=09=09=09=09= vacuumingOptions=20*vacopts,=0A=20=09=09=09=09=09=09=09=09int=20stage,=0A= =20=09=09=09=09=09=09=09=09SimpleStringList=20*tables,=0A@@=20-408,7=20= +408,7=20@@=20main(int=20argc,=20char=20*argv[])=0A=20=20*=20a=20list=20= of=20tables=20from=20the=20database.=0A=20=20*/=0A=20static=20void=0A= -vacuum_one_database(const=20ConnParams=20*cparams,=0A= +vacuum_one_database(ConnParams=20*cparams,=0A=20=09=09=09=09=09= vacuumingOptions=20*vacopts,=0A=20=09=09=09=09=09int=20stage,=0A=20=09=09= =09=09=09SimpleStringList=20*tables,=0A@@=20-421,13=20+421,14=20@@=20= vacuum_one_database(const=20ConnParams=20*cparams,=0A=20=09PGresult=20=20= =20*res;=0A=20=09PGconn=09=20=20=20*conn;=0A=20=09SimpleStringListCell=20= *cell;=0A-=09ParallelSlot=20*slots;=0A+=09ParallelSlotArray=20*sa;=0A=20=09= SimpleStringList=20dbtables=20=3D=20{NULL,=20NULL};=0A=20=09int=09=09=09= i;=0A=20=09int=09=09=09ntups;=0A=20=09bool=09=09failed=20=3D=20false;=0A=20= =09bool=09=09tables_listed=20=3D=20false;=0A=20=09bool=09=09has_where=20= =3D=20false;=0A+=09const=20char=20*initcmd;=0A=20=09const=20char=20= *stage_commands[]=20=3D=20{=0A=20=09=09"SET=20= default_statistics_target=3D1;=20SET=20vacuum_cost_delay=3D0;",=0A=20=09=09= "SET=20default_statistics_target=3D10;=20RESET=20vacuum_cost_delay;",=0A= @@=20-684,26=20+685,25=20@@=20vacuum_one_database(const=20ConnParams=20= *cparams,=0A=20=09=09concurrentCons=20=3D=201;=0A=20=0A=20=09/*=0A-=09=20= *=20Setup=20the=20database=20connections.=20We=20reuse=20the=20= connection=20we=20already=20have=0A-=09=20*=20for=20the=20first=20slot.=20= =20If=20not=20in=20parallel=20mode,=20the=20first=20slot=20in=20the=0A-=09= =20*=20array=20contains=20the=20connection.=0A+=09=20*=20All=20slots=20= need=20to=20be=20prepared=20to=20run=20the=20appropriate=20analyze=20= stage,=20if=0A+=09=20*=20caller=20requested=20that=20mode.=20=20We=20= have=20to=20prepare=20the=20initial=20connection=0A+=09=20*=20ourselves=20= before=20setting=20up=20the=20slots.=0A=20=09=20*/=0A-=09slots=20=3D=20= ParallelSlotsSetup(cparams,=20progname,=20echo,=20conn,=20= concurrentCons);=0A+=09if=20(stage=20=3D=3D=20ANALYZE_NO_STAGE)=0A+=09=09= initcmd=20=3D=20NULL;=0A+=09else=0A+=09{=0A+=09=09initcmd=20=3D=20= stage_commands[stage];=0A+=09=09executeCommand(conn,=20initcmd,=20echo);=0A= +=09}=0A=20=0A=20=09/*=0A-=09=20*=20Prepare=20all=20the=20connections=20= to=20run=20the=20appropriate=20analyze=20stage,=20if=0A-=09=20*=20caller=20= requested=20that=20mode.=0A+=09=20*=20Setup=20the=20database=20= connections.=20We=20reuse=20the=20connection=20we=20already=20have=0A+=09= =20*=20for=20the=20first=20slot.=20=20If=20not=20in=20parallel=20mode,=20= the=20first=20slot=20in=20the=0A+=09=20*=20array=20contains=20the=20= connection.=0A=20=09=20*/=0A-=09if=20(stage=20!=3D=20ANALYZE_NO_STAGE)=0A= -=09{=0A-=09=09int=09=09=09j;=0A-=0A-=09=09/*=20We=20already=20emitted=20= the=20message=20above=20*/=0A-=0A-=09=09for=20(j=20=3D=200;=20j=20<=20= concurrentCons;=20j++)=0A-=09=09=09executeCommand((slots=20+=20= j)->connection,=0A-=09=09=09=09=09=09=20=20=20stage_commands[stage],=20= echo);=0A-=09}=0A+=09sa=20=3D=20ParallelSlotsSetup(concurrentCons,=20= cparams,=20progname,=20echo,=20initcmd);=0A+=09= ParallelSlotsAdoptConn(sa,=20conn);=0A=20=0A=20=09initPQExpBuffer(&sql);=0A= =20=0A@@=20-719,7=20+719,7=20@@=20vacuum_one_database(const=20ConnParams=20= *cparams,=0A=20=09=09=09goto=20finish;=0A=20=09=09}=0A=20=0A-=09=09= free_slot=20=3D=20ParallelSlotsGetIdle(slots,=20concurrentCons);=0A+=09=09= free_slot=20=3D=20ParallelSlotsGetIdle(sa,=20NULL);=0A=20=09=09if=20= (!free_slot)=0A=20=09=09{=0A=20=09=09=09failed=20=3D=20true;=0A@@=20= -740,12=20+740,12=20@@=20vacuum_one_database(const=20ConnParams=20= *cparams,=0A=20=09=09cell=20=3D=20cell->next;=0A=20=09}=20while=20(cell=20= !=3D=20NULL);=0A=20=0A-=09if=20(!ParallelSlotsWaitCompletion(slots,=20= concurrentCons))=0A+=09if=20(!ParallelSlotsWaitCompletion(sa))=0A=20=09=09= failed=20=3D=20true;=0A=20=0A=20finish:=0A-=09= ParallelSlotsTerminate(slots,=20concurrentCons);=0A-=09pg_free(slots);=0A= +=09ParallelSlotsTerminate(sa);=0A+=09pg_free(sa);=0A=20=0A=20=09= termPQExpBuffer(&sql);=0A=20=0Adiff=20--git=20= a/src/fe_utils/parallel_slot.c=20b/src/fe_utils/parallel_slot.c=0Aindex=20= b625deb254..a09e5460e5=20100644=0A---=20a/src/fe_utils/parallel_slot.c=0A= +++=20b/src/fe_utils/parallel_slot.c=0A@@=20-25,31=20+25,23=20@@=0A=20= #include=20"common/logging.h"=0A=20#include=20"fe_utils/cancel.h"=0A=20= #include=20"fe_utils/parallel_slot.h"=0A+#include=20= "fe_utils/query_utils.h"=0A=20=0A=20#define=20ERRCODE_UNDEFINED_TABLE=20=20= "42P01"=0A=20=0A-static=20void=20init_slot(ParallelSlot=20*slot,=20= PGconn=20*conn);=0A=20static=20int=09select_loop(int=20maxFd,=20fd_set=20= *workerset);=0A=20static=20bool=20processQueryResult(ParallelSlot=20= *slot,=20PGresult=20*result);=0A=20=0A-static=20void=0A= -init_slot(ParallelSlot=20*slot,=20PGconn=20*conn)=0A-{=0A-=09= slot->connection=20=3D=20conn;=0A-=09/*=20Initially=20assume=20= connection=20is=20idle=20*/=0A-=09slot->isFree=20=3D=20true;=0A-=09= ParallelSlotClearHandler(slot);=0A-}=0A-=0A=20/*=0A=20=20*=20Process=20= (and=20delete)=20a=20query=20result.=20=20Returns=20true=20if=20there's=20= no=20problem,=0A-=20*=20false=20otherwise.=20It's=20up=20to=20the=20= handler=20to=20decide=20what=20cosntitutes=20a=0A+=20*=20false=20= otherwise.=20It's=20up=20to=20the=20handler=20to=20decide=20what=20= constitutes=20a=0A=20=20*=20problem.=0A=20=20*/=0A=20static=20bool=0A=20= processQueryResult(ParallelSlot=20*slot,=20PGresult=20*result)=0A=20{=0A=20= =09Assert(slot->handler=20!=3D=20NULL);=0A+=09Assert(slot->connection=20= !=3D=20NULL);=0A=20=0A=20=09/*=20On=20failure,=20the=20handler=20should=20= return=20NULL=20after=20freeing=20the=20result=20*/=0A=20=09if=20= (!slot->handler(result,=20slot->connection,=20slot->handler_context))=0A= @@=20-71,6=20+63,9=20@@=20consumeQueryResult(ParallelSlot=20*slot)=0A=20=09= bool=09=09ok=20=3D=20true;=0A=20=09PGresult=20=20=20*result;=0A=20=0A+=09= Assert(slot=20!=3D=20NULL);=0A+=09Assert(slot->connection=20!=3D=20= NULL);=0A+=0A=20=09SetCancelConn(slot->connection);=0A=20=09while=20= ((result=20=3D=20PQgetResult(slot->connection))=20!=3D=20NULL)=0A=20=09{=0A= @@=20-137,151=20+132,316=20@@=20select_loop(int=20maxFd,=20fd_set=20= *workerset)=0A=20}=0A=20=0A=20/*=0A-=20*=20ParallelSlotsGetIdle=0A-=20*=09= =09Return=20a=20connection=20slot=20that=20is=20ready=20to=20execute=20a=20= command.=0A-=20*=0A-=20*=20This=20returns=20the=20first=20slot=20we=20= find=20that=20is=20marked=20isFree,=20if=20one=20is;=0A-=20*=20= otherwise,=20we=20loop=20on=20select()=20until=20one=20socket=20becomes=20= available.=20=20When=0A-=20*=20this=20happens,=20we=20read=20the=20whole=20= set=20and=20mark=20as=20free=20all=20sockets=20that=0A-=20*=20become=20= available.=20=20If=20an=20error=20occurs,=20NULL=20is=20returned.=0A+=20= *=20Return=20the=20offset=20of=20a=20suitable=20idle=20slot,=20or=20-1=20= if=20none=20are=20available.=20=20If=0A+=20*=20the=20given=20dbname=20is=20= not=20null,=20only=20idle=20slots=20connected=20to=20the=20given=0A+=20*=20= database=20are=20considered=20suitable,=20otherwise=20all=20idle=20= connected=20slots=20are=0A+=20*=20considered=20suitable.=0A=20=20*/=0A= -ParallelSlot=20*=0A-ParallelSlotsGetIdle(ParallelSlot=20*slots,=20int=20= numslots)=0A+static=20int=0A+find_matching_idle_slot(const=20= ParallelSlotArray=20*sa,=20const=20char=20*dbname)=0A=20{=0A=20=09int=09=09= =09i;=0A-=09int=09=09=09firstFree=20=3D=20-1;=0A=20=0A-=09/*=0A-=09=20*=20= Look=20for=20any=20connection=20currently=20free.=20=20If=20there=20is=20= one,=20mark=20it=20as=0A-=09=20*=20taken=20and=20let=20the=20caller=20= know=20the=20slot=20to=20use.=0A-=09=20*/=0A-=09for=20(i=20=3D=200;=20i=20= <=20numslots;=20i++)=0A+=09for=20(i=20=3D=200;=20i=20<=20sa->numslots;=20= i++)=0A=20=09{=0A-=09=09if=20(slots[i].isFree)=0A-=09=09{=0A-=09=09=09= slots[i].isFree=20=3D=20false;=0A-=09=09=09return=20slots=20+=20i;=0A-=09= =09}=0A+=09=09if=20(sa->slots[i].inUse)=0A+=09=09=09continue;=0A+=0A+=09=09= if=20(sa->slots[i].connection=20=3D=3D=20NULL)=0A+=09=09=09continue;=0A+=0A= +=09=09if=20(dbname=20=3D=3D=20NULL=20||=0A+=09=09=09= strcmp(PQdb(sa->slots[i].connection),=20dbname)=20=3D=3D=200)=0A+=09=09=09= return=20i;=0A+=09}=0A+=09return=20-1;=0A+}=0A+=0A+/*=0A+=20*=20Return=20= the=20offset=20of=20the=20first=20slot=20without=20a=20database=20= connection,=20or=20-1=20if=0A+=20*=20all=20slots=20are=20connected.=0A+=20= */=0A+static=20int=0A+find_unconnected_slot(const=20ParallelSlotArray=20= *sa)=0A+{=0A+=09int=09=09=09i;=0A+=0A+=09for=20(i=20=3D=200;=20i=20<=20= sa->numslots;=20i++)=0A+=09{=0A+=09=09if=20(sa->slots[i].inUse)=0A+=09=09= =09continue;=0A+=0A+=09=09if=20(sa->slots[i].connection=20=3D=3D=20NULL)=0A= +=09=09=09return=20i;=0A+=09}=0A+=0A+=09return=20-1;=0A+}=0A+=0A+/*=0A+=20= *=20Return=20the=20offset=20of=20the=20first=20idle=20slot,=20or=20-1=20= if=20all=20slots=20are=20busy.=0A+=20*/=0A+static=20int=0A= +find_any_idle_slot(const=20ParallelSlotArray=20*sa)=0A+{=0A+=09int=09=09= =09i;=0A+=0A+=09for=20(i=20=3D=200;=20i=20<=20sa->numslots;=20i++)=0A+=09= =09if=20(!sa->slots[i].inUse)=0A+=09=09=09return=20i;=0A+=0A+=09return=20= -1;=0A+}=0A+=0A+/*=0A+=20*=20Wait=20for=20any=20slot's=20connection=20to=20= have=20query=20results,=20consume=20the=20results,=0A+=20*=20and=20= update=20the=20slot's=20status=20as=20appropriate.=20=20Returns=20true=20= on=20success,=0A+=20*=20false=20on=20cancellation,=20on=20error,=20or=20= if=20no=20slots=20are=20connected.=0A+=20*/=0A+static=20bool=0A= +wait_on_slots(ParallelSlotArray=20*sa)=0A+{=0A+=09int=09=09=09i;=0A+=09= fd_set=09=09slotset;=0A+=09int=09=09=09maxFd=20=3D=200;=0A+=09PGconn=09=20= =20=20*cancelconn=20=3D=20NULL;=0A+=0A+=09/*=20We=20must=20reconstruct=20= the=20fd_set=20for=20each=20call=20to=20select_loop=20*/=0A+=09= FD_ZERO(&slotset);=0A+=0A+=09for=20(i=20=3D=200;=20i=20<=20sa->numslots;=20= i++)=0A+=09{=0A+=09=09int=09=09=09sock;=0A+=0A+=09=09/*=20We=20shouldn't=20= get=20here=20if=20we=20still=20have=20slots=20without=20connections=20*/=0A= +=09=09Assert(sa->slots[i].connection=20!=3D=20NULL);=0A+=0A+=09=09sock=20= =3D=20PQsocket(sa->slots[i].connection);=0A+=0A+=09=09/*=0A+=09=09=20*=20= We=20don't=20really=20expect=20any=20connections=20to=20lose=20their=20= sockets=20after=0A+=09=09=20*=20startup,=20but=20just=20in=20case,=20= cope=20by=20ignoring=20them.=0A+=09=09=20*/=0A+=09=09if=20(sock=20<=200)=0A= +=09=09=09continue;=0A+=0A+=09=09/*=20Keep=20track=20of=20the=20first=20= valid=20connection=20we=20see.=20*/=0A+=09=09if=20(cancelconn=20=3D=3D=20= NULL)=0A+=09=09=09cancelconn=20=3D=20sa->slots[i].connection;=0A+=0A+=09=09= FD_SET(sock,=20&slotset);=0A+=09=09if=20(sock=20>=20maxFd)=0A+=09=09=09= maxFd=20=3D=20sock;=0A=20=09}=0A=20=0A=20=09/*=0A-=09=20*=20No=20free=20= slot=20found,=20so=20wait=20until=20one=20of=20the=20connections=20has=20= finished=0A-=09=20*=20its=20task=20and=20return=20the=20available=20= slot.=0A+=09=20*=20If=20we=20get=20this=20far=20with=20no=20valid=20= connections,=20processing=20cannot=0A+=09=20*=20continue.=0A=20=09=20*/=0A= -=09while=20(firstFree=20<=200)=0A+=09if=20(cancelconn=20=3D=3D=20NULL)=0A= +=09=09return=20false;=0A+=0A+=09SetCancelConn(sa->slots->connection);=0A= +=09i=20=3D=20select_loop(maxFd,=20&slotset);=0A+=09ResetCancelConn();=0A= +=0A+=09/*=20failure?=20*/=0A+=09if=20(i=20<=200)=0A+=09=09return=20= false;=0A+=0A+=09for=20(i=20=3D=200;=20i=20<=20sa->numslots;=20i++)=0A=20= =09{=0A-=09=09fd_set=09=09slotset;=0A-=09=09int=09=09=09maxFd=20=3D=200;=0A= +=09=09int=09=09=09sock;=0A=20=0A-=09=09/*=20We=20must=20reconstruct=20= the=20fd_set=20for=20each=20call=20to=20select_loop=20*/=0A-=09=09= FD_ZERO(&slotset);=0A+=09=09sock=20=3D=20= PQsocket(sa->slots[i].connection);=0A=20=0A-=09=09for=20(i=20=3D=200;=20= i=20<=20numslots;=20i++)=0A+=09=09if=20(sock=20>=3D=200=20&&=20= FD_ISSET(sock,=20&slotset))=0A=20=09=09{=0A-=09=09=09int=09=09=09sock=20= =3D=20PQsocket(slots[i].connection);=0A-=0A-=09=09=09/*=0A-=09=09=09=20*=20= We=20don't=20really=20expect=20any=20connections=20to=20lose=20their=20= sockets=0A-=09=09=09=20*=20after=20startup,=20but=20just=20in=20case,=20= cope=20by=20ignoring=20them.=0A-=09=09=09=20*/=0A-=09=09=09if=20(sock=20= <=200)=0A-=09=09=09=09continue;=0A-=0A-=09=09=09FD_SET(sock,=20= &slotset);=0A-=09=09=09if=20(sock=20>=20maxFd)=0A-=09=09=09=09maxFd=20=3D=20= sock;=0A+=09=09=09/*=20select()=20says=20input=20is=20available,=20so=20= consume=20it=20*/=0A+=09=09=09PQconsumeInput(sa->slots[i].connection);=0A= =20=09=09}=0A=20=0A-=09=09SetCancelConn(slots->connection);=0A-=09=09i=20= =3D=20select_loop(maxFd,=20&slotset);=0A-=09=09ResetCancelConn();=0A-=0A= -=09=09/*=20failure?=20*/=0A-=09=09if=20(i=20<=200)=0A-=09=09=09return=20= NULL;=0A-=0A-=09=09for=20(i=20=3D=200;=20i=20<=20numslots;=20i++)=0A+=09=09= /*=20Collect=20result(s)=20as=20long=20as=20any=20are=20available=20*/=0A= +=09=09while=20(!PQisBusy(sa->slots[i].connection))=0A=20=09=09{=0A-=09=09= =09int=09=09=09sock=20=3D=20PQsocket(slots[i].connection);=0A+=09=09=09= PGresult=20=20=20*result=20=3D=20PQgetResult(sa->slots[i].connection);=0A= =20=0A-=09=09=09if=20(sock=20>=3D=200=20&&=20FD_ISSET(sock,=20&slotset))=0A= +=09=09=09if=20(result=20!=3D=20NULL)=0A=20=09=09=09{=0A-=09=09=09=09/*=20= select()=20says=20input=20is=20available,=20so=20consume=20it=20*/=0A-=09= =09=09=09PQconsumeInput(slots[i].connection);=0A+=09=09=09=09/*=20Handle=20= and=20discard=20the=20command=20result=20*/=0A+=09=09=09=09if=20= (!processQueryResult(&sa->slots[i],=20result))=0A+=09=09=09=09=09return=20= false;=0A=20=09=09=09}=0A-=0A-=09=09=09/*=20Collect=20result(s)=20as=20= long=20as=20any=20are=20available=20*/=0A-=09=09=09while=20= (!PQisBusy(slots[i].connection))=0A+=09=09=09else=0A=20=09=09=09{=0A-=09=09= =09=09PGresult=20=20=20*result=20=3D=20PQgetResult(slots[i].connection);=0A= -=0A-=09=09=09=09if=20(result=20!=3D=20NULL)=0A-=09=09=09=09{=0A-=09=09=09= =09=09/*=20Handle=20and=20discard=20the=20command=20result=20*/=0A-=09=09= =09=09=09if=20(!processQueryResult(slots=20+=20i,=20result))=0A-=09=09=09= =09=09=09return=20NULL;=0A-=09=09=09=09}=0A-=09=09=09=09else=0A-=09=09=09= =09{=0A-=09=09=09=09=09/*=20This=20connection=20has=20become=20idle=20*/=0A= -=09=09=09=09=09slots[i].isFree=20=3D=20true;=0A-=09=09=09=09=09= ParallelSlotClearHandler(slots=20+=20i);=0A-=09=09=09=09=09if=20= (firstFree=20<=200)=0A-=09=09=09=09=09=09firstFree=20=3D=20i;=0A-=09=09=09= =09=09break;=0A-=09=09=09=09}=0A+=09=09=09=09/*=20This=20connection=20= has=20become=20idle=20*/=0A+=09=09=09=09sa->slots[i].inUse=20=3D=20= false;=0A+=09=09=09=09ParallelSlotClearHandler(&sa->slots[i]);=0A+=09=09=09= =09break;=0A=20=09=09=09}=0A=20=09=09}=0A=20=09}=0A+=09return=20true;=0A= +}=0A=20=0A-=09slots[firstFree].isFree=20=3D=20false;=0A-=09return=20= slots=20+=20firstFree;=0A+/*=0A+=20*=20Open=20a=20new=20database=20= connection=20using=20the=20stored=20connection=20parameters=20and=0A+=20= *=20optionally=20a=20given=20dbname=20if=20not=20null,=20execute=20the=20= stored=20initial=20command=20if=0A+=20*=20any,=20and=20associate=20the=20= new=20connection=20with=20the=20given=20slot.=0A+=20*/=0A+static=20void=0A= +connect_slot(ParallelSlotArray=20*sa,=20int=20slotno,=20const=20char=20= *dbname)=0A+{=0A+=09const=20char=20*old_override;=0A+=09ParallelSlot=20= *slot=20=3D=20&sa->slots[slotno];=0A+=0A+=09old_override=20=3D=20= sa->cparams->override_dbname;=0A+=09if=20(dbname)=0A+=09=09= sa->cparams->override_dbname=20=3D=20dbname;=0A+=09slot->connection=20=3D=20= connectDatabase(sa->cparams,=20sa->progname,=20sa->echo,=20false,=20= true);=0A+=09sa->cparams->override_dbname=20=3D=20old_override;=0A+=0A+=09= if=20(PQsocket(slot->connection)=20>=3D=20FD_SETSIZE)=0A+=09{=0A+=09=09= pg_log_fatal("too=20many=20jobs=20for=20this=20platform");=0A+=09=09= exit(1);=0A+=09}=0A+=0A+=09/*=20Setup=20the=20connection=20using=20the=20= supplied=20command,=20if=20any.=20*/=0A+=09if=20(sa->initcmd)=0A+=09=09= executeCommand(slot->connection,=20sa->initcmd,=20sa->echo);=0A=20}=0A=20= =0A=20/*=0A-=20*=20ParallelSlotsSetup=0A-=20*=09=09Prepare=20a=20set=20= of=20parallel=20slots=20to=20use=20on=20a=20given=20database.=0A+=20*=20= ParallelSlotsGetIdle=0A+=20*=09=09Return=20a=20connection=20slot=20that=20= is=20ready=20to=20execute=20a=20command.=0A+=20*=0A+=20*=20The=20slot=20= returned=20is=20chosen=20as=20follows:=0A+=20*=0A+=20*=20If=20any=20idle=20= slot=20already=20has=20an=20open=20connection,=20and=20if=20either=20= dbname=20is=0A+=20*=20null=20or=20the=20existing=20connection=20is=20to=20= the=20given=20database,=20that=20slot=20will=20be=0A+=20*=20returned=20= allowing=20the=20connection=20to=20be=20reused.=0A+=20*=0A+=20*=20= Otherwise,=20if=20any=20idle=20slot=20is=20not=20yet=20connected=20to=20= any=20database,=20the=20slot=0A+=20*=20will=20be=20returned=20with=20= it's=20connection=20opened=20using=20the=20stored=20cparams=20and=0A+=20= *=20optionally=20the=20given=20dbname=20if=20not=20null.=0A+=20*=0A+=20*=20= Otherwise,=20if=20any=20idle=20slot=20exists,=20an=20idle=20slot=20will=20= be=20chosen=20and=20returned=0A+=20*=20after=20having=20it's=20= connection=20disconnected=20and=20reconnected=20using=20the=20stored=0A+=20= *=20cparams=20and=20optionally=20the=20given=20dbname=20if=20not=20null.=0A= =20=20*=0A-=20*=20This=20creates=20and=20initializes=20a=20set=20of=20= connections=20to=20the=20database=0A-=20*=20using=20the=20information=20= given=20by=20the=20caller,=20marking=20all=20parallel=20slots=0A-=20*=20= as=20free=20and=20ready=20to=20use.=20=20"conn"=20is=20an=20initial=20= connection=20set=20up=0A-=20*=20by=20the=20caller=20and=20is=20= associated=20with=20the=20first=20slot=20in=20the=20parallel=0A-=20*=20= set.=0A+=20*=20Otherwise,=20if=20any=20slots=20have=20connections=20that=20= are=20busy,=20we=20loop=20on=20select()=0A+=20*=20until=20one=20socket=20= becomes=20available.=20=20When=20this=20happens,=20we=20read=20the=20= whole=0A+=20*=20set=20and=20mark=20as=20free=20all=20sockets=20that=20= become=20available.=20=20We=20then=20select=20a=0A+=20*=20slot=20using=20= the=20same=20rules=20as=20above.=0A+=20*=0A+=20*=20Otherwise,=20we=20= cannot=20return=20a=20slot,=20which=20is=20an=20error,=20and=20NULL=20is=20= returned.=0A+=20*=0A+=20*=20For=20any=20connection=20created,=20if=20the=20= stored=20initcmd=20is=20not=20null,=20it=20will=20be=0A+=20*=20executed=20= as=20a=20command=20on=20the=20newly=20formed=20connection=20before=20the=20= slot=20is=0A+=20*=20returned.=0A+=20*=0A+=20*=20If=20an=20error=20= occurs,=20NULL=20is=20returned.=0A=20=20*/=0A=20ParallelSlot=20*=0A= -ParallelSlotsSetup(const=20ConnParams=20*cparams,=0A-=09=09=09=09=20=20=20= const=20char=20*progname,=20bool=20echo,=0A-=09=09=09=09=20=20=20PGconn=20= *conn,=20int=20numslots)=0A+ParallelSlotsGetIdle(ParallelSlotArray=20= *sa,=20const=20char=20*dbname)=0A=20{=0A-=09ParallelSlot=20*slots;=0A-=09= int=09=09=09i;=0A+=09int=09=09=09offset;=0A=20=0A-=09Assert(conn=20!=3D=20= NULL);=0A+=09Assert(sa);=0A+=09Assert(sa->numslots=20>=200);=0A=20=0A-=09= slots=20=3D=20(ParallelSlot=20*)=20pg_malloc(sizeof(ParallelSlot)=20*=20= numslots);=0A-=09init_slot(slots,=20conn);=0A-=09if=20(numslots=20>=201)=0A= +=09while=20(1)=0A=20=09{=0A-=09=09for=20(i=20=3D=201;=20i=20<=20= numslots;=20i++)=0A+=09=09/*=20First=20choice:=20a=20slot=20already=20= connected=20to=20the=20desired=20database.=20*/=0A+=09=09offset=20=3D=20= find_matching_idle_slot(sa,=20dbname);=0A+=09=09if=20(offset=20>=3D=200)=0A= =20=09=09{=0A-=09=09=09conn=20=3D=20connectDatabase(cparams,=20progname,=20= echo,=20false,=20true);=0A-=0A-=09=09=09/*=0A-=09=09=09=20*=20Fail=20and=20= exit=20immediately=20if=20trying=20to=20use=20a=20socket=20in=20an=0A-=09= =09=09=20*=20unsupported=20range.=20=20POSIX=20requires=20open(2)=20to=20= use=20the=20lowest=0A-=09=09=09=20*=20unused=20file=20descriptor=20and=20= the=20hint=20given=20relies=20on=20that.=0A-=09=09=09=20*/=0A-=09=09=09= if=20(PQsocket(conn)=20>=3D=20FD_SETSIZE)=0A-=09=09=09{=0A-=09=09=09=09= pg_log_fatal("too=20many=20jobs=20for=20this=20platform=20--=20try=20= %d",=20i);=0A-=09=09=09=09exit(1);=0A-=09=09=09}=0A+=09=09=09= sa->slots[offset].inUse=20=3D=20true;=0A+=09=09=09return=20= &sa->slots[offset];=0A+=09=09}=0A+=0A+=09=09/*=20Second=20choice:=20a=20= slot=20not=20connected=20to=20any=20database.=20*/=0A+=09=09offset=20=3D=20= find_unconnected_slot(sa);=0A+=09=09if=20(offset=20>=3D=200)=0A+=09=09{=0A= +=09=09=09connect_slot(sa,=20offset,=20dbname);=0A+=09=09=09= sa->slots[offset].inUse=20=3D=20true;=0A+=09=09=09return=20= &sa->slots[offset];=0A+=09=09}=0A=20=0A-=09=09=09init_slot(slots=20+=20= i,=20conn);=0A+=09=09/*=20Third=20choice:=20a=20slot=20connected=20to=20= the=20wrong=20database.=20*/=0A+=09=09offset=20=3D=20= find_any_idle_slot(sa);=0A+=09=09if=20(offset=20>=3D=200)=0A+=09=09{=0A+=09= =09=09disconnectDatabase(sa->slots[offset].connection);=0A+=09=09=09= sa->slots[offset].connection=20=3D=20NULL;=0A+=09=09=09connect_slot(sa,=20= offset,=20dbname);=0A+=09=09=09sa->slots[offset].inUse=20=3D=20true;=0A+=09= =09=09return=20&sa->slots[offset];=0A=20=09=09}=0A+=0A+=09=09/*=0A+=09=09= =20*=20Fourth=20choice:=20block=20until=20one=20or=20more=20slots=20= become=20available.=20If=0A+=09=09=20*=20any=20slot's=20hit=20a=20fatal=20= error,=20we'll=20find=20out=20about=20that=20here=20and=0A+=09=09=20*=20= return=20NULL.=0A+=09=09=20*/=0A+=09=09if=20(!wait_on_slots(sa))=0A+=09=09= =09return=20NULL;=0A=20=09}=0A+}=0A+=0A+/*=0A+=20*=20ParallelSlotsSetup=0A= +=20*=09=09Prepare=20a=20set=20of=20parallel=20slots=20but=20do=20not=20= connect=20to=20any=20database.=0A+=20*=0A+=20*=20This=20creates=20and=20= initializes=20a=20set=20of=20slots,=20marking=20all=20parallel=20slots=20= as=0A+=20*=20free=20and=20ready=20to=20use.=20=20Establishing=20= connections=20is=20delayed=20until=20requesting=0A+=20*=20a=20free=20= slot.=20=20The=20cparams,=20progname,=20echo,=20and=20initcmd=20are=20= stored=20for=20later=0A+=20*=20use=20and=20must=20remain=20valid=20for=20= the=20lifetime=20of=20the=20returned=20array.=0A+=20*/=0A= +ParallelSlotArray=20*=0A+ParallelSlotsSetup(int=20numslots,=20= ConnParams=20*cparams,=20const=20char=20*progname,=0A+=09=09=09=09=20=20=20= bool=20echo,=20const=20char=20*initcmd)=0A+{=0A+=09ParallelSlotArray=20= *sa;=0A=20=0A-=09return=20slots;=0A+=09Assert(numslots=20>=200);=0A+=09= Assert(cparams=20!=3D=20NULL);=0A+=09Assert(progname=20!=3D=20NULL);=0A+=0A= +=09sa=20=3D=20(ParallelSlotArray=20*)=20= palloc0(offsetof(ParallelSlotArray,=20slots)=20+=0A+=09=09=09=09=09=09=09= =09=09=20=20=20numslots=20*=20sizeof(ParallelSlot));=0A+=0A+=09= sa->numslots=20=3D=20numslots;=0A+=09sa->cparams=20=3D=20cparams;=0A+=09= sa->progname=20=3D=20progname;=0A+=09sa->echo=20=3D=20echo;=0A+=09= sa->initcmd=20=3D=20initcmd;=0A+=0A+=09return=20sa;=0A+}=0A+=0A+/*=0A+=20= *=20ParallelSlotsAdoptConn=0A+=20*=09=09Assign=20an=20open=20connection=20= to=20the=20slots=20array=20for=20reuse.=0A+=20*=0A+=20*=20This=20turns=20= over=20ownership=20of=20an=20open=20connection=20to=20a=20slots=20array.=20= =20The=0A+=20*=20caller=20should=20not=20further=20use=20or=20close=20= the=20connection.=20=20All=20the=20connection's=0A+=20*=20parameters=20= (user,=20host,=20port,=20etc.)=20except=20possibly=20dbname=20should=20= match=0A+=20*=20those=20of=20the=20slots=20array's=20cparams,=20as=20= given=20in=20ParallelSlotsSetup.=20=20If=0A+=20*=20these=20parameters=20= differ,=20subsequent=20behavior=20is=20undefined.=0A+=20*/=0A+void=0A= +ParallelSlotsAdoptConn(ParallelSlotArray=20*sa,=20PGconn=20*conn)=0A+{=0A= +=09int=09=09offset;=0A+=0A+=09offset=20=3D=20find_unconnected_slot(sa);=0A= +=09if=20(offset=20>=3D=200)=0A+=09=09sa->slots[offset].connection=20=3D=20= conn;=0A+=09else=0A+=09=09disconnectDatabase(conn);=0A=20}=0A=20=0A=20/*=0A= @@=20-292,13=20+452,13=20@@=20ParallelSlotsSetup(const=20ConnParams=20= *cparams,=0A=20=20*=20terminate=20all=20connections.=0A=20=20*/=0A=20= void=0A-ParallelSlotsTerminate(ParallelSlot=20*slots,=20int=20numslots)=0A= +ParallelSlotsTerminate(ParallelSlotArray=20*sa)=0A=20{=0A=20=09int=09=09= =09i;=0A=20=0A-=09for=20(i=20=3D=200;=20i=20<=20numslots;=20i++)=0A+=09= for=20(i=20=3D=200;=20i=20<=20sa->numslots;=20i++)=0A=20=09{=0A-=09=09= PGconn=09=20=20=20*conn=20=3D=20slots[i].connection;=0A+=09=09PGconn=09=20= =20=20*conn=20=3D=20sa->slots[i].connection;=0A=20=0A=20=09=09if=20(conn=20= =3D=3D=20NULL)=0A=20=09=09=09continue;=0A@@=20-314,13=20+474,15=20@@=20= ParallelSlotsTerminate(ParallelSlot=20*slots,=20int=20numslots)=0A=20=20= *=20error=20has=20been=20found=20on=20the=20way.=0A=20=20*/=0A=20bool=0A= -ParallelSlotsWaitCompletion(ParallelSlot=20*slots,=20int=20numslots)=0A= +ParallelSlotsWaitCompletion(ParallelSlotArray=20*sa)=0A=20{=0A=20=09int=09= =09=09i;=0A=20=0A-=09for=20(i=20=3D=200;=20i=20<=20numslots;=20i++)=0A+=09= for=20(i=20=3D=200;=20i=20<=20sa->numslots;=20i++)=0A=20=09{=0A-=09=09if=20= (!consumeQueryResult(slots=20+=20i))=0A+=09=09if=20= (sa->slots[i].connection=20=3D=3D=20NULL)=0A+=09=09=09continue;=0A+=09=09= if=20(!consumeQueryResult(&sa->slots[i]))=0A=20=09=09=09return=20false;=0A= =20=09}=0A=20=0A@@=20-350,6=20+512,9=20@@=20= ParallelSlotsWaitCompletion(ParallelSlot=20*slots,=20int=20numslots)=0A=20= bool=0A=20TableCommandResultHandler(PGresult=20*res,=20PGconn=20*conn,=20= void=20*context)=0A=20{=0A+=09Assert(res=20!=3D=20NULL);=0A+=09= Assert(conn=20!=3D=20NULL);=0A+=0A=20=09/*=0A=20=09=20*=20If=20it's=20an=20= error,=20report=20it.=20=20Errors=20about=20a=20missing=20table=20are=20= harmless=0A=20=09=20*=20so=20we=20continue=20processing;=20but=20die=20= for=20other=20errors.=0Adiff=20--git=20= a/src/include/fe_utils/parallel_slot.h=20= b/src/include/fe_utils/parallel_slot.h=0Aindex=208902f8d4f4..b7e2b0a29b=20= 100644=0A---=20a/src/include/fe_utils/parallel_slot.h=0A+++=20= b/src/include/fe_utils/parallel_slot.h=0A@@=20-21,7=20+21,7=20@@=20= typedef=20bool=20(*ParallelSlotResultHandler)=20(PGresult=20*res,=20= PGconn=20*conn,=0A=20typedef=20struct=20ParallelSlot=0A=20{=0A=20=09= PGconn=09=20=20=20*connection;=09=09/*=20One=20connection=20*/=0A-=09= bool=09=09isFree;=09=09=09/*=20Is=20it=20known=20to=20be=20idle?=20*/=0A= +=09bool=09=09inUse;=09=09=09/*=20Is=20the=20slot=20being=20used?=20*/=0A= =20=0A=20=09/*=0A=20=09=20*=20Prior=20to=20issuing=20a=20command=20or=20= query=20on=20'connection',=20a=20handler=20callback=0A@@=20-33,6=20= +33,16=20@@=20typedef=20struct=20ParallelSlot=0A=20=09void=09=20=20=20= *handler_context;=0A=20}=20ParallelSlot;=0A=20=0A+typedef=20struct=20= ParallelSlotArray=0A+{=0A+=09int=09=09=09numslots;=0A+=09ConnParams=20= *cparams;=0A+=09const=20char=20*progname;=0A+=09bool=09=09echo;=0A+=09= const=20char=20*initcmd;=0A+=09ParallelSlot=20= slots[FLEXIBLE_ARRAY_MEMBER];=0A+}=20ParallelSlotArray;=0A+=0A=20static=20= inline=20void=0A=20ParallelSlotSetHandler(ParallelSlot=20*slot,=20= ParallelSlotResultHandler=20handler,=0A=20=09=09=09=09=09=20=20=20void=20= *context)=0A@@=20-48,15=20+58,18=20@@=20= ParallelSlotClearHandler(ParallelSlot=20*slot)=0A=20=09= slot->handler_context=20=3D=20NULL;=0A=20}=0A=20=0A-extern=20= ParallelSlot=20*ParallelSlotsGetIdle(ParallelSlot=20*slots,=20int=20= numslots);=0A+extern=20ParallelSlot=20= *ParallelSlotsGetIdle(ParallelSlotArray=20*slots,=0A+=09=09=09=09=09=09=09= =09=09=09=20=20const=20char=20*dbname);=0A+=0A+extern=20= ParallelSlotArray=20*ParallelSlotsSetup(int=20numslots,=20ConnParams=20= *cparams,=0A+=09=09=09=09=09=09=09=09=09=09=09=20const=20char=20= *progname,=20bool=20echo,=0A+=09=09=09=09=09=09=09=09=09=09=09=20const=20= char=20*initcmd);=0A=20=0A-extern=20ParallelSlot=20= *ParallelSlotsSetup(const=20ConnParams=20*cparams,=0A-=09=09=09=09=09=09=09= =09=09=09const=20char=20*progname,=20bool=20echo,=0A-=09=09=09=09=09=09=09= =09=09=09PGconn=20*conn,=20int=20numslots);=0A+extern=20void=20= ParallelSlotsAdoptConn(ParallelSlotArray=20*sa,=20PGconn=20*conn);=0A=20=0A= -extern=20void=20ParallelSlotsTerminate(ParallelSlot=20*slots,=20int=20= numslots);=0A+extern=20void=20ParallelSlotsTerminate(ParallelSlotArray=20= *sa);=0A=20=0A-extern=20bool=20ParallelSlotsWaitCompletion(ParallelSlot=20= *slots,=20int=20numslots);=0A+extern=20bool=20= ParallelSlotsWaitCompletion(ParallelSlotArray=20*sa);=0A=20=0A=20extern=20= bool=20TableCommandResultHandler(PGresult=20*res,=20PGconn=20*conn,=0A=20= =09=09=09=09=09=09=09=09=09=20=20void=20*context);=0Adiff=20--git=20= a/src/tools/pgindent/typedefs.list=20b/src/tools/pgindent/typedefs.list=0A= index=208bd95aefa1..b1dec43f9d=20100644=0A---=20= a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-403,6=20+403,7=20@@=20= ConfigData=0A=20ConfigVariable=0A=20ConnCacheEntry=0A=20ConnCacheKey=0A= +ConnParams=0A=20ConnStatusType=0A=20ConnType=0A=20ConnectionStateEnum=0A= @@=20-1729,6=20+1730,7=20@@=20ParallelHashJoinState=0A=20= ParallelIndexScanDesc=0A=20ParallelReadyList=0A=20ParallelSlot=0A= +ParallelSlotArray=0A=20ParallelState=0A=20ParallelTableScanDesc=0A=20= ParallelTableScanDescData=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A Content-Disposition: attachment; filename=v42-0002-Adding-contrib-module-pg_amcheck.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v42-0002-Adding-contrib-module-pg_amcheck.patch" Content-Transfer-Encoding: quoted-printable =46rom=20b59f43cc27ae69c73d8c1209346166b9c13ef68e=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Tue,=202=20Mar=202021=2008:34:40=20-0800=0ASubject:=20[PATCH=20= v42=202/3]=20Adding=20contrib=20module=20pg_amcheck=0A=0AAdding=20new=20= contrib=20module=20pg_amcheck,=20which=20is=20a=20command=20line=0A= interface=20for=20running=20amcheck's=20verifications=20against=20tables=20= and=0Aindexes.=0A---=0A=20contrib/Makefile=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= contrib/pg_amcheck/.gitignore=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= =20=20=203=20+=0A=20contrib/pg_amcheck/Makefile=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20|=20=20=2029=20+=0A=20= contrib/pg_amcheck/pg_amcheck.c=20=20=20=20=20=20=20=20=20=20=20=20|=20= 1939=20++++++++++++++++++++=0A=20contrib/pg_amcheck/t/001_basic.pl=20=20=20= =20=20=20=20=20=20=20|=20=20=20=209=20+=0A=20= contrib/pg_amcheck/t/002_nonesuch.pl=20=20=20=20=20=20=20|=20=20213=20= +++=0A=20contrib/pg_amcheck/t/003_check.pl=20=20=20=20=20=20=20=20=20=20= |=20=20497=20+++++=0A=20contrib/pg_amcheck/t/004_verify_heapam.pl=20=20|=20= =20487=20+++++=0A=20contrib/pg_amcheck/t/005_opclass_damage.pl=20|=20=20=20= 54=20+=0A=20doc/src/sgml/contrib.sgml=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20|=20=20=20=201=20+=0A=20doc/src/sgml/filelist.sgml=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20=20=201=20+=0A=20= doc/src/sgml/pgamcheck.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20668=20+++++++=0A=20src/tools/msvc/Install.pm=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=20=202=20+-=0A=20= src/tools/msvc/Mkvcbuild.pm=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= |=20=20=20=206=20+-=0A=20src/tools/pgindent/typedefs.list=20=20=20=20=20=20= =20=20=20=20=20|=20=20=20=203=20+=0A=2015=20files=20changed,=203909=20= insertions(+),=204=20deletions(-)=0A=20create=20mode=20100644=20= contrib/pg_amcheck/.gitignore=0A=20create=20mode=20100644=20= contrib/pg_amcheck/Makefile=0A=20create=20mode=20100644=20= contrib/pg_amcheck/pg_amcheck.c=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/001_basic.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/002_nonesuch.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/003_check.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/004_verify_heapam.pl=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/005_opclass_damage.pl=0A=20create=20mode=20100644=20= doc/src/sgml/pgamcheck.sgml=0A=0Adiff=20--git=20a/contrib/Makefile=20= b/contrib/Makefile=0Aindex=20f27e458482..a72dcf7304=20100644=0A---=20= a/contrib/Makefile=0A+++=20b/contrib/Makefile=0A@@=20-30,6=20+30,7=20@@=20= SUBDIRS=20=3D=20\=0A=20=09=09old_snapshot=09\=0A=20=09=09pageinspect=09\=0A= =20=09=09passwordcheck=09\=0A+=09=09pg_amcheck=09\=0A=20=09=09= pg_buffercache=09\=0A=20=09=09pg_freespacemap=20\=0A=20=09=09pg_prewarm=09= \=0Adiff=20--git=20a/contrib/pg_amcheck/.gitignore=20= b/contrib/pg_amcheck/.gitignore=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..c21a14de31=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/.gitignore=0A@@=20-0,0=20+1,3=20@@=0A+pg_amcheck=0A= +=0A+/tmp_check/=0Adiff=20--git=20a/contrib/pg_amcheck/Makefile=20= b/contrib/pg_amcheck/Makefile=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..bc61ee7970=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/Makefile=0A@@=20-0,0=20+1,29=20@@=0A+#=20= contrib/pg_amcheck/Makefile=0A+=0A+PGFILEDESC=20=3D=20"pg_amcheck=20-=20= detects=20corruption=20within=20database=20relations"=0A+PGAPPICON=20=3D=20= win32=0A+=0A+PROGRAM=20=3D=20pg_amcheck=0A+OBJS=20=3D=20\=0A+=09= $(WIN32RES)=20\=0A+=09pg_amcheck.o=0A+=0A+REGRESS_OPTS=20+=3D=20= --load-extension=3Damcheck=20--load-extension=3Dpageinspect=0A= +EXTRA_INSTALL=20+=3D=20contrib/amcheck=20contrib/pageinspect=0A+=0A= +TAP_TESTS=20=3D=201=0A+=0A+PG_CPPFLAGS=20=3D=20-I$(libpq_srcdir)=0A= +PG_LIBS_INTERNAL=20=3D=20-L$(top_builddir)/src/fe_utils=20-lpgfeutils=20= $(libpq_pgport)=0A+=0A+ifdef=20USE_PGXS=0A+PG_CONFIG=20=3D=20pg_config=0A= +PGXS=20:=3D=20$(shell=20$(PG_CONFIG)=20--pgxs)=0A+include=20$(PGXS)=0A= +else=0A+SHLIB_PREREQS=20=3D=20submake-libpq=0A+subdir=20=3D=20= contrib/pg_amcheck=0A+top_builddir=20=3D=20../..=0A+include=20= $(top_builddir)/src/Makefile.global=0A+include=20= $(top_srcdir)/contrib/contrib-global.mk=0A+endif=0Adiff=20--git=20= a/contrib/pg_amcheck/pg_amcheck.c=20b/contrib/pg_amcheck/pg_amcheck.c=0A= new=20file=20mode=20100644=0Aindex=200000000000..63982fd66b=0A---=20= /dev/null=0A+++=20b/contrib/pg_amcheck/pg_amcheck.c=0A@@=20-0,0=20= +1,1939=20@@=0A= +/*-----------------------------------------------------------------------= --=0A+=20*=0A+=20*=20pg_amcheck.c=0A+=20*=09=09Detects=20corruption=20= within=20database=20relations.=0A+=20*=0A+=20*=20Copyright=20(c)=20= 2017-2021,=20PostgreSQL=20Global=20Development=20Group=0A+=20*=0A+=20*=20= IDENTIFICATION=0A+=20*=09=20=20contrib/pg_amcheck/pg_amcheck.c=0A+=20*=0A= +=20= *-------------------------------------------------------------------------= =0A+=20*/=0A+#include=20"postgres_fe.h"=0A+=0A+#include=20=0A+=0A= +#include=20"catalog/pg_am_d.h"=0A+#include=20"catalog/pg_namespace_d.h"=0A= +#include=20"common/logging.h"=0A+#include=20"common/username.h"=0A= +#include=20"fe_utils/cancel.h"=0A+#include=20"fe_utils/option_utils.h"=0A= +#include=20"fe_utils/parallel_slot.h"=0A+#include=20= "fe_utils/query_utils.h"=0A+#include=20"fe_utils/simple_list.h"=0A= +#include=20"fe_utils/string_utils.h"=0A+#include=20"getopt_long.h"=09=09= /*=20pgrminclude=20ignore=20*/=0A+#include=20"pgtime.h"=0A+#include=20= "storage/block.h"=0A+=0A+/*=20pg_amcheck=20command=20line=20options=20= controlled=20by=20user=20flags=20*/=0A+typedef=20struct=20AmcheckOptions=0A= +{=0A+=09bool=09=09alldb;=0A+=09bool=09=09echo;=0A+=09bool=09=09quiet;=0A= +=09bool=09=09verbose;=0A+=09bool=09=09strict_names;=0A+=09bool=09=09= show_progress;=0A+=09int=09=09=09jobs;=0A+=0A+=09/*=0A+=09=20*=20= Relations=20to=20check=20or=20not=20to=20check=20(both=20heap=20and=20= btree=20together),=20as=0A+=09=20*=20lists=20of=20PatternInfo=20structs.=0A= +=09=20*/=0A+=09SimplePtrList=20include;=0A+=09SimplePtrList=20exclude;=0A= +=0A+=09/*=0A+=09=20*=20As=20an=20optimization,=20if=20any=20pattern=20= in=20the=20exclude=20list=20applies=20to=20heap=0A+=09=20*=20tables,=20= or=20similarly=20if=20any=20such=20pattern=20applies=20to=20btree=20= indexes,=20then=0A+=09=20*=20these=20will=20be=20true,=20otherwise=20= false.=20=20These=20should=20always=20agree=20with=0A+=09=20*=20what=20= you'd=20conclude=20by=20grep'ing=20through=20the=20exclude=20list.=0A+=09= =20*/=0A+=09bool=09=09excludetbl;=0A+=09bool=09=09excludeidx;=0A+=0A+=09= /*=0A+=09=20*=20If=20any=20inclusion=20pattern=20exists,=20then=20we=20= should=20only=20be=20checking=0A+=09=20*=20matching=20relations=20rather=20= than=20all=20relations,=20so=20this=20is=20true=20iff=0A+=09=20*=20= include=20is=20empty.=0A+=09=20*/=0A+=09bool=09=09allrel;=0A+=0A+=09/*=20= heap=20table=20checking=20options=20*/=0A+=09bool=09=09= no_toast_expansion;=0A+=09bool=09=09reconcile_toast;=0A+=09bool=09=09= on_error_stop;=0A+=09long=09=09startblock;=0A+=09long=09=09endblock;=0A+=09= const=20char=20*skip;=0A+=0A+=09/*=20btree=20index=20checking=20options=20= */=0A+=09bool=09=09parent_check;=0A+=09bool=09=09rootdescend;=0A+=09bool=09= =09heapallindexed;=0A+=0A+=09/*=20heap=20and=20btree=20hybrid=20option=20= */=0A+=09bool=09=09no_index_expansion;=0A+}=20AmcheckOptions;=0A+=0A= +static=20AmcheckOptions=20opts=20=3D=20{=0A+=09.alldb=20=3D=20false,=0A= +=09.echo=20=3D=20false,=0A+=09.quiet=20=3D=20false,=0A+=09.verbose=20=3D=20= false,=0A+=09.strict_names=20=3D=20true,=0A+=09.show_progress=20=3D=20= false,=0A+=09.jobs=20=3D=201,=0A+=09.include=20=3D=20{NULL,=20NULL},=0A+=09= .exclude=20=3D=20{NULL,=20NULL},=0A+=09.excludetbl=20=3D=20false,=0A+=09= .excludeidx=20=3D=20false,=0A+=09.allrel=20=3D=20true,=0A+=09= .no_toast_expansion=20=3D=20false,=0A+=09.reconcile_toast=20=3D=20true,=0A= +=09.on_error_stop=20=3D=20false,=0A+=09.startblock=20=3D=20-1,=0A+=09= .endblock=20=3D=20-1,=0A+=09.skip=20=3D=20"none",=0A+=09.parent_check=20= =3D=20false,=0A+=09.rootdescend=20=3D=20false,=0A+=09.heapallindexed=20=3D= =20false,=0A+=09.no_index_expansion=20=3D=20false=0A+};=0A+=0A+static=20= const=20char=20*progname=20=3D=20NULL;=0A+=0A+typedef=20struct=20= PatternInfo=0A+{=0A+=09int=09=09=09pattern_id;=09=09/*=20Unique=20ID=20= of=20this=20pattern=20*/=0A+=09const=20char=20*pattern;=09=09/*=20= Unaltered=20pattern=20from=20the=20command=20line=20*/=0A+=09char=09=20=20= =20*db_regex;=09=09/*=20Database=20regexp=20parsed=20from=20pattern,=20= or=0A+=09=09=09=09=09=09=09=09=20*=20NULL=20*/=0A+=09char=09=20=20=20= *nsp_regex;=09=09/*=20Schema=20regexp=20parsed=20from=20pattern,=20or=20= NULL=20*/=0A+=09char=09=20=20=20*rel_regex;=09=09/*=20Relation=20regexp=20= parsed=20from=20pattern,=20or=0A+=09=09=09=09=09=09=09=09=20*=20NULL=20= */=0A+=09bool=09=09table_only;=09=09/*=20true=20if=20rel_regex=20should=20= only=20match=20tables=20*/=0A+=09bool=09=09index_only;=09=09/*=20true=20= if=20rel_regex=20should=20only=20match=20indexes=20*/=0A+=09bool=09=09= matched;=09=09/*=20true=20if=20the=20pattern=20matched=20in=20any=20= database=20*/=0A+}=09=09=09PatternInfo;=0A+=0A+/*=20Unique=20pattern=20= id=20counter=20*/=0A+static=20int=09next_id=20=3D=201;=0A+=0A+/*=20= Whether=20all=20relations=20have=20so=20far=20passed=20their=20= corruption=20checks=20*/=0A+static=20bool=20all_checks_pass=20=3D=20= true;=0A+=0A+/*=20Time=20last=20progress=20report=20was=20displayed=20*/=0A= +static=20pg_time_t=20last_progress_report=20=3D=200;=0A+=0A+typedef=20= struct=20DatabaseInfo=0A+{=0A+=09char=09=20=20=20*datname;=0A+=09char=09=20= =20=20*amcheck_schema;=20/*=20escaped,=20quoted=20literal=20*/=0A+}=20= DatabaseInfo;=0A+=0A+typedef=20struct=20RelationInfo=0A+{=0A+=09const=20= DatabaseInfo=20*datinfo;=09/*=20shared=20by=20other=20relinfos=20*/=0A+=09= Oid=09=09=09reloid;=0A+=09bool=09=09is_table;=09=09/*=20true=20if=20= heap,=20false=20if=20btree=20*/=0A+}=20RelationInfo;=0A+=0A+/*=0A+=20*=20= Query=20for=20determining=20if=20contrib's=20amcheck=20is=20installed.=20= =20If=20so,=20selects=20the=0A+=20*=20namespace=20name=20where=20= amcheck's=20functions=20can=20be=20found.=0A+=20*/=0A+static=20const=20= char=20*amcheck_sql=20=3D=0A+"SELECT=20n.nspname,=20x.extversion"=0A= +"\nFROM=20pg_catalog.pg_extension=20x"=0A+"\nJOIN=20= pg_catalog.pg_namespace=20n"=0A+"\nON=20x.extnamespace=20=3D=20n.oid"=0A= +"\nWHERE=20x.extname=20=3D=20'amcheck'";=0A+=0A+static=20void=20= prepare_table_command(PQExpBuffer=20sql,=20Oid=20reloid,=0A+=09=09=09=09=09= =09=09=09=20=20const=20char=20*nspname);=0A+static=20void=20= prepare_btree_command(PQExpBuffer=20sql,=20Oid=20reloid,=0A+=09=09=09=09=09= =09=09=09=20=20const=20char=20*nspname);=0A+static=20void=20= run_command(ParallelSlot=20*slot,=20const=20char=20*sql,=0A+=09=09=09=09=09= =09ConnParams=20*cparams);=0A+static=20bool=20= verify_heapam_slot_handler(PGresult=20*res,=20PGconn=20*conn,=0A+=09=09=09= =09=09=09=09=09=09=20=20=20void=20*context);=0A+static=20bool=20= verify_btree_slot_handler(PGresult=20*res,=20PGconn=20*conn,=20void=20= *context);=0A+static=20void=20help(const=20char=20*progname);=0A+static=20= void=20progress_report(uint64=20relations_total,=20uint64=20= relations_checked,=0A+=09=09=09=09=09=09=09const=20char=20*datname,=20= bool=20force,=20bool=20finished);=0A+=0A+static=20void=20= append_database_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=0A+=09=09=09=09=09=09=09=09=09int=20encoding);=0A+static=20= void=20append_schema_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=0A+=09=09=09=09=09=09=09=09=20=20int=20encoding);=0A+static=20= void=20append_relation_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=0A+=09=09=09=09=09=09=09=09=09int=20encoding);=0A+static=20= void=20append_table_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=0A+=09=09=09=09=09=09=09=09=20int=20encoding);=0A+static=20= void=20append_index_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=0A+=09=09=09=09=09=09=09=09=20int=20encoding);=0A+static=20= void=20compile_database_list(PGconn=20*conn,=20SimplePtrList=20= *databases);=0A+static=20void=20compile_relation_list_one_db(PGconn=20= *conn,=20SimplePtrList=20*relations,=0A+=09=09=09=09=09=09=09=09=09=09=20= const=20DatabaseInfo=20*datinfo);=0A+=0A+int=0A+main(int=20argc,=20char=20= *argv[])=0A+{=0A+=09PGconn=09=20=20=20*conn;=0A+=09SimplePtrListCell=20= *cell;=0A+=09SimplePtrList=20databases=20=3D=20{NULL,=20NULL};=0A+=09= SimplePtrList=20relations=20=3D=20{NULL,=20NULL};=0A+=09bool=09=09failed=20= =3D=20false;=0A+=09const=20char=20*latest_datname;=0A+=09int=09=09=09= parallel_workers;=0A+=09ParallelSlotArray=20*sa;=0A+=09PQExpBufferData=20= sql;=0A+=09long=20long=20int=20reltotal;=0A+=09long=20long=20int=20= relprogress;=0A+=0A+=09static=20struct=20option=20long_options[]=20=3D=20= {=0A+=09=09/*=20Connection=20options=20*/=0A+=09=09{"host",=20= required_argument,=20NULL,=20'h'},=0A+=09=09{"port",=20= required_argument,=20NULL,=20'p'},=0A+=09=09{"username",=20= required_argument,=20NULL,=20'U'},=0A+=09=09{"no-password",=20= no_argument,=20NULL,=20'w'},=0A+=09=09{"password",=20no_argument,=20= NULL,=20'W'},=0A+=09=09{"maintenance-db",=20required_argument,=20NULL,=20= 1},=0A+=0A+=09=09/*=20check=20options=20*/=0A+=09=09{"all",=20= no_argument,=20NULL,=20'a'},=0A+=09=09{"dbname",=20required_argument,=20= NULL,=20'd'},=0A+=09=09{"exclude-dbname",=20required_argument,=20NULL,=20= 'D'},=0A+=09=09{"echo",=20no_argument,=20NULL,=20'e'},=0A+=09=09= {"index",=20required_argument,=20NULL,=20'i'},=0A+=09=09= {"exclude-index",=20required_argument,=20NULL,=20'I'},=0A+=09=09{"jobs",=20= required_argument,=20NULL,=20'j'},=0A+=09=09{"quiet",=20no_argument,=20= NULL,=20'q'},=0A+=09=09{"relation",=20required_argument,=20NULL,=20'r'},=0A= +=09=09{"exclude-relation",=20required_argument,=20NULL,=20'R'},=0A+=09=09= {"schema",=20required_argument,=20NULL,=20's'},=0A+=09=09= {"exclude-schema",=20required_argument,=20NULL,=20'S'},=0A+=09=09= {"table",=20required_argument,=20NULL,=20't'},=0A+=09=09= {"exclude-table",=20required_argument,=20NULL,=20'T'},=0A+=09=09= {"verbose",=20no_argument,=20NULL,=20'v'},=0A+=09=09= {"no-index-expansion",=20no_argument,=20NULL,=202},=0A+=09=09= {"no-toast-expansion",=20no_argument,=20NULL,=203},=0A+=09=09= {"exclude-toast-pointers",=20no_argument,=20NULL,=204},=0A+=09=09= {"on-error-stop",=20no_argument,=20NULL,=205},=0A+=09=09{"skip",=20= required_argument,=20NULL,=206},=0A+=09=09{"startblock",=20= required_argument,=20NULL,=207},=0A+=09=09{"endblock",=20= required_argument,=20NULL,=208},=0A+=09=09{"rootdescend",=20no_argument,=20= NULL,=209},=0A+=09=09{"no-strict-names",=20no_argument,=20NULL,=2010},=0A= +=09=09{"progress",=20no_argument,=20NULL,=2011},=0A+=09=09= {"heapallindexed",=20no_argument,=20NULL,=2012},=0A+=09=09= {"parent-check",=20no_argument,=20NULL,=2013},=0A+=0A+=09=09{NULL,=200,=20= NULL,=200}=0A+=09};=0A+=0A+=09int=09=09=09optindex;=0A+=09int=09=09=09c;=0A= +=0A+=09/*=0A+=09=20*=20If=20a=20maintenance=20database=20is=20= specified,=20that=20will=20be=20used=20for=20the=0A+=09=20*=20initial=20= connection.=20=20Failing=20that,=20the=20first=20plain=20argument=20= (without=20a=0A+=09=20*=20flag)=20will=20be=20used.=20=20If=20neither=20= of=20those=20are=20given,=20the=20first=20database=0A+=09=20*=20= specified=20with=20-d.=0A+=09=20*/=0A+=09const=20char=20*primary_db=20=3D=20= NULL;=0A+=09const=20char=20*secondary_db=20=3D=20NULL;=0A+=09const=20= char=20*tertiary_db=20=3D=20NULL;=0A+=0A+=09const=20char=20*host=20=3D=20= NULL;=0A+=09const=20char=20*port=20=3D=20NULL;=0A+=09const=20char=20= *username=20=3D=20NULL;=0A+=09enum=20trivalue=20prompt_password=20=3D=20= TRI_DEFAULT;=0A+=09int=09=09=09encoding=20=3D=20= pg_get_encoding_from_locale(NULL,=20false);=0A+=09ConnParams=09cparams;=0A= +=0A+=09pg_logging_init(argv[0]);=0A+=09progname=20=3D=20= get_progname(argv[0]);=0A+=09set_pglocale_pgservice(argv[0],=20= PG_TEXTDOMAIN("contrib"));=0A+=0A+=09handle_help_version_opts(argc,=20= argv,=20progname,=20help);=0A+=0A+=09/*=20process=20command-line=20= options=20*/=0A+=09while=20((c=20=3D=20getopt_long(argc,=20argv,=20= "ad:D:eh:Hi:I:j:p:Pqr:R:s:S:t:T:U:wWv",=0A+=09=09=09=09=09=09=09= long_options,=20&optindex))=20!=3D=20-1)=0A+=09{=0A+=09=09char=09=20=20=20= *endptr;=0A+=0A+=09=09switch=20(c)=0A+=09=09{=0A+=09=09=09case=20'a':=0A= +=09=09=09=09opts.alldb=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'd':=0A+=09=09=09=09if=20(tertiary_db=20=3D=3D=20NULL)=0A+=09=09=09= =09=09tertiary_db=20=3D=20optarg;=0A+=09=09=09=09= append_database_pattern(&opts.include,=20optarg,=20encoding);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'D':=0A+=09=09=09=09= append_database_pattern(&opts.exclude,=20optarg,=20encoding);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'e':=0A+=09=09=09=09opts.echo=20=3D=20true;=0A= +=09=09=09=09break;=0A+=09=09=09case=20'h':=0A+=09=09=09=09host=20=3D=20= pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'i':=0A+=09=09= =09=09opts.allrel=20=3D=20false;=0A+=09=09=09=09= append_index_pattern(&opts.include,=20optarg,=20encoding);=0A+=09=09=09=09= break;=0A+=09=09=09case=20'I':=0A+=09=09=09=09opts.excludeidx=20=3D=20= true;=0A+=09=09=09=09append_index_pattern(&opts.exclude,=20optarg,=20= encoding);=0A+=09=09=09=09break;=0A+=09=09=09case=20'j':=0A+=09=09=09=09= opts.jobs=20=3D=20atoi(optarg);=0A+=09=09=09=09if=20(opts.jobs=20<=201)=0A= +=09=09=09=09{=0A+=09=09=09=09=09pg_log_error("number=20of=20parallel=20= jobs=20must=20be=20at=20least=201");=0A+=09=09=09=09=09exit(1);=0A+=09=09= =09=09}=0A+=09=09=09=09break;=0A+=09=09=09case=20'p':=0A+=09=09=09=09= port=20=3D=20pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20= 'q':=0A+=09=09=09=09opts.quiet=20=3D=20true;=0A+=09=09=09=09break;=0A+=09= =09=09case=20'r':=0A+=09=09=09=09opts.allrel=20=3D=20false;=0A+=09=09=09=09= append_relation_pattern(&opts.include,=20optarg,=20encoding);=0A+=09=09=09= =09break;=0A+=09=09=09case=20'R':=0A+=09=09=09=09opts.excludeidx=20=3D=20= true;=0A+=09=09=09=09opts.excludetbl=20=3D=20true;=0A+=09=09=09=09= append_relation_pattern(&opts.exclude,=20optarg,=20encoding);=0A+=09=09=09= =09break;=0A+=09=09=09case=20's':=0A+=09=09=09=09opts.allrel=20=3D=20= false;=0A+=09=09=09=09append_schema_pattern(&opts.include,=20optarg,=20= encoding);=0A+=09=09=09=09break;=0A+=09=09=09case=20'S':=0A+=09=09=09=09= append_schema_pattern(&opts.exclude,=20optarg,=20encoding);=0A+=09=09=09=09= break;=0A+=09=09=09case=20't':=0A+=09=09=09=09opts.allrel=20=3D=20false;=0A= +=09=09=09=09append_table_pattern(&opts.include,=20optarg,=20encoding);=0A= +=09=09=09=09break;=0A+=09=09=09case=20'T':=0A+=09=09=09=09= opts.excludetbl=20=3D=20true;=0A+=09=09=09=09= append_table_pattern(&opts.exclude,=20optarg,=20encoding);=0A+=09=09=09=09= break;=0A+=09=09=09case=20'U':=0A+=09=09=09=09username=20=3D=20= pg_strdup(optarg);=0A+=09=09=09=09break;=0A+=09=09=09case=20'w':=0A+=09=09= =09=09prompt_password=20=3D=20TRI_NO;=0A+=09=09=09=09break;=0A+=09=09=09= case=20'W':=0A+=09=09=09=09prompt_password=20=3D=20TRI_YES;=0A+=09=09=09=09= break;=0A+=09=09=09case=20'v':=0A+=09=09=09=09opts.verbose=20=3D=20true;=0A= +=09=09=09=09pg_logging_increase_verbosity();=0A+=09=09=09=09break;=0A+=09= =09=09case=201:=0A+=09=09=09=09primary_db=20=3D=20pg_strdup(optarg);=0A+=09= =09=09=09break;=0A+=09=09=09case=202:=0A+=09=09=09=09= opts.no_index_expansion=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09= case=203:=0A+=09=09=09=09opts.no_toast_expansion=20=3D=20true;=0A+=09=09=09= =09break;=0A+=09=09=09case=204:=0A+=09=09=09=09opts.reconcile_toast=20=3D=20= false;=0A+=09=09=09=09break;=0A+=09=09=09case=205:=0A+=09=09=09=09= opts.on_error_stop=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20= 6:=0A+=09=09=09=09if=20(pg_strcasecmp(optarg,=20"all-visible")=20=3D=3D=20= 0)=0A+=09=09=09=09=09opts.skip=20=3D=20"all=20visible";=0A+=09=09=09=09= else=20if=20(pg_strcasecmp(optarg,=20"all-frozen")=20=3D=3D=200)=0A+=09=09= =09=09=09opts.skip=20=3D=20"all=20frozen";=0A+=09=09=09=09else=0A+=09=09=09= =09{=0A+=09=09=09=09=09fprintf(stderr,=20"invalid=20skip=20options");=0A= +=09=09=09=09=09exit(1);=0A+=09=09=09=09}=0A+=09=09=09=09break;=0A+=09=09= =09case=207:=0A+=09=09=09=09opts.startblock=20=3D=20strtol(optarg,=20= &endptr,=2010);=0A+=09=09=09=09if=20(*endptr=20!=3D=20'\0')=0A+=09=09=09=09= {=0A+=09=09=09=09=09fprintf(stderr,=0A+=09=09=09=09=09=09=09"relation=20= starting=20block=20argument=20contains=20garbage=20characters");=0A+=09=09= =09=09=09exit(1);=0A+=09=09=09=09}=0A+=09=09=09=09if=20(opts.startblock=20= >=20(long)=20MaxBlockNumber)=0A+=09=09=09=09{=0A+=09=09=09=09=09= fprintf(stderr,=0A+=09=09=09=09=09=09=09"relation=20starting=20block=20= argument=20out=20of=20bounds");=0A+=09=09=09=09=09exit(1);=0A+=09=09=09=09= }=0A+=09=09=09=09break;=0A+=09=09=09case=208:=0A+=09=09=09=09= opts.endblock=20=3D=20strtol(optarg,=20&endptr,=2010);=0A+=09=09=09=09if=20= (*endptr=20!=3D=20'\0')=0A+=09=09=09=09{=0A+=09=09=09=09=09= fprintf(stderr,=0A+=09=09=09=09=09=09=09"relation=20ending=20block=20= argument=20contains=20garbage=20characters");=0A+=09=09=09=09=09exit(1);=0A= +=09=09=09=09}=0A+=09=09=09=09if=20(opts.startblock=20>=20(long)=20= MaxBlockNumber)=0A+=09=09=09=09{=0A+=09=09=09=09=09fprintf(stderr,=0A+=09= =09=09=09=09=09=09"relation=20ending=20block=20argument=20out=20of=20= bounds");=0A+=09=09=09=09=09exit(1);=0A+=09=09=09=09}=0A+=09=09=09=09= break;=0A+=09=09=09case=209:=0A+=09=09=09=09opts.rootdescend=20=3D=20= true;=0A+=09=09=09=09opts.parent_check=20=3D=20true;=0A+=09=09=09=09= break;=0A+=09=09=09case=2010:=0A+=09=09=09=09opts.strict_names=20=3D=20= false;=0A+=09=09=09=09break;=0A+=09=09=09case=2011:=0A+=09=09=09=09= opts.show_progress=20=3D=20true;=0A+=09=09=09=09break;=0A+=09=09=09case=20= 12:=0A+=09=09=09=09opts.heapallindexed=20=3D=20true;=0A+=09=09=09=09= break;=0A+=09=09=09case=2013:=0A+=09=09=09=09opts.parent_check=20=3D=20= true;=0A+=09=09=09=09break;=0A+=09=09=09default:=0A+=09=09=09=09= fprintf(stderr,=0A+=09=09=09=09=09=09"Try=20\"%s=20--help\"=20for=20more=20= information.\n",=0A+=09=09=09=09=09=09progname);=0A+=09=09=09=09exit(1);=0A= +=09=09}=0A+=09}=0A+=0A+=09if=20(opts.endblock=20>=3D=200=20&&=20= opts.endblock=20<=20opts.startblock)=0A+=09{=0A+=09=09= pg_log_error("relation=20ending=20block=20argument=20precedes=20starting=20= block=20argument");=0A+=09=09exit(1);=0A+=09}=0A+=0A+=09/*=20non-option=20= arguments=20specify=20database=20names=20*/=0A+=09while=20(optind=20<=20= argc)=0A+=09{=0A+=09=09if=20(secondary_db=20=3D=3D=20NULL)=0A+=09=09=09= secondary_db=20=3D=20argv[optind];=0A+=09=09= append_database_pattern(&opts.include,=20argv[optind],=20encoding);=0A+=09= =09optind++;=0A+=09}=0A+=0A+=09/*=20fill=20cparams=20except=20for=20= dbname,=20which=20is=20set=20below=20*/=0A+=09cparams.pghost=20=3D=20= host;=0A+=09cparams.pgport=20=3D=20port;=0A+=09cparams.pguser=20=3D=20= username;=0A+=09cparams.prompt_password=20=3D=20prompt_password;=0A+=09= cparams.override_dbname=20=3D=20NULL;=0A+=0A+=09= setup_cancel_handler(NULL);=0A+=0A+=09/*=20choose=20the=20database=20for=20= our=20initial=20connection=20*/=0A+=09if=20(primary_db)=0A+=09=09= cparams.dbname=20=3D=20primary_db;=0A+=09else=20if=20(secondary_db=20!=3D=20= NULL)=0A+=09=09cparams.dbname=20=3D=20secondary_db;=0A+=09else=20if=20= (tertiary_db=20!=3D=20NULL)=0A+=09=09cparams.dbname=20=3D=20tertiary_db;=0A= +=09else=0A+=09{=0A+=09=09const=20char=20*default_db;=0A+=0A+=09=09if=20= (getenv("PGDATABASE"))=0A+=09=09=09default_db=20=3D=20= getenv("PGDATABASE");=0A+=09=09else=20if=20(getenv("PGUSER"))=0A+=09=09=09= default_db=20=3D=20getenv("PGUSER");=0A+=09=09else=0A+=09=09=09= default_db=20=3D=20get_user_name_or_exit(progname);=0A+=0A+=09=09/*=0A+=09= =09=20*=20Users=20expect=20the=20database=20name=20inferred=20from=20the=20= environment=20to=20get=0A+=09=09=20*=20checked,=20not=20just=20get=20= used=20for=20the=20initial=20connection.=0A+=09=09=20*/=0A+=09=09= append_database_pattern(&opts.include,=20default_db,=20encoding);=0A+=0A= +=09=09cparams.dbname=20=3D=20default_db;=0A+=09}=0A+=0A+=09conn=20=3D=20= connectMaintenanceDatabase(&cparams,=20progname,=20opts.echo);=0A+=09= compile_database_list(conn,=20&databases);=0A+=09= disconnectDatabase(conn);=0A+=0A+=09if=20(databases.head=20=3D=3D=20= NULL)=0A+=09{=0A+=09=09fprintf(stderr,=20"%s:=20no=20databases=20to=20= check\n",=20progname);=0A+=09=09exit(0);=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20= Compile=20a=20list=20of=20all=20relations=20spanning=20all=20databases=20= to=20be=20checked.=0A+=09=20*/=0A+=09for=20(cell=20=3D=20databases.head;=20= cell;=20cell=20=3D=20cell->next)=0A+=09{=0A+=09=09PGresult=20=20=20= *result;=0A+=09=09int=09=09=09ntups;=0A+=09=09const=20char=20= *amcheck_schema=20=3D=20NULL;=0A+=09=09DatabaseInfo=20*dat=20=3D=20= (DatabaseInfo=20*)=20cell->ptr;=0A+=0A+=09=09cparams.override_dbname=20=3D= =20dat->datname;=0A+=09=09conn=20=3D=20connectDatabase(&cparams,=20= progname,=20opts.echo,=20false,=20true);=0A+=0A+=09=09/*=0A+=09=09=20*=20= Verify=20that=20amcheck=20is=20installed=20for=20this=20next=20database.=20= =20User=0A+=09=09=20*=20error=20could=20result=20in=20a=20database=20not=20= having=20amcheck=20that=20should=0A+=09=09=20*=20have=20it,=20but=20we=20= also=20could=20be=20iterating=20over=20multiple=20databases=0A+=09=09=20= *=20where=20not=20all=20of=20them=20have=20amcheck=20installed=20(for=20= example,=0A+=09=09=20*=20'template1').=0A+=09=09=20*/=0A+=09=09result=20= =3D=20executeQuery(conn,=20amcheck_sql,=20opts.echo);=0A+=09=09if=20= (PQresultStatus(result)=20!=3D=20PGRES_TUPLES_OK)=0A+=09=09{=0A+=09=09=09= /*=20Querying=20the=20catalog=20failed.=20*/=0A+=09=09=09= pg_log_error("database=20\"%s\":=20%s\n",=0A+=09=09=09=09=09=09=20= PQdb(conn),=20PQerrorMessage(conn));=0A+=09=09=09pg_log_error("query=20= was:=20%s",=20amcheck_sql);=0A+=09=09=09PQclear(result);=0A+=09=09=09= disconnectDatabase(conn);=0A+=09=09=09exit(1);=0A+=09=09}=0A+=09=09ntups=20= =3D=20PQntuples(result);=0A+=09=09if=20(ntups=20=3D=3D=200)=0A+=09=09{=0A= +=09=09=09/*=20Querying=20the=20catalog=20succeeded,=20but=20amcheck=20= is=20missing.=20*/=0A+=09=09=09fprintf(stderr,=0A+=09=09=09=09=09"%s:=20= skipping=20database=20\"%s\":=20amcheck=20is=20not=20installed\n",=0A+=09= =09=09=09=09progname,=20PQdb(conn));=0A+=09=09=09= disconnectDatabase(conn);=0A+=09=09=09continue;=0A+=09=09}=0A+=09=09= amcheck_schema=20=3D=20PQgetvalue(result,=200,=200);=0A+=09=09if=20= (opts.verbose)=0A+=09=09=09fprintf(stderr,=0A+=09=09=09=09=09"%s:=20in=20= database=20\"%s\":=20using=20amcheck=20version=20\"%s\"=20in=20schema=20= \"%s\"\n",=0A+=09=09=09=09=09progname,=20PQdb(conn),=20= PQgetvalue(result,=200,=201),=0A+=09=09=09=09=09amcheck_schema);=0A+=09=09= dat->amcheck_schema=20=3D=20PQescapeIdentifier(conn,=20amcheck_schema,=0A= +=09=09=09=09=09=09=09=09=09=09=09=09=20strlen(amcheck_schema));=0A+=09=09= PQclear(result);=0A+=0A+=09=09compile_relation_list_one_db(conn,=20= &relations,=20dat);=0A+=09=09disconnectDatabase(conn);=0A+=09}=0A+=0A+=09= /*=0A+=09=20*=20Check=20that=20all=20inclusion=20patterns=20matched=20at=20= least=20one=20schema=20or=0A+=09=20*=20relation=20that=20we=20can=20= check.=0A+=09=20*/=0A+=09for=20(cell=20=3D=20opts.include.head;=20cell;=20= cell=20=3D=20cell->next)=0A+=09{=0A+=09=09PatternInfo=20*pat=20=3D=20= (PatternInfo=20*)=20cell->ptr;=0A+=0A+=09=09if=20(!pat->matched=20&&=20= (pat->nsp_regex=20!=3D=20NULL=20||=20pat->rel_regex=20!=3D=20NULL))=0A+=09= =09{=0A+=09=09=09failed=20=3D=20opts.strict_names;=0A+=0A+=09=09=09if=20= (!opts.quiet)=0A+=09=09=09{=0A+=09=09=09=09if=20(pat->table_only)=0A+=09=09= =09=09=09fprintf(stderr,=20"%s:=20no=20tables=20to=20check=20for=20= \"%s\"\n",=0A+=09=09=09=09=09=09=09progname,=20pat->pattern);=0A+=09=09=09= =09else=20if=20(pat->index_only)=0A+=09=09=09=09=09fprintf(stderr,=20= "%s:=20no=20btree=20indexes=20to=20check=20for=20\"%s\"\n",=0A+=09=09=09=09= =09=09=09progname,=20pat->pattern);=0A+=09=09=09=09else=20if=20= (pat->rel_regex=20=3D=3D=20NULL)=0A+=09=09=09=09=09fprintf(stderr,=20= "%s:=20no=20relations=20to=20check=20in=20schemas=20for=20\"%s\"\n",=0A+=09= =09=09=09=09=09=09progname,=20pat->pattern);=0A+=09=09=09=09else=0A+=09=09= =09=09=09fprintf(stderr,=20"%s:=20no=20relations=20to=20check=20for=20= \"%s\"\n",=0A+=09=09=09=09=09=09=09progname,=20pat->pattern);=0A+=09=09=09= }=0A+=09=09}=0A+=09}=0A+=0A+=09if=20(failed)=0A+=09=09exit(1);=0A+=0A+=09= /*=0A+=09=20*=20Set=20parallel_workers=20to=20the=20lesser=20of=20= opts.jobs=20and=20the=20number=20of=0A+=09=20*=20relations.=0A+=09=20*/=0A= +=09reltotal=20=3D=200;=0A+=09parallel_workers=20=3D=200;=0A+=09for=20= (cell=20=3D=20relations.head;=20cell;=20cell=20=3D=20cell->next)=0A+=09{=0A= +=09=09reltotal++;=0A+=09=09if=20(parallel_workers=20<=20opts.jobs)=0A+=09= =09=09parallel_workers++;=0A+=09}=0A+=0A+=09if=20(reltotal=20=3D=3D=200)=0A= +=09{=0A+=09=09fprintf(stderr,=20"%s:=20no=20relations=20to=20check",=20= progname);=0A+=09=09exit(1);=0A+=09}=0A+=09progress_report(reltotal,=20= 0,=20NULL,=20true,=20false);=0A+=0A+=09/*=0A+=09=20*=20Main=20event=20= loop.=0A+=09=20*=0A+=09=20*=20We=20use=20server-side=20parallelism=20to=20= check=20up=20to=20parallel_workers=0A+=09=20*=20relations=20in=20= parallel.=20=20The=20list=20of=20relations=20was=20computed=20in=20= database=0A+=09=20*=20order,=20which=20minimizes=20the=20number=20of=20= connects=20and=20disconnects=20as=20we=0A+=09=20*=20process=20the=20= list.=0A+=09=20*/=0A+=09latest_datname=20=3D=20NULL;=0A+=09sa=20=3D=20= ParallelSlotsSetup(parallel_workers,=20&cparams,=20progname,=20= opts.echo,=0A+=09=09=09=09=09=09=09NULL);=0A+=0A+=09= initPQExpBuffer(&sql);=0A+=09for=20(relprogress=20=3D=200,=20cell=20=3D=20= relations.head;=20cell;=20cell=20=3D=20cell->next)=0A+=09{=0A+=09=09= ParallelSlot=20*free_slot;=0A+=09=09RelationInfo=20*rel;=0A+=0A+=09=09= rel=20=3D=20(RelationInfo=20*)=20cell->ptr;=0A+=0A+=09=09if=20= (CancelRequested)=0A+=09=09{=0A+=09=09=09failed=20=3D=20true;=0A+=09=09=09= break;=0A+=09=09}=0A+=0A+=09=09/*=0A+=09=09=20*=20The=20list=20of=20= relations=20is=20in=20database=20sorted=20order.=20=20If=20this=20next=0A= +=09=09=20*=20relation=20is=20in=20a=20different=20database=20than=20the=20= last=20one=20seen,=20we=20are=0A+=09=09=20*=20about=20to=20start=20= checking=20this=20database.=20=20Note=20that=20other=20slots=20may=0A+=09= =09=20*=20still=20be=20working=20on=20relations=20from=20prior=20= databases.=0A+=09=09=20*/=0A+=09=09latest_datname=20=3D=20= rel->datinfo->datname;=0A+=0A+=09=09progress_report(reltotal,=20= relprogress,=20latest_datname,=20false,=20false);=0A+=0A+=09=09= relprogress++;=0A+=0A+=09=09/*=0A+=09=09=20*=20Get=20a=20parallel=20slot=20= for=20the=20next=20amcheck=20command,=20blocking=20if=0A+=09=09=20*=20= necessary=20until=20one=20is=20available,=20or=20until=20a=20previously=20= issued=20slot=0A+=09=09=20*=20command=20fails,=20indicating=20that=20we=20= should=20abort=20checking=20the=0A+=09=09=20*=20remaining=20objects.=0A+=09= =09=20*/=0A+=09=09free_slot=20=3D=20ParallelSlotsGetIdle(sa,=20= rel->datinfo->datname);=0A+=09=09if=20(!free_slot)=0A+=09=09{=0A+=09=09=09= /*=0A+=09=09=09=20*=20Something=20failed.=20=20We=20don't=20need=20to=20= know=20what=20it=20was,=20because=0A+=09=09=09=20*=20the=20handler=20= should=20already=20have=20emitted=20the=20necessary=20error=0A+=09=09=09=20= *=20messages.=0A+=09=09=09=20*/=0A+=09=09=09failed=20=3D=20true;=0A+=09=09= =09break;=0A+=09=09}=0A+=0A+=09=09/*=0A+=09=09=20*=20Execute=20the=20= appropriate=20amcheck=20command=20for=20this=20relation=20using=20our=0A= +=09=09=20*=20slot's=20database=20connection.=20=20We=20do=20not=20wait=20= for=20the=20command=20to=0A+=09=09=20*=20complete,=20nor=20do=20we=20= perform=20any=20error=20checking,=20as=20that=20is=20done=20by=0A+=09=09=20= *=20the=20parallel=20slots=20and=20our=20handler=20callback=20functions.=0A= +=09=09=20*/=0A+=09=09if=20(rel->is_table)=0A+=09=09{=0A+=09=09=09= prepare_table_command(&sql,=20rel->reloid,=0A+=09=09=09=09=09=09=09=09=20= =20rel->datinfo->amcheck_schema);=0A+=09=09=09= ParallelSlotSetHandler(free_slot,=20verify_heapam_slot_handler,=0A+=09=09= =09=09=09=09=09=09=20=20=20sql.data);=0A+=09=09=09run_command(free_slot,=20= sql.data,=20&cparams);=0A+=09=09}=0A+=09=09else=0A+=09=09{=0A+=09=09=09= prepare_btree_command(&sql,=20rel->reloid,=0A+=09=09=09=09=09=09=09=09=20= =20rel->datinfo->amcheck_schema);=0A+=09=09=09= ParallelSlotSetHandler(free_slot,=20verify_btree_slot_handler,=20NULL);=0A= +=09=09=09run_command(free_slot,=20sql.data,=20&cparams);=0A+=09=09}=0A+=09= }=0A+=09termPQExpBuffer(&sql);=0A+=0A+=09if=20(!failed)=0A+=09{=0A+=0A+=09= =09/*=0A+=09=09=20*=20Wait=20for=20all=20slots=20to=20complete,=20or=20= for=20one=20to=20indicate=20that=20an=20error=0A+=09=09=20*=20occurred.=20= =20Like=20above,=20we=20rely=20on=20the=20handler=20emitting=20the=20= necessary=0A+=09=09=20*=20error=20messages.=0A+=09=09=20*/=0A+=09=09if=20= (sa=20&&=20!ParallelSlotsWaitCompletion(sa))=0A+=09=09=09failed=20=3D=20= true;=0A+=0A+=09=09progress_report(reltotal,=20relprogress,=20NULL,=20= true,=20true);=0A+=09}=0A+=0A+=09if=20(sa)=0A+=09{=0A+=09=09= ParallelSlotsTerminate(sa);=0A+=09=09pg_free(sa);=0A+=09}=0A+=0A+=09if=20= (failed)=0A+=09=09exit(1);=0A+=0A+=09if=20(!all_checks_pass)=0A+=09=09= exit(2);=0A+}=0A+=0A+/*=0A+=20*=20prepare_table_command=0A+=20*=0A+=20*=20= Creates=20a=20SQL=20command=20for=20running=20amcheck=20checking=20on=20= the=20given=20heap=0A+=20*=20relation.=20=20The=20command=20is=20phrased=20= as=20a=20SQL=20query,=20with=20column=20order=20and=0A+=20*=20names=20= matching=20the=20expectations=20of=20verify_heapam_slot_handler,=20which=20= will=0A+=20*=20receive=20and=20handle=20each=20row=20returned=20from=20= the=20verify_heapam()=20function.=0A+=20*=0A+=20*=20sql:=20buffer=20into=20= which=20the=20table=20checking=20command=20will=20be=20written=0A+=20*=20= reloid:=20relation=20of=20the=20table=20to=20be=20checked=0A+=20*=20= amcheck_schema:=20escaped=20and=20quoted=20name=20of=20schema=20in=20= which=20amcheck=20contrib=0A+=20*=20module=20is=20installed=0A+=20*/=0A= +static=20void=0A+prepare_table_command(PQExpBuffer=20sql,=20Oid=20= reloid,=20const=20char=20*amcheck_schema)=0A+{=0A+=09= resetPQExpBuffer(sql);=0A+=09appendPQExpBuffer(sql,=0A+=09=09=09=09=09=20= =20"SELECT=20n.nspname,=20c.relname,=20v.blkno,=20v.offnum,=20"=0A+=09=09= =09=09=09=20=20"v.attnum,=20v.msg"=0A+=09=09=09=09=09=20=20"\nFROM=20= %s.verify_heapam("=0A+=09=09=09=09=09=20=20"\nrelation=20:=3D=20%u,"=0A+=09= =09=09=09=09=20=20"\non_error_stop=20:=3D=20%s,"=0A+=09=09=09=09=09=20=20= "\ncheck_toast=20:=3D=20%s,"=0A+=09=09=09=09=09=20=20"\nskip=20:=3D=20= '%s'",=0A+=09=09=09=09=09=20=20amcheck_schema,=0A+=09=09=09=09=09=20=20= reloid,=0A+=09=09=09=09=09=20=20opts.on_error_stop=20?=20"true"=20:=20= "false",=0A+=09=09=09=09=09=20=20opts.reconcile_toast=20?=20"true"=20:=20= "false",=0A+=09=09=09=09=09=20=20opts.skip);=0A+=09if=20(opts.startblock=20= >=3D=200)=0A+=09=09appendPQExpBuffer(sql,=20",\nstartblock=20:=3D=20= %ld",=20opts.startblock);=0A+=09if=20(opts.endblock=20>=3D=200)=0A+=09=09= appendPQExpBuffer(sql,=20",\nendblock=20:=3D=20%ld",=20opts.endblock);=0A= +=09appendPQExpBuffer(sql,=20"\n)=20v,"=0A+=09=09=09=09=09=20=20= "\npg_catalog.pg_class=20c"=0A+=09=09=09=09=09=20=20"\nJOIN=20= pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=20=20"\nON=20= c.relnamespace=20=3D=20n.oid"=0A+=09=09=09=09=09=20=20"\nWHERE=20c.oid=20= =3D=20%u",=0A+=09=09=09=09=09=20=20reloid);=0A+}=0A+=0A+/*=0A+=20*=20= prepare_btree_command=0A+=20*=0A+=20*=20Creates=20a=20SQL=20command=20= for=20running=20amcheck=20checking=20on=20the=20given=20btree=20index=0A= +=20*=20relation.=20=20The=20command=20does=20not=20select=20any=20= columns,=20as=20btree=20checking=0A+=20*=20functions=20do=20not=20return=20= any,=20but=20rather=20return=20corruption=20information=20by=0A+=20*=20= raising=20errors,=20which=20verify_btree_slot_handler=20expects.=0A+=20*=0A= +=20*=20sql:=20buffer=20into=20which=20the=20table=20checking=20command=20= will=20be=20written=0A+=20*=20reloid:=20relation=20of=20the=20table=20to=20= be=20checked=0A+=20*=20amcheck_schema:=20escaped=20and=20quoted=20name=20= of=20schema=20in=20which=20amcheck=20contrib=0A+=20*=20module=20is=20= installed=0A+=20*/=0A+static=20void=0A+prepare_btree_command(PQExpBuffer=20= sql,=20Oid=20reloid,=20const=20char=20*amcheck_schema)=0A+{=0A+=09= resetPQExpBuffer(sql);=0A+=09if=20(opts.parent_check)=0A+=09=09= appendPQExpBuffer(sql,=0A+=09=09=09=09=09=09=20=20"SELECT=20= %s.bt_index_parent_check("=0A+=09=09=09=09=09=09=20=20"\nindex=20:=3D=20= '%u'::regclass,"=0A+=09=09=09=09=09=09=20=20"\nheapallindexed=20:=3D=20= %s,"=0A+=09=09=09=09=09=09=20=20"\nrootdescend=20:=3D=20%s)",=0A+=09=09=09= =09=09=09=20=20amcheck_schema,=0A+=09=09=09=09=09=09=20=20reloid,=0A+=09=09= =09=09=09=09=20=20(opts.heapallindexed=20?=20"true"=20:=20"false"),=0A+=09= =09=09=09=09=09=20=20(opts.rootdescend=20?=20"true"=20:=20"false"));=0A+=09= else=0A+=09=09appendPQExpBuffer(sql,=0A+=09=09=09=09=09=09=20=20"SELECT=20= %s.bt_index_check("=0A+=09=09=09=09=09=09=20=20"\nindex=20:=3D=20= '%u'::regclass,"=0A+=09=09=09=09=09=09=20=20"\nheapallindexed=20:=3D=20= %s)",=0A+=09=09=09=09=09=09=20=20amcheck_schema,=0A+=09=09=09=09=09=09=20= =20reloid,=0A+=09=09=09=09=09=09=20=20(opts.heapallindexed=20?=20"true"=20= :=20"false"));=0A+}=0A+=0A+/*=0A+=20*=20run_command=0A+=20*=0A+=20*=20= Sends=20a=20command=20to=20the=20server=20without=20waiting=20for=20the=20= command=20to=20complete.=0A+=20*=20Logs=20an=20error=20if=20the=20= command=20cannot=20be=20sent,=20but=20otherwise=20any=20errors=20are=0A+=20= *=20expected=20to=20be=20handled=20by=20a=20ParallelSlotHandler.=0A+=20*=0A= +=20*=20If=20reconnecting=20to=20the=20database=20is=20necessary,=20the=20= cparams=20argument=20may=20be=0A+=20*=20modified.=0A+=20*=0A+=20*=20= slot:=20slot=20with=20connection=20to=20the=20server=20we=20should=20use=20= for=20the=20command=0A+=20*=20sql:=20query=20to=20send=0A+=20*=20= cparams:=20connection=20parameters=20in=20case=20the=20slot=20needs=20to=20= be=20reconnected=0A+=20*/=0A+static=20void=0A+run_command(ParallelSlot=20= *slot,=20const=20char=20*sql,=20ConnParams=20*cparams)=0A+{=0A+=09if=20= (opts.echo)=0A+=09=09printf("%s\n",=20sql);=0A+=0A+=09if=20= (PQsendQuery(slot->connection,=20sql)=20=3D=3D=200)=0A+=09{=0A+=09=09= pg_log_error("error=20sending=20command=20to=20database=20\"%s\":=20%s",=0A= +=09=09=09=09=09=20PQdb(slot->connection),=0A+=09=09=09=09=09=20= PQerrorMessage(slot->connection));=0A+=09=09pg_log_error("command=20was:=20= %s",=20sql);=0A+=09=09exit(1);=0A+=09}=0A+}=0A+=0A+/*=0A+=20*=20= should_processing_continue=0A+=20*=0A+=20*=20Checks=20a=20query=20result=20= returned=20from=20a=20query=20(presumably=20issued=20on=20a=20slot's=0A+=20= *=20connection)=20to=20determine=20if=20parallel=20slots=20should=20= continue=20issuing=20further=0A+=20*=20commands.=0A+=20*=0A+=20*=20Note:=20= Heap=20relation=20corruption=20is=20reported=20by=20verify_heapam()=20= via=20the=20result=0A+=20*=20set,=20rather=20than=20an=20ERROR,=20but=20= running=20verify_heapam()=20on=20a=20corrupted=20table=0A+=20*=20may=20= still=20result=20in=20an=20error=20being=20returned=20from=20the=20= server=20due=20to=20missing=0A+=20*=20relation=20files,=20bad=20= checksums,=20etc.=20=20The=20btree=20corruption=20checking=20functions=0A= +=20*=20always=20use=20errors=20to=20communicate=20corruption=20= messages.=20=20We=20can't=20just=20abort=0A+=20*=20processing=20because=20= we=20got=20a=20mere=20ERROR.=0A+=20*=0A+=20*=20res:=20result=20from=20an=20= executed=20sql=20query=0A+=20*/=0A+static=20bool=0A= +should_processing_continue(PGresult=20*res)=0A+{=0A+=09const=20char=20= *severity;=0A+=0A+=09switch=20(PQresultStatus(res))=0A+=09{=0A+=09=09=09= /*=20These=20are=20expected=20and=20ok=20*/=0A+=09=09case=20= PGRES_COMMAND_OK:=0A+=09=09case=20PGRES_TUPLES_OK:=0A+=09=09case=20= PGRES_NONFATAL_ERROR:=0A+=09=09=09break;=0A+=0A+=09=09=09/*=20This=20is=20= expected=20but=20requires=20closer=20scrutiny=20*/=0A+=09=09case=20= PGRES_FATAL_ERROR:=0A+=09=09=09severity=20=3D=20PQresultErrorField(res,=20= PG_DIAG_SEVERITY_NONLOCALIZED);=0A+=09=09=09if=20(strcmp(severity,=20= "FATAL")=20=3D=3D=200)=0A+=09=09=09=09return=20false;=0A+=09=09=09if=20= (strcmp(severity,=20"PANIC")=20=3D=3D=200)=0A+=09=09=09=09return=20= false;=0A+=09=09=09break;=0A+=0A+=09=09=09/*=20These=20are=20unexpected=20= */=0A+=09=09case=20PGRES_BAD_RESPONSE:=0A+=09=09case=20= PGRES_EMPTY_QUERY:=0A+=09=09case=20PGRES_COPY_OUT:=0A+=09=09case=20= PGRES_COPY_IN:=0A+=09=09case=20PGRES_COPY_BOTH:=0A+=09=09case=20= PGRES_SINGLE_TUPLE:=0A+=09=09=09return=20false;=0A+=09}=0A+=09return=20= true;=0A+}=0A+=0A+/*=0A+=20*=20verify_heapam_slot_handler=0A+=20*=0A+=20= *=20ParallelSlotHandler=20that=20receives=20results=20from=20a=20table=20= checking=20command=0A+=20*=20created=20by=20prepare_table_command=20and=20= outputs=20the=20results=20for=20the=20user.=0A+=20*=0A+=20*=20res:=20= result=20from=20an=20executed=20sql=20query=0A+=20*=20conn:=20connection=20= on=20which=20the=20sql=20query=20was=20executed=0A+=20*=20context:=20the=20= sql=20query=20being=20handled,=20as=20a=20cstring=0A+=20*/=0A+static=20= bool=0A+verify_heapam_slot_handler(PGresult=20*res,=20PGconn=20*conn,=20= void=20*context)=0A+{=0A+=09if=20(PQresultStatus(res)=20=3D=3D=20= PGRES_TUPLES_OK)=0A+=09{=0A+=09=09int=09=09=09i;=0A+=09=09int=09=09=09= ntups=20=3D=20PQntuples(res);=0A+=0A+=09=09if=20(ntups=20>=200)=0A+=09=09= =09all_checks_pass=20=3D=20false;=0A+=0A+=09=09for=20(i=20=3D=200;=20i=20= <=20ntups;=20i++)=0A+=09=09{=0A+=09=09=09if=20(!PQgetisnull(res,=20i,=20= 4))=0A+=09=09=09=09printf("relation=20%s.%s.%s,=20block=20%s,=20offset=20= %s,=20attribute=20%s\n=20=20=20=20%s\n",=0A+=09=09=09=09=09=20=20=20= PQdb(conn),=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=200),=09/*=20= schema=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=201),=09/*=20= relname=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=202),=09/*=20= blkno=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=203),=09/*=20= offnum=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=204),=09/*=20= attnum=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=205));=09/*=20= msg=20*/=0A+=0A+=09=09=09else=20if=20(!PQgetisnull(res,=20i,=203))=0A+=09= =09=09=09printf("relation=20%s.%s.%s,=20block=20%s,=20offset=20%s\n=20=20= =20=20%s\n",=0A+=09=09=09=09=09=20=20=20PQdb(conn),=0A+=09=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=200),=09/*=20schema=20*/=0A+=09=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=201),=09/*=20relname=20*/=0A+=09=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=202),=09/*=20blkno=20*/=0A+=09=09=09=09=09=20=20= =20PQgetvalue(res,=20i,=203),=09/*=20offnum=20*/=0A+=09=09=09=09/*=20= attnum=20is=20null:=204=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20= i,=205));=09/*=20msg=20*/=0A+=0A+=09=09=09else=20if=20(!PQgetisnull(res,=20= i,=202))=0A+=09=09=09=09printf("relation=20%s.%s.%s,=20block=20%s\n=20=20= =20=20%s\n",=0A+=09=09=09=09=09=20=20=20PQdb(conn),=0A+=09=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=200),=09/*=20schema=20*/=0A+=09=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=201),=09/*=20relname=20*/=0A+=09=09=09=09=09=20= =20=20PQgetvalue(res,=20i,=202),=09/*=20blkno=20*/=0A+=09=09=09=09/*=20= offnum=20is=20null:=203=20*/=0A+=09=09=09=09/*=20attnum=20is=20null:=204=20= */=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=205));=09/*=20msg=20= */=0A+=0A+=09=09=09else=20if=20(!PQgetisnull(res,=20i,=201))=0A+=09=09=09= =09printf("relation=20%s.%s.%s\n=20=20=20=20%s\n",=0A+=09=09=09=09=09=20=20= =20PQdb(conn),=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=200),=09= /*=20schema=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=201),=09= /*=20relname=20*/=0A+=09=09=09=09/*=20blkno=20is=20null:=20=202=20*/=0A+=09= =09=09=09/*=20offnum=20is=20null:=203=20*/=0A+=09=09=09=09/*=20attnum=20= is=20null:=204=20*/=0A+=09=09=09=09=09=20=20=20PQgetvalue(res,=20i,=20= 5));=09/*=20msg=20*/=0A+=0A+=09=09=09else=0A+=09=09=09=09= printf("%s.%s\n",=0A+=09=09=09=09=09=20=20=20PQdb(conn),=0A+=09=09=09=09=09= =20=20=20PQgetvalue(res,=20i,=205));=09/*=20msg=20*/=0A+=09=09}=0A+=09}=0A= +=09else=20if=20(PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09{=0A= +=09=09all_checks_pass=20=3D=20false;=0A+=09=09printf("%s:=20%s\n",=20= PQdb(conn),=20PQerrorMessage(conn));=0A+=09=09printf("%s:=20query=20was:=20= %s\n",=20PQdb(conn),=20(const=20char=20*)=20context);=0A+=09}=0A+=0A+=09= return=20should_processing_continue(res);=0A+}=0A+=0A+/*=0A+=20*=20= verify_btree_slot_handler=0A+=20*=0A+=20*=20ParallelSlotHandler=20that=20= receives=20results=20from=20a=20btree=20checking=20command=0A+=20*=20= created=20by=20prepare_btree_command=20and=20outputs=20them=20for=20the=20= user.=20=20The=20results=0A+=20*=20from=20the=20btree=20checking=20= command=20is=20assumed=20to=20be=20empty,=20but=20when=20the=20results=0A= +=20*=20are=20an=20error=20code,=20the=20useful=20information=20about=20= the=20corruption=20is=20expected=0A+=20*=20in=20the=20connection's=20= error=20message.=0A+=20*=0A+=20*=20res:=20result=20from=20an=20executed=20= sql=20query=0A+=20*=20conn:=20connection=20on=20which=20the=20sql=20= query=20was=20executed=0A+=20*=20context:=20unused=0A+=20*/=0A+static=20= bool=0A+verify_btree_slot_handler(PGresult=20*res,=20PGconn=20*conn,=20= void=20*context)=0A+{=0A+=09if=20(PQresultStatus(res)=20=3D=3D=20= PGRES_TUPLES_OK)=0A+=09{=0A+=09=09int=09=09=09ntups=20=3D=20= PQntuples(res);=0A+=0A+=09=09if=20(ntups=20!=3D=201)=0A+=09=09{=0A+=09=09= =09/*=0A+=09=09=09=20*=20We=20expect=20the=20btree=20checking=20= functions=20to=20return=20one=20void=20row=0A+=09=09=09=20*=20each,=20so=20= we=20should=20output=20some=20sort=20of=20warning=20if=20we=20get=0A+=09=09= =09=20*=20anything=20else,=20not=20because=20it=20indicates=20= corruption,=20but=20because=0A+=09=09=09=20*=20it=20suggests=20a=20= mismatch=20between=20amcheck=20and=20pg_amcheck=20versions.=0A+=09=09=09=20= *=0A+=09=09=09=20*=20In=20conjunction=20with=20--progress,=20anything=20= written=20to=20stderr=20at=0A+=09=09=09=20*=20this=20time=20would=20= present=20strangely=20to=20the=20user=20without=20an=20extra=0A+=09=09=09= =20*=20newline,=20so=20we=20print=20one.=20=20If=20we=20were=20= multithreaded,=20we'd=20have=0A+=09=09=09=20*=20to=20avoid=20splitting=20= this=20across=20multiple=20calls,=20but=20we're=20in=20an=0A+=09=09=09=20= *=20event=20loop,=20so=20it=20doesn't=20matter.=0A+=09=09=09=20*/=0A+=09=09= =09if=20(opts.show_progress)=0A+=09=09=09=09fprintf(stderr,=20"\n");=0A+=09= =09=09fprintf(stderr,=20"%s:=20btree=20checking=20function=20returned=20= unexpected=20number=20of=20rows:=20%d\n",=0A+=09=09=09=09=09progname,=20= ntups);=0A+=09=09=09fprintf(stderr,=20"%s:=20are=20%s's=20and=20= amcheck's=20versions=20compatible?\n",=0A+=09=09=09=09=09progname,=20= progname);=0A+=09=09}=0A+=09}=0A+=09else=0A+=09{=0A+=09=09= all_checks_pass=20=3D=20false;=0A+=09=09printf("%s:=20%s\n",=20= PQdb(conn),=20PQerrorMessage(conn));=0A+=09}=0A+=0A+=09return=20= should_processing_continue(res);=0A+}=0A+=0A+/*=0A+=20*=20help=0A+=20*=0A= +=20*=20Prints=20help=20page=20for=20the=20program=0A+=20*=0A+=20*=20= progname:=20the=20name=20of=20the=20executed=20program,=20such=20as=20= "pg_amcheck"=0A+=20*/=0A+static=20void=0A+help(const=20char=20*progname)=0A= +{=0A+=09printf("%s=20checks=20objects=20in=20a=20PostgreSQL=20database=20= for=20corruption.\n\n",=20progname);=0A+=09printf("Usage:\n");=0A+=09= printf("=20=20%s=20[OPTION]...=20[DBNAME]\n",=20progname);=0A+=09= printf("\nTarget=20Options:\n");=0A+=09printf("=20=20-a,=20--all=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20check=20all=20databases\n");=0A= +=09printf("=20=20-d,=20--dbname=3DDBNAME=20=20=20=20=20=20=20check=20= specific=20database(s)\n");=0A+=09printf("=20=20-D,=20= --exclude-dbname=3DDBNAME=20do=20NOT=20check=20specific=20= database(s)\n");=0A+=09printf("=20=20-i,=20--index=3DINDEX=20=20=20=20=20= =20=20=20=20check=20specific=20index(es)\n");=0A+=09printf("=20=20-I,=20= --exclude-index=3DINDEX=20do=20NOT=20check=20specific=20index(es)\n");=0A= +=09printf("=20=20-r,=20--relation=3DRELNAME=20=20=20=20check=20specific=20= relation(s)\n");=0A+=09printf("=20=20-R,=20--exclude-relation=3DRELNAME=20= do=20NOT=20check=20specific=20relation(s)\n");=0A+=09printf("=20=20-s,=20= --schema=3DSCHEMA=20=20=20=20=20=20=20check=20specific=20schema(s)\n");=0A= +=09printf("=20=20-S,=20--exclude-schema=3DSCHEMA=20do=20NOT=20check=20= specific=20schema(s)\n");=0A+=09printf("=20=20-t,=20--table=3DTABLE=20=20= =20=20=20=20=20=20=20check=20specific=20table(s)\n");=0A+=09printf("=20=20= -T,=20--exclude-table=3DTABLE=20do=20NOT=20check=20specific=20= table(s)\n");=0A+=09printf("=20=20=20=20=20=20--no-index-expansion=20=20= do=20NOT=20expand=20list=20of=20relations=20to=20include=20indexes\n");=0A= +=09printf("=20=20=20=20=20=20--no-toast-expansion=20=20do=20NOT=20= expand=20list=20of=20relations=20to=20include=20toast\n");=0A+=09= printf("=20=20=20=20=20=20--no-strict-names=20=20=20=20=20do=20NOT=20= require=20patterns=20to=20match=20objects\n");=0A+=09printf("\nTable=20= Checking=20Options:\n");=0A+=09printf("=20=20=20=20=20=20= --exclude-toast-pointers=20do=20NOT=20follow=20relation=20toast=20= pointers\n");=0A+=09printf("=20=20=20=20=20=20--on-error-stop=20=20=20=20= =20=20=20stop=20checking=20at=20end=20of=20first=20corrupt=20page\n");=0A= +=09printf("=20=20=20=20=20=20--skip=3DOPTION=20=20=20=20=20=20=20=20=20= do=20NOT=20check=20\"all-frozen\"=20or=20\"all-visible\"=20blocks\n");=0A= +=09printf("=20=20=20=20=20=20--startblock=3DBLOCK=20=20=20=20begin=20= checking=20table(s)=20at=20the=20given=20block=20number\n");=0A+=09= printf("=20=20=20=20=20=20--endblock=3DBLOCK=20=20=20=20=20=20check=20= table(s)=20only=20up=20to=20the=20given=20block=20number\n");=0A+=09= printf("\nBtree=20Index=20Checking=20Options:\n");=0A+=09printf("=20=20=20= =20=20=20--heapallindexed=20=20=20=20=20=20check=20all=20heap=20tuples=20= are=20found=20within=20indexes\n");=0A+=09printf("=20=20=20=20=20=20= --parent-check=20=20=20=20=20=20=20=20check=20index=20parent/child=20= relationships\n");=0A+=09printf("=20=20=20=20=20=20--rootdescend=20=20=20= =20=20=20=20=20=20search=20from=20root=20page=20to=20refind=20= tuples\n");=0A+=09printf("\nConnection=20options:\n");=0A+=09printf("=20=20= -h,=20--host=3DHOSTNAME=20=20=20=20=20=20=20database=20server=20host=20= or=20socket=20directory\n");=0A+=09printf("=20=20-p,=20--port=3DPORT=20=20= =20=20=20=20=20=20=20=20=20database=20server=20port\n");=0A+=09printf("=20= =20-U,=20--username=3DUSERNAME=20=20=20user=20name=20to=20connect=20= as\n");=0A+=09printf("=20=20-w,=20--no-password=20=20=20=20=20=20=20=20=20= never=20prompt=20for=20password\n");=0A+=09printf("=20=20-W,=20= --password=20=20=20=20=20=20=20=20=20=20=20=20force=20password=20= prompt\n");=0A+=09printf("=20=20--maintenance-db=3DDBNAME=20=20=20= alternate=20maintenance=20database\n");=0A+=09printf("\nOther=20= Options:\n");=0A+=09printf("=20=20-e,=20--echo=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20show=20the=20commands=20being=20sent=20to=20the=20= server\n");=0A+=09printf("=20=20-j,=20--jobs=3DNUM=20=20=20=20=20=20=20=20= =20=20=20=20use=20this=20many=20concurrent=20connections=20to=20the=20= server\n");=0A+=09printf("=20=20-q,=20--quiet=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20don't=20write=20any=20messages\n");=0A+=09printf("=20=20= -v,=20--verbose=20=20=20=20=20=20=20=20=20=20=20=20=20write=20a=20lot=20= of=20output\n");=0A+=09printf("=20=20-V,=20--version=20=20=20=20=20=20=20= =20=20=20=20=20=20output=20version=20information,=20then=20exit\n");=0A+=09= printf("=20=20=20=20=20=20--progress=20=20=20=20=20=20=20=20=20=20=20=20= show=20progress=20information\n");=0A+=09printf("=20=20-?,=20--help=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20show=20this=20help,=20then=20= exit\n");=0A+=0A+=09printf("\nRead=20the=20description=20of=20the=20= amcheck=20contrib=20module=20for=20details.\n");=0A+=09printf("\nReport=20= bugs=20to=20<%s>.\n",=20PACKAGE_BUGREPORT);=0A+=09printf("%s=20home=20= page:=20<%s>\n",=20PACKAGE_NAME,=20PACKAGE_URL);=0A+}=0A+=0A+/*=0A+=20*=20= Print=20a=20progress=20report=20based=20on=20the=20global=20variables.=20= If=20verbose=20output=0A+=20*=20is=20enabled,=20also=20print=20the=20= current=20file=20name.=0A+=20*=0A+=20*=20Progress=20report=20is=20= written=20at=20maximum=20once=20per=20second,=20unless=20the=20force=0A+=20= *=20parameter=20is=20set=20to=20true.=0A+=20*=0A+=20*=20If=20finished=20= is=20set=20to=20true,=20this=20is=20the=20last=20progress=20report.=20= The=20cursor=0A+=20*=20is=20moved=20to=20the=20next=20line.=0A+=20*/=0A= +static=20void=0A+progress_report(uint64=20relations_total,=20uint64=20= relations_checked,=0A+=09=09=09=09const=20char=20*datname,=20bool=20= force,=20bool=20finished)=0A+{=0A+=09int=09=09=09percent=20=3D=200;=0A+=09= char=09=09checked_str[32];=0A+=09char=09=09total_str[32];=0A+=09= pg_time_t=09now;=0A+=0A+=09if=20(!opts.show_progress)=0A+=09=09return;=0A= +=0A+=09now=20=3D=20time(NULL);=0A+=09if=20(now=20=3D=3D=20= last_progress_report=20&&=20!force=20&&=20!finished)=0A+=09=09return;=09=09= =09=09=09/*=20Max=20once=20per=20second=20*/=0A+=0A+=09= last_progress_report=20=3D=20now;=0A+=09if=20(relations_total)=0A+=09=09= percent=20=3D=20(int)=20(relations_checked=20*=20100=20/=20= relations_total);=0A+=0A+=09/*=0A+=09=20*=20Separate=20step=20to=20keep=20= platform-dependent=20format=20code=20out=20of=20fprintf=0A+=09=20*=20= calls.=20=20We=20only=20test=20for=20INT64_FORMAT=20availability=20in=20= snprintf,=20not=0A+=09=20*=20fprintf.=0A+=09=20*/=0A+=09= snprintf(checked_str,=20sizeof(checked_str),=20INT64_FORMAT,=20= relations_checked);=0A+=09snprintf(total_str,=20sizeof(total_str),=20= INT64_FORMAT,=20relations_total);=0A+=0A+#define=20= VERBOSE_DATNAME_LENGTH=2035=0A+=09if=20(opts.verbose)=0A+=09{=0A+=09=09= if=20(!datname)=0A+=0A+=09=09=09/*=0A+=09=09=09=20*=20No=20datname=20= given,=20so=20clear=20the=20status=20line=20(used=20for=20first=20and=0A= +=09=09=09=20*=20last=20call)=0A+=09=09=09=20*/=0A+=09=09=09= fprintf(stderr,=0A+=09=09=09=09=09"%*s/%s=20(%d%%)=20%*s",=0A+=09=09=09=09= =09(int)=20strlen(total_str),=0A+=09=09=09=09=09checked_str,=20= total_str,=20percent,=0A+=09=09=09=09=09VERBOSE_DATNAME_LENGTH=20+=202,=20= "");=0A+=09=09else=0A+=09=09{=0A+=09=09=09bool=09=09truncate=20=3D=20= (strlen(datname)=20>=20VERBOSE_DATNAME_LENGTH);=0A+=0A+=09=09=09= fprintf(stderr,=0A+=09=09=09=09=09"%*s/%s=20(%d%%),=20(%s%-*.*s)",=0A+=09= =09=09=09=09(int)=20strlen(total_str),=0A+=09=09=09=09=09checked_str,=20= total_str,=20percent,=0A+=09=09=09/*=20Prefix=20with=20"..."=20if=20we=20= do=20leading=20truncation=20*/=0A+=09=09=09=09=09truncate=20?=20"..."=20= :=20"",=0A+=09=09=09=09=09truncate=20?=20VERBOSE_DATNAME_LENGTH=20-=203=20= :=20VERBOSE_DATNAME_LENGTH,=0A+=09=09=09=09=09truncate=20?=20= VERBOSE_DATNAME_LENGTH=20-=203=20:=20VERBOSE_DATNAME_LENGTH,=0A+=09=09=09= /*=20Truncate=20datname=20at=20beginning=20if=20it's=20too=20long=20*/=0A= +=09=09=09=09=09truncate=20?=20datname=20+=20strlen(datname)=20-=20= VERBOSE_DATNAME_LENGTH=20+=203=20:=20datname);=0A+=09=09}=0A+=09}=0A+=09= else=0A+=09=09fprintf(stderr,=0A+=09=09=09=09"%*s/%s=20(%d%%)",=0A+=09=09= =09=09(int)=20strlen(total_str),=0A+=09=09=09=09checked_str,=20= total_str,=20percent);=0A+=0A+=09/*=0A+=09=20*=20Stay=20on=20the=20same=20= line=20if=20reporting=20to=20a=20terminal=20and=20we're=20not=20done=0A+=09= =20*=20yet.=0A+=09=20*/=0A+=09fputc((!finished=20&&=20= isatty(fileno(stderr)))=20?=20'\r'=20:=20'\n',=20stderr);=0A+}=0A+=0A+/*=0A= +=20*=20append_database_pattern=0A+=20*=0A+=20*=20Adds=20to=20a=20list=20= the=20given=20pattern=20interpreted=20as=20a=20database=20name=20= pattern.=0A+=20*=0A+=20*=20list:=20the=20list=20to=20be=20appended=0A+=20= *=20pattern:=20the=20database=20name=20pattern=0A+=20*=20encoding:=20= client=20encoding=20for=20parsing=20the=20pattern=0A+=20*/=0A+static=20= void=0A+append_database_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=20int=20encoding)=0A+{=0A+=09PQExpBufferData=20buf;=0A+=09= PatternInfo=20*info=20=3D=20(PatternInfo=20*)=20= palloc0(sizeof(PatternInfo));=0A+=0A+=09info->pattern_id=20=3D=20= next_id++;=0A+=0A+=09initPQExpBuffer(&buf);=0A+=09= patternToSQLRegex(encoding,=20NULL,=20NULL,=20&buf,=20pattern,=20false);=0A= +=09info->pattern=20=3D=20pattern;=0A+=09info->db_regex=20=3D=20= pstrdup(buf.data);=0A+=0A+=09termPQExpBuffer(&buf);=0A+=0A+=09= simple_ptr_list_append(list,=20info);=0A+}=0A+=0A+/*=0A+=20*=20= append_schema_pattern=0A+=20*=0A+=20*=20Adds=20to=20a=20list=20the=20= given=20pattern=20interpreted=20as=20a=20schema=20name=20pattern.=0A+=20= *=0A+=20*=20list:=20the=20list=20to=20be=20appended=0A+=20*=20pattern:=20= the=20schema=20name=20pattern=0A+=20*=20encoding:=20client=20encoding=20= for=20parsing=20the=20pattern=0A+=20*/=0A+static=20void=0A= +append_schema_pattern(SimplePtrList=20*list,=20const=20char=20*pattern,=20= int=20encoding)=0A+{=0A+=09PQExpBufferData=20buf;=0A+=09PatternInfo=20= *info=20=3D=20(PatternInfo=20*)=20palloc0(sizeof(PatternInfo));=0A+=0A+=09= info->pattern_id=20=3D=20next_id++;=0A+=0A+=09initPQExpBuffer(&buf);=0A+=09= patternToSQLRegex(encoding,=20NULL,=20NULL,=20&buf,=20pattern,=20false);=0A= +=09info->pattern=20=3D=20pattern;=0A+=09info->nsp_regex=20=3D=20= pstrdup(buf.data);=0A+=09termPQExpBuffer(&buf);=0A+=0A+=09= simple_ptr_list_append(list,=20info);=0A+}=0A+=0A+/*=0A+=20*=20= append_relation_pattern_helper=0A+=20*=0A+=20*=20Adds=20to=20a=20list=20= the=20given=20pattern=20interpreted=20as=20a=20relation=20pattern.=0A+=20= *=0A+=20*=20list:=20the=20list=20to=20be=20appended=0A+=20*=20pattern:=20= the=20relation=20name=20pattern=0A+=20*=20encoding:=20client=20encoding=20= for=20parsing=20the=20pattern=0A+=20*=20table_only:=20whether=20the=20= pattern=20should=20only=20be=20matched=20against=20heap=20tables=0A+=20*=20= index_only:=20whether=20the=20pattern=20should=20only=20be=20matched=20= against=20btree=20indexes=0A+=20*/=0A+static=20void=0A= +append_relation_pattern_helper(SimplePtrList=20*list,=20const=20char=20= *pattern,=0A+=09=09=09=09=09=09=09=20=20=20int=20encoding,=20bool=20= table_only,=20bool=20index_only)=0A+{=0A+=09PQExpBufferData=20dbbuf;=0A+=09= PQExpBufferData=20nspbuf;=0A+=09PQExpBufferData=20relbuf;=0A+=09= PatternInfo=20*info=20=3D=20(PatternInfo=20*)=20= palloc0(sizeof(PatternInfo));=0A+=0A+=09info->pattern_id=20=3D=20= next_id++;=0A+=0A+=09initPQExpBuffer(&dbbuf);=0A+=09= initPQExpBuffer(&nspbuf);=0A+=09initPQExpBuffer(&relbuf);=0A+=0A+=09= patternToSQLRegex(encoding,=20&dbbuf,=20&nspbuf,=20&relbuf,=20pattern,=20= false);=0A+=09info->pattern=20=3D=20pattern;=0A+=09if=20(dbbuf.data[0])=0A= +=09=09info->db_regex=20=3D=20pstrdup(dbbuf.data);=0A+=09if=20= (nspbuf.data[0])=0A+=09=09info->nsp_regex=20=3D=20pstrdup(nspbuf.data);=0A= +=09if=20(relbuf.data[0])=0A+=09=09info->rel_regex=20=3D=20= pstrdup(relbuf.data);=0A+=0A+=09termPQExpBuffer(&dbbuf);=0A+=09= termPQExpBuffer(&nspbuf);=0A+=09termPQExpBuffer(&relbuf);=0A+=0A+=09= info->table_only=20=3D=20table_only;=0A+=09info->index_only=20=3D=20= index_only;=0A+=0A+=09simple_ptr_list_append(list,=20info);=0A+}=0A+=0A= +/*=0A+=20*=20append_relation_pattern=0A+=20*=0A+=20*=20Adds=20to=20a=20= list=20the=20given=20pattern=20interpreted=20as=20a=20relation=20= pattern,=20to=20be=0A+=20*=20matched=20against=20both=20tables=20and=20= indexes.=0A+=20*=0A+=20*=20list:=20the=20list=20to=20be=20appended=0A+=20= *=20pattern:=20the=20relation=20name=20pattern=0A+=20*=20encoding:=20= client=20encoding=20for=20parsing=20the=20pattern=0A+=20*/=0A+static=20= void=0A+append_relation_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=20int=20encoding)=0A+{=0A+=09= append_relation_pattern_helper(list,=20pattern,=20encoding,=20false,=20= false);=0A+}=0A+=0A+/*=0A+=20*=20append_table_pattern=0A+=20*=0A+=20*=20= Adds=20to=20a=20list=20the=20given=20pattern=20interpreted=20as=20a=20= relation=20pattern,=20to=20be=0A+=20*=20matched=20only=20against=20= tables.=0A+=20*=0A+=20*=20list:=20the=20list=20to=20be=20appended=0A+=20= *=20pattern:=20the=20relation=20name=20pattern=0A+=20*=20encoding:=20= client=20encoding=20for=20parsing=20the=20pattern=0A+=20*/=0A+static=20= void=0A+append_table_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=20int=20encoding)=0A+{=0A+=09= append_relation_pattern_helper(list,=20pattern,=20encoding,=20true,=20= false);=0A+}=0A+=0A+/*=0A+=20*=20append_index_pattern=0A+=20*=0A+=20*=20= Adds=20to=20a=20list=20the=20given=20pattern=20interpreted=20as=20a=20= relation=20pattern,=20to=20be=0A+=20*=20matched=20only=20against=20= indexes.=0A+=20*=0A+=20*=20list:=20the=20list=20to=20be=20appended=0A+=20= *=20pattern:=20the=20relation=20name=20pattern=0A+=20*=20encoding:=20= client=20encoding=20for=20parsing=20the=20pattern=0A+=20*/=0A+static=20= void=0A+append_index_pattern(SimplePtrList=20*list,=20const=20char=20= *pattern,=20int=20encoding)=0A+{=0A+=09= append_relation_pattern_helper(list,=20pattern,=20encoding,=20false,=20= true);=0A+}=0A+=0A+/*=0A+=20*=20append_db_pattern_cte=0A+=20*=0A+=20*=20= Appends=20to=20the=20buffer=20the=20body=20of=20a=20Common=20Table=20= Expression=20(CTE)=20containing=0A+=20*=20the=20database=20portions=20= filtered=20from=20the=20list=20of=20patterns=20expressed=20as=20three=0A= +=20*=20columns:=0A+=20*=0A+=20*=20=20=20=20=20id:=20the=20unique=20= pattern=20ID=0A+=20*=20=20=20=20=20pat:=20the=20full=20user=20specified=20= pattern=20from=20the=20command=20line=0A+=20*=20=20=20=20=20rgx:=20the=20= database=20regular=20expression=20parsed=20from=20the=20pattern=0A+=20*=0A= +=20*=20Patterns=20without=20a=20database=20portion=20are=20skipped.=20=20= Patterns=20with=20more=20than=0A+=20*=20just=20a=20database=20portion=20= are=20optionally=20skipped,=20depending=20on=20argument=0A+=20*=20= 'inclusive'.=0A+=20*=0A+=20*=20buf:=20the=20buffer=20to=20be=20appended=0A= +=20*=20patterns:=20the=20list=20of=20patterns=20to=20be=20inserted=20= into=20the=20CTE=0A+=20*=20conn:=20the=20database=20connection=0A+=20*=20= inclusive:=20whether=20to=20include=20patterns=20with=20schema=20and/or=20= relation=20parts=0A+=20*/=0A+static=20void=0A= +append_db_pattern_cte(PQExpBuffer=20buf,=20const=20SimplePtrList=20= *patterns,=0A+=09=09=09=09=09=20=20PGconn=20*conn,=20bool=20inclusive)=0A= +{=0A+=09SimplePtrListCell=20*cell;=0A+=09const=20char=20*comma;=0A+=09= bool=09=09have_values;=0A+=0A+=09comma=20=3D=20"";=0A+=09have_values=20=3D= =20false;=0A+=09for=20(cell=20=3D=20patterns->head;=20cell;=20cell=20=3D=20= cell->next)=0A+=09{=0A+=09=09PatternInfo=20*info=20=3D=20(PatternInfo=20= *)=20cell->ptr;=0A+=0A+=09=09if=20(info->db_regex=20!=3D=20NULL=20&&=0A+=09= =09=09(inclusive=20||=20(info->nsp_regex=20=3D=3D=20NULL=20&&=20= info->rel_regex=20=3D=3D=20NULL)))=0A+=09=09{=0A+=09=09=09if=20= (!have_values)=0A+=09=09=09=09appendPQExpBufferStr(buf,=20"\nVALUES");=0A= +=09=09=09have_values=20=3D=20true;=0A+=09=09=09appendPQExpBuffer(buf,=20= "%s\n(%d,=20",=20comma,=20info->pattern_id);=0A+=09=09=09= appendStringLiteralConn(buf,=20info->pattern,=20conn);=0A+=09=09=09= appendPQExpBufferStr(buf,=20",=20");=0A+=09=09=09= appendStringLiteralConn(buf,=20info->db_regex,=20conn);=0A+=09=09=09= appendPQExpBufferStr(buf,=20")");=0A+=09=09=09comma=20=3D=20",";=0A+=09=09= }=0A+=09}=0A+=0A+=09if=20(!have_values)=0A+=09=09= appendPQExpBufferStr(buf,=20"\nSELECT=20NULL,=20NULL,=20NULL=20WHERE=20= false");=0A+}=0A+=0A+/*=0A+=20*=20compile_database_list=0A+=20*=0A+=20*=20= Compiles=20a=20list=20of=20databases=20to=20check=20based=20on=20the=20= user=20supplied=20options,=0A+=20*=20sorted=20to=20preserve=20the=20= order=20they=20were=20specified=20on=20the=20command=20line.=20=20In=0A+=20= *=20the=20event=20that=20multiple=20databases=20match=20a=20single=20= command=20line=20pattern,=20they=0A+=20*=20are=20secondarily=20sorted=20= by=20name.=0A+=20*=0A+=20*=20conn:=20connection=20to=20the=20initial=20= database=0A+=20*=20databases:=20the=20list=20onto=20which=20databases=20= should=20be=20appended=0A+=20*/=0A+static=20void=0A= +compile_database_list(PGconn=20*conn,=20SimplePtrList=20*databases)=0A= +{=0A+=09PGresult=20=20=20*res;=0A+=09PQExpBufferData=20sql;=0A+=09int=09= =09=09ntups;=0A+=09int=09=09=09i;=0A+=09bool=09=09fatal;=0A+=0A+=09= initPQExpBuffer(&sql);=0A+=0A+=09/*=20Append=20the=20include=20patterns=20= CTE.=20*/=0A+=09appendPQExpBufferStr(&sql,=20"WITH=20include_raw=20(id,=20= pat,=20rgx)=20AS=20(");=0A+=09append_db_pattern_cte(&sql,=20= &opts.include,=20conn,=20true);=0A+=0A+=09/*=20Append=20the=20exclude=20= patterns=20CTE.=20*/=0A+=09appendPQExpBufferStr(&sql,=20= "\n),\nexclude_raw=20(id,=20pat,=20rgx)=20AS=20(");=0A+=09= append_db_pattern_cte(&sql,=20&opts.exclude,=20conn,=20false);=0A+=09= appendPQExpBufferStr(&sql,=20"\n),");=0A+=0A+=09/*=0A+=09=20*=20Append=20= the=20database=20CTE,=20which=20includes=20whether=20each=20database=20= is=0A+=09=20*=20connectable=20and=20also=20joins=20against=20exclude_raw=20= to=20determine=20whether=0A+=09=20*=20each=20database=20is=20excluded.=0A= +=09=20*/=0A+=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=20= "\ndatabase=20(datname)=20AS=20("=0A+=09=09=09=09=09=09=20"\nSELECT=20= d.datname"=0A+=09=09=09=09=09=09=20"\nFROM=20pg_catalog.pg_database=20d"=0A= +=09=09=09=09=09=09=20"\nLEFT=20OUTER=20JOIN=20exclude_raw=20e"=0A+=09=09= =09=09=09=09=20"\nON=20d.datname=20~=20e.rgx"=0A+=09=09=09=09=09=09=20= "\nWHERE=20d.datallowconn"=0A+=09=09=09=09=09=09=20"\nAND=20e.id=20IS=20= NULL"=0A+=09=09=09=09=09=09=20"\n),"=0A+=0A+=09/*=0A+=09=20*=20Append=20= the=20include_pat=20CTE,=20which=20joins=20the=20include_raw=20CTE=20= against=20the=0A+=09=20*=20databases=20CTE=20to=20determine=20if=20all=20= the=20inclusion=20patterns=20had=20matches,=0A+=09=20*=20and=20whether=20= each=20matched=20pattern=20had=20the=20misfortune=20of=20only=20matching=0A= +=09=20*=20excluded=20or=20unconnectable=20databases.=0A+=09=20*/=0A+=09=09= =09=09=09=09=20"\ninclude_pat=20(id,=20pat,=20checkable)=20AS=20("=0A+=09= =09=09=09=09=09=20"\nSELECT=20i.id,=20i.pat,"=0A+=09=09=09=09=09=09=20= "\nCOUNT(*)=20FILTER=20("=0A+=09=09=09=09=09=09=20"\nWHERE=20d=20IS=20= NOT=20NULL"=0A+=09=09=09=09=09=09=20"\n)=20AS=20checkable"=0A+=09=09=09=09= =09=09=20"\nFROM=20include_raw=20i"=0A+=09=09=09=09=09=09=20"\nLEFT=20= OUTER=20JOIN=20database=20d"=0A+=09=09=09=09=09=09=20"\nON=20d.datname=20= ~=20i.rgx"=0A+=09=09=09=09=09=09=20"\nGROUP=20BY=20i.id,=20i.pat"=0A+=09=09= =09=09=09=09=20"\n),"=0A+=0A+=09/*=0A+=09=20*=20Append=20the=20= filtered_databases=20CTE,=20which=20selects=20from=20the=20database=20= CTE=0A+=09=20*=20optionally=20joined=20against=20the=20include_raw=20CTE=20= to=20only=20select=20databases=0A+=09=20*=20that=20match=20an=20= inclusion=20pattern.=20=20This=20appears=20to=20duplicate=20what=20the=0A= +=09=20*=20include_pat=20CTE=20already=20did=20above,=20but=20here=20we=20= want=20only=20databses,=20and=0A+=09=20*=20there=20we=20wanted=20= patterns.=0A+=09=20*/=0A+=09=09=09=09=09=09=20"\nfiltered_databases=20= (datname)=20AS=20("=0A+=09=09=09=09=09=09=20"\nSELECT=20DISTINCT=20= d.datname"=0A+=09=09=09=09=09=09=20"\nFROM=20database=20d");=0A+=09if=20= (!opts.alldb)=0A+=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09= =20"\nINNER=20JOIN=20include_raw=20i"=0A+=09=09=09=09=09=09=09=20"\nON=20= d.datname=20~=20i.rgx");=0A+=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09= =09=09=20"\n)"=0A+=0A+=09/*=0A+=09=20*=20Select=20the=20checkable=20= databases=20and=20the=20unmatched=20inclusion=20patterns.=0A+=09=20*/=0A= +=09=09=09=09=09=09=20"\nSELECT=20pat,=20datname"=0A+=09=09=09=09=09=09=20= "\nFROM=20("=0A+=09=09=09=09=09=09=20"\nSELECT=20id,=20pat,=20NULL::TEXT=20= AS=20datname"=0A+=09=09=09=09=09=09=20"\nFROM=20include_pat"=0A+=09=09=09= =09=09=09=20"\nWHERE=20checkable=20=3D=200"=0A+=09=09=09=09=09=09=20= "\nUNION=20ALL"=0A+=09=09=09=09=09=09=20"\nSELECT=20NULL,=20NULL,=20= datname"=0A+=09=09=09=09=09=09=20"\nFROM=20filtered_databases"=0A+=09=09=09= =09=09=09=20"\n)=20AS=20combined_records"=0A+=09=09=09=09=09=09=20= "\nORDER=20BY=20id=20NULLS=20LAST,=20datname");=0A+=0A+=09res=20=3D=20= executeQuery(conn,=20sql.data,=20opts.echo);=0A+=09if=20= (PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09{=0A+=09=09= pg_log_error("query=20failed:=20%s",=20PQerrorMessage(conn));=0A+=09=09= pg_log_error("query=20was:=20%s",=20sql.data);=0A+=09=09= disconnectDatabase(conn);=0A+=09=09exit(1);=0A+=09}=0A+=09= termPQExpBuffer(&sql);=0A+=0A+=09ntups=20=3D=20PQntuples(res);=0A+=09for=20= (fatal=20=3D=20false,=20i=20=3D=200;=20i=20<=20ntups;=20i++)=0A+=09{=0A+=09= =09const=20char=20*pat=20=3D=20NULL;=0A+=09=09const=20char=20*datname=20= =3D=20NULL;=0A+=0A+=09=09if=20(!PQgetisnull(res,=20i,=200))=0A+=09=09=09= pat=20=3D=20PQgetvalue(res,=20i,=200);=0A+=09=09if=20(!PQgetisnull(res,=20= i,=201))=0A+=09=09=09datname=20=3D=20PQgetvalue(res,=20i,=201);=0A+=0A+=09= =09if=20(pat=20!=3D=20NULL)=0A+=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20= Current=20record=20pertains=20to=20an=20inclusion=20pattern=20that=20= matched=20no=0A+=09=09=09=20*=20checkable=20databases.=0A+=09=09=09=20*/=0A= +=09=09=09fatal=20=3D=20opts.strict_names;=0A+=09=09=09fprintf(stderr,=20= "%s:=20no=20checkable=20database:=20\"%s\"\n",=0A+=09=09=09=09=09= progname,=20pat);=0A+=09=09}=0A+=09=09else=0A+=09=09{=0A+=09=09=09/*=20= Current=20record=20pertains=20to=20a=20database=20*/=0A+=09=09=09= Assert(datname=20!=3D=20NULL);=0A+=0A+=09=09=09DatabaseInfo=20*dat=20=3D=20= (DatabaseInfo=20*)=20palloc0(sizeof(DatabaseInfo));=0A+=0A+=09=09=09/*=20= This=20database=20is=20included.=20=20Add=20to=20list=20*/=0A+=09=09=09= if=20(opts.verbose)=0A+=09=09=09=09fprintf(stderr,=20"%s:=20including=20= database:=20\"%s\"\n",=20progname,=0A+=09=09=09=09=09=09datname);=0A+=0A= +=09=09=09dat->datname=20=3D=20pstrdup(datname);=0A+=09=09=09= simple_ptr_list_append(databases,=20dat);=0A+=09=09}=0A+=09}=0A+=09= PQclear(res);=0A+=0A+=09if=20(fatal)=0A+=09{=0A+=09=09= disconnectDatabase(conn);=0A+=09=09exit(1);=0A+=09}=0A+}=0A+=0A+/*=0A+=20= *=20append_rel_pattern_raw_cte=0A+=20*=0A+=20*=20Appends=20to=20the=20= buffer=20the=20body=20of=20a=20Common=20Table=20Expression=20(CTE)=20= containing=0A+=20*=20the=20patterns=20from=20the=20given=20list=20as=20= seven=20columns:=0A+=20*=0A+=20*=20=20=20=20=20id:=20the=20unique=20= pattern=20ID=0A+=20*=20=20=20=20=20pat:=20the=20full=20user=20specified=20= pattern=20from=20the=20command=20line=0A+=20*=20=20=20=20=20db_regex:=20= the=20database=20regexp=20parsed=20from=20the=20pattern,=20or=20NULL=20= if=20the=0A+=20*=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20pattern=20= had=20no=20database=20part=0A+=20*=20=20=20=20=20nsp_regex:=20the=20= namespace=20regexp=20parsed=20from=20the=20pattern,=20or=20NULL=20if=20= the=0A+=20*=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20pattern=20had=20= no=20namespace=20part=0A+=20*=20=20=20=20=20rel_regex:=20the=20relname=20= regexp=20parsed=20from=20the=20pattern,=20or=20NULL=20if=20the=0A+=20*=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20pattern=20had=20no=20= relname=20part=0A+=20*=20=20=20=20=20table_only:=20true=20if=20the=20= pattern=20applies=20only=20to=20tables=20(not=20indexes)=0A+=20*=20=20=20= =20=20index_only:=20true=20if=20the=20pattern=20applies=20only=20to=20= indexes=20(not=20tables)=0A+=20*=0A+=20*=20buf:=20the=20buffer=20to=20be=20= appended=0A+=20*=20patterns:=20the=20list=20of=20patterns=20to=20be=20= inserted=20into=20the=20CTE=0A+=20*=20conn:=20the=20database=20= connection=0A+=20*/=0A+static=20void=0A= +append_rel_pattern_raw_cte(PQExpBuffer=20buf,=20const=20SimplePtrList=20= *patterns,=0A+=09=09=09=09=09=09=20=20=20PGconn=20*conn)=0A+{=0A+=09= SimplePtrListCell=20*cell;=0A+=09const=20char=20*comma;=0A+=09bool=09=09= have_values;=0A+=0A+=09comma=20=3D=20"";=0A+=09have_values=20=3D=20= false;=0A+=09for=20(cell=20=3D=20patterns->head;=20cell;=20cell=20=3D=20= cell->next)=0A+=09{=0A+=09=09PatternInfo=20*info=20=3D=20(PatternInfo=20= *)=20cell->ptr;=0A+=0A+=09=09if=20(!have_values)=0A+=09=09=09= appendPQExpBufferStr(buf,=20"\nVALUES");=0A+=09=09have_values=20=3D=20= true;=0A+=09=09appendPQExpBuffer(buf,=20"%s\n(%d::INTEGER,=20",=20comma,=20= info->pattern_id);=0A+=09=09appendStringLiteralConn(buf,=20= info->pattern,=20conn);=0A+=09=09appendPQExpBufferStr(buf,=20"::TEXT,=20= ");=0A+=09=09if=20(info->db_regex=20=3D=3D=20NULL)=0A+=09=09=09= appendPQExpBufferStr(buf,=20"NULL");=0A+=09=09else=0A+=09=09=09= appendStringLiteralConn(buf,=20info->db_regex,=20conn);=0A+=09=09= appendPQExpBufferStr(buf,=20"::TEXT,=20");=0A+=09=09if=20= (info->nsp_regex=20=3D=3D=20NULL)=0A+=09=09=09appendPQExpBufferStr(buf,=20= "NULL");=0A+=09=09else=0A+=09=09=09appendStringLiteralConn(buf,=20= info->nsp_regex,=20conn);=0A+=09=09appendPQExpBufferStr(buf,=20"::TEXT,=20= ");=0A+=09=09if=20(info->rel_regex=20=3D=3D=20NULL)=0A+=09=09=09= appendPQExpBufferStr(buf,=20"NULL");=0A+=09=09else=0A+=09=09=09= appendStringLiteralConn(buf,=20info->rel_regex,=20conn);=0A+=09=09if=20= (info->table_only)=0A+=09=09=09appendPQExpBufferStr(buf,=20"::TEXT,=20= true::BOOLEAN");=0A+=09=09else=0A+=09=09=09appendPQExpBufferStr(buf,=20= "::TEXT,=20false::BOOLEAN");=0A+=09=09if=20(info->index_only)=0A+=09=09=09= appendPQExpBufferStr(buf,=20",=20true::BOOLEAN");=0A+=09=09else=0A+=09=09= =09appendPQExpBufferStr(buf,=20",=20false::BOOLEAN");=0A+=09=09= appendPQExpBufferStr(buf,=20")");=0A+=09=09comma=20=3D=20",";=0A+=09}=0A= +=0A+=09if=20(!have_values)=0A+=09=09appendPQExpBufferStr(buf,=0A+=09=09=09= =09=09=09=09=20"\nSELECT=20NULL::INTEGER,=20NULL::TEXT,=20NULL::TEXT,"=0A= +=09=09=09=09=09=09=09=20"\nNULL::TEXT,=20NULL::TEXT,=20NULL::BOOLEAN,"=0A= +=09=09=09=09=09=09=09=20"\nNULL::BOOLEAN"=0A+=09=09=09=09=09=09=09=20= "\nWHERE=20false");=0A+}=0A+=0A+/*=0A+=20*=20= append_rel_pattern_filtered_cte=0A+=20*=0A+=20*=20Appends=20to=20the=20= buffer=20a=20Common=20Table=20Expression=20(CTE)=20which=20selects=0A+=20= *=20all=20patterns=20from=20the=20named=20raw=20CTE,=20filtered=20by=20= database.=20=20All=20patterns=0A+=20*=20which=20have=20no=20database=20= portion=20or=20whose=20database=20portion=20matches=20our=0A+=20*=20= connection's=20database=20name=20are=20selected,=20with=20other=20= patterns=20excluded.=0A+=20*=0A+=20*=20The=20basic=20idea=20here=20is=20= that=20if=20we're=20connected=20to=20database=20"foo"=20and=20we=20have=0A= +=20*=20patterns=20"foo.bar.baz",=20"alpha.beta"=20and=20= "one.two.three",=20we=20only=20want=20to=0A+=20*=20use=20the=20first=20= two=20while=20processing=20relations=20in=20this=20database,=20as=20the=20= third=0A+=20*=20one=20is=20not=20relevant.=0A+=20*=0A+=20*=20buf:=20the=20= buffer=20to=20be=20appended=0A+=20*=20raw:=20the=20name=20of=20the=20CTE=20= to=20select=20from=0A+=20*=20filtered:=20the=20name=20of=20the=20CTE=20= to=20create=0A+=20*=20conn:=20the=20database=20connection=0A+=20*/=0A= +static=20void=0A+append_rel_pattern_filtered_cte(PQExpBuffer=20buf,=20= const=20char=20*raw,=0A+=09=09=09=09=09=09=09=09const=20char=20= *filtered,=20PGconn=20*conn)=0A+{=0A+=09appendPQExpBuffer(buf,=0A+=09=09=09= =09=09=20=20"\n%s=20(id,=20pat,=20nsp_regex,=20rel_regex,=20table_only,=20= index_only)=20AS=20("=0A+=09=09=09=09=09=20=20"\nSELECT=20id,=20pat,=20= nsp_regex,=20rel_regex,=20table_only,=20index_only"=0A+=09=09=09=09=09=20= =20"\nFROM=20%s=20r"=0A+=09=09=09=09=09=20=20"\nWHERE=20(r.db_regex=20IS=20= NULL"=0A+=09=09=09=09=09=20=20"\nOR=20",=0A+=09=09=09=09=09=20=20= filtered,=20raw);=0A+=09appendStringLiteralConn(buf,=20PQdb(conn),=20= conn);=0A+=09appendPQExpBufferStr(buf,=20"=20~=20r.db_regex)");=0A+=09= appendPQExpBufferStr(buf,=0A+=09=09=09=09=09=09=20"\nAND=20(r.nsp_regex=20= IS=20NOT=20NULL"=0A+=09=09=09=09=09=09=20"\nOR=20r.rel_regex=20IS=20NOT=20= NULL)"=0A+=09=09=09=09=09=09=20"\n),");=0A+}=0A+=0A+/*=0A+=20*=20= compile_relation_list_one_db=0A+=20*=0A+=20*=20Compiles=20a=20list=20of=20= relations=20to=20check=20within=20the=20currently=20connected=0A+=20*=20= database=20based=20on=20the=20user=20supplied=20options,=20sorted=20by=20= descending=20size,=0A+=20*=20and=20appends=20them=20to=20the=20given=20= list=20of=20relations.=0A+=20*=0A+=20*=20The=20cells=20of=20the=20= constructed=20list=20contain=20all=20information=20about=20the=20= relation=0A+=20*=20necessary=20to=20connect=20to=20the=20database=20and=20= check=20the=20object,=20including=20which=0A+=20*=20database=20to=20= connect=20to,=20where=20contrib/amcheck=20is=20installed,=20and=20the=20= Oid=20and=0A+=20*=20type=20of=20object=20(table=20vs.=20index).=20=20= Rather=20than=20duplicating=20the=20database=0A+=20*=20details=20per=20= relation,=20the=20relation=20structs=20use=20references=20to=20the=20= same=0A+=20*=20database=20object,=20provided=20by=20the=20caller.=0A+=20= *=0A+=20*=20conn:=20connection=20to=20this=20next=20database,=20which=20= should=20be=20the=20same=20as=20in=20'dat'=0A+=20*=20relations:=20list=20= onto=20which=20the=20relations=20information=20should=20be=20appended=0A= +=20*=20dat:=20the=20database=20info=20struct=20for=20use=20by=20each=20= relation=0A+=20*/=0A+static=20void=0A= +compile_relation_list_one_db(PGconn=20*conn,=20SimplePtrList=20= *relations,=0A+=09=09=09=09=09=09=09=20const=20DatabaseInfo=20*dat)=0A+{=0A= +=09PGresult=20=20=20*res;=0A+=09PQExpBufferData=20sql;=0A+=09int=09=09=09= ntups;=0A+=09int=09=09=09i;=0A+=09const=20char=20*datname;=0A+=0A+=09= initPQExpBuffer(&sql);=0A+=09appendPQExpBufferStr(&sql,=20"WITH");=0A+=0A= +=09/*=20Append=20CTEs=20for=20the=20relation=20inclusion=20patterns,=20= if=20any=20*/=0A+=09if=20(!opts.allrel)=0A+=09{=0A+=09=09= appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=20"\ninclude_raw=20= (id,=20pat,=20db_regex,=20nsp_regex,=20rel_regex,=20table_only,=20= index_only)=20AS=20(");=0A+=09=09append_rel_pattern_raw_cte(&sql,=20= &opts.include,=20conn);=0A+=09=09appendPQExpBufferStr(&sql,=20"\n),");=0A= +=09=09append_rel_pattern_filtered_cte(&sql,=20"include_raw",=20= "include_pat",=20conn);=0A+=09}=0A+=0A+=09/*=20Append=20CTEs=20for=20the=20= relation=20exclusion=20patterns,=20if=20any=20*/=0A+=09if=20= (opts.excludetbl=20||=20opts.excludeidx)=0A+=09{=0A+=09=09= appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=20"\nexclude_raw=20= (id,=20pat,=20db_regex,=20nsp_regex,=20rel_regex,=20table_only,=20= index_only)=20AS=20(");=0A+=09=09append_rel_pattern_raw_cte(&sql,=20= &opts.exclude,=20conn);=0A+=09=09appendPQExpBufferStr(&sql,=20"\n),");=0A= +=09=09append_rel_pattern_filtered_cte(&sql,=20"exclude_raw",=20= "exclude_pat",=20conn);=0A+=09}=0A+=0A+=09/*=20Append=20the=20relation=20= CTE.=20*/=0A+=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=20= "\nrelation=20(id,=20pat,=20oid,=20reltoastrelid,=20relpages,=20= is_table,=20is_index)=20AS=20("=0A+=09=09=09=09=09=09=20"\nSELECT=20= DISTINCT=20ON=20(c.oid");=0A+=09if=20(!opts.allrel)=0A+=09=09= appendPQExpBufferStr(&sql,=20",=20ip.id)=20ip.id,=20ip.pat,");=0A+=09= else=0A+=09=09appendPQExpBufferStr(&sql,=20")=20NULL::INTEGER=20AS=20id,=20= NULL::TEXT=20AS=20pat,");=0A+=09appendPQExpBuffer(&sql,=0A+=09=09=09=09=09= =20=20"\nc.oid,=20c.reltoastrelid,=20c.relpages,"=0A+=09=09=09=09=09=20=20= "\nc.relam=20=3D=20%u=20AS=20is_table,"=0A+=09=09=09=09=09=20=20= "\nc.relam=20=3D=20%u=20AS=20is_index"=0A+=09=09=09=09=09=20=20"\nFROM=20= pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=20=20"\nINNER=20JOIN=20= pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=20=20"\nON=20= c.relnamespace=20=3D=20n.oid",=0A+=09=09=09=09=09=20=20= HEAP_TABLE_AM_OID,=20BTREE_AM_OID);=0A+=09if=20(!opts.allrel)=0A+=09=09= appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=09=20=20"\nINNER=20JOIN=20= include_pat=20ip"=0A+=09=09=09=09=09=09=20=20"\nON=20(n.nspname=20~=20= ip.nsp_regex=20OR=20ip.nsp_regex=20IS=20NULL)"=0A+=09=09=09=09=09=09=20=20= "\nAND=20(c.relname=20~=20ip.rel_regex=20OR=20ip.rel_regex=20IS=20NULL)"=0A= +=09=09=09=09=09=09=20=20"\nAND=20(c.relam=20=3D=20%u=20OR=20NOT=20= ip.table_only)"=0A+=09=09=09=09=09=09=20=20"\nAND=20(c.relam=20=3D=20%u=20= OR=20NOT=20ip.index_only)",=0A+=09=09=09=09=09=09=20=20= HEAP_TABLE_AM_OID,=20BTREE_AM_OID);=0A+=09if=20(opts.excludetbl=20||=20= opts.excludeidx)=0A+=09=09appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=09=20= =20"\nLEFT=20OUTER=20JOIN=20exclude_pat=20ep"=0A+=09=09=09=09=09=09=20=20= "\nON=20(n.nspname=20~=20ep.nsp_regex=20OR=20ep.nsp_regex=20IS=20NULL)"=0A= +=09=09=09=09=09=09=20=20"\nAND=20(c.relname=20~=20ep.rel_regex=20OR=20= ep.rel_regex=20IS=20NULL)"=0A+=09=09=09=09=09=09=20=20"\nAND=20(c.relam=20= =3D=20%u=20OR=20NOT=20ep.table_only)"=0A+=09=09=09=09=09=09=20=20"\nAND=20= (c.relam=20=3D=20%u=20OR=20NOT=20ep.index_only)",=0A+=09=09=09=09=09=09=20= =20HEAP_TABLE_AM_OID,=20BTREE_AM_OID);=0A+=0A+=09if=20(opts.excludetbl=20= ||=20opts.excludeidx)=0A+=09=09appendPQExpBufferStr(&sql,=20"\nWHERE=20= ep.pat=20IS=20NULL");=0A+=09else=0A+=09=09appendPQExpBufferStr(&sql,=20= "\nWHERE=20true");=0A+=0A+=09/*=0A+=09=20*=20We=20need=20to=20be=20= careful=20not=20to=20break=20the=20--no-toast-expansion=20and=0A+=09=20*=20= --no-index-expansion=20options.=20=20By=20default,=20the=20indexes,=20= toast=20tables,=0A+=09=20*=20and=20toast=20table=20indexes=20associated=20= with=20primary=20tables=20are=20included,=0A+=09=20*=20using=20their=20= own=20CTEs=20below.=20=20We=20implement=20the=20--exclude-*=20options=20= by=0A+=09=20*=20not=20creating=20those=20CTEs,=20but=20that's=20no=20use=20= if=20we've=20already=20selected=0A+=09=20*=20the=20toast=20and=20indexes=20= here.=20=20On=20the=20other=20hand,=20we=20want=20inclusion=0A+=09=20*=20= patterns=20that=20match=20indexes=20or=20toast=20tables=20to=20be=20= honored.=20=20So,=20if=0A+=09=20*=20inclusion=20patterns=20were=20given,=20= we=20want=20to=20select=20all=20tables,=20toast=0A+=09=20*=20tables,=20= or=20indexes=20that=20match=20the=20patterns.=20=20But=20if=20no=20= inclusion=0A+=09=20*=20patterns=20were=20given,=20and=20we're=20simply=20= matching=20all=20relations,=20then=20we=0A+=09=20*=20only=20want=20to=20= match=20the=20primary=20tables=20here.=0A+=09=20*/=0A+=09if=20= (opts.allrel)=0A+=09=09appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=09=20=20= "\nAND=20c.relam=20=3D=20%u"=0A+=09=09=09=09=09=09=20=20"\nAND=20= c.relkind=20IN=20('r',=20'm',=20't')"=0A+=09=09=09=09=09=09=20=20"\nAND=20= c.relnamespace=20!=3D=20%u",=0A+=09=09=09=09=09=09=20=20= HEAP_TABLE_AM_OID,=20PG_TOAST_NAMESPACE);=0A+=09else=0A+=09=09= appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=09=20=20"\nAND=20c.relam=20IN=20= (%u,=20%u)"=0A+=09=09=09=09=09=09=20=20"\nAND=20c.relkind=20IN=20('r',=20= 'm',=20't',=20'i')"=0A+=09=09=09=09=09=09=20=20"\nAND=20((c.relam=20=3D=20= %u=20AND=20c.relkind=20IN=20('r',=20'm',=20't'))=20OR"=0A+=09=09=09=09=09= =09=20=20"\n(c.relam=20=3D=20%u=20AND=20c.relkind=20=3D=20'i'))",=0A+=09=09= =09=09=09=09=20=20HEAP_TABLE_AM_OID,=20BTREE_AM_OID,=0A+=09=09=09=09=09=09= =20=20HEAP_TABLE_AM_OID,=20BTREE_AM_OID);=0A+=0A+=09= appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=20"\nORDER=20BY=20= c.oid"=0A+=09=09=09=09=09=09=20"\n)");=0A+=0A+=09if=20= (!opts.no_toast_expansion)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20Include=20= a=20CTE=20for=20toast=20tables=20associated=20with=20primary=20tables=0A= +=09=09=20*=20selected=20above,=20filtering=20by=20exclusion=20patterns=20= (if=20any)=20that=20match=0A+=09=09=20*=20toast=20table=20names.=0A+=09=09= =20*/=0A+=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=20= ",\ntoast=20(oid,=20relpages)=20AS=20("=0A+=09=09=09=09=09=09=09=20= "\nSELECT=20t.oid,=20t.relpages"=0A+=09=09=09=09=09=09=09=20"\nFROM=20= pg_catalog.pg_class=20t"=0A+=09=09=09=09=09=09=09=20"\nINNER=20JOIN=20= relation=20r"=0A+=09=09=09=09=09=09=09=20"\nON=20r.reltoastrelid=20=3D=20= t.oid");=0A+=09=09if=20(opts.excludetbl)=0A+=09=09=09= appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=09=20"\nLEFT=20OUTER=20= JOIN=20exclude_pat=20ep"=0A+=09=09=09=09=09=09=09=09=20"\nON=20= ('pg_toast'=20~=20ep.nsp_regex=20OR=20ep.nsp_regex=20IS=20NULL)"=0A+=09=09= =09=09=09=09=09=09=20"\nAND=20(t.relname=20~=20ep.rel_regex=20OR=20= ep.rel_regex=20IS=20NULL)"=0A+=09=09=09=09=09=09=09=09=20"\nAND=20= ep.table_only"=0A+=09=09=09=09=09=09=09=09=20"\nWHERE=20ep.id=20IS=20= NULL");=0A+=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=20= "\n)");=0A+=09}=0A+=09if=20(!opts.no_index_expansion)=0A+=09{=0A+=09=09= /*=0A+=09=09=20*=20Include=20a=20CTE=20for=20btree=20indexes=20= associated=20with=20primary=20tables=0A+=09=09=20*=20selected=20above,=20= filtering=20by=20exclusion=20patterns=20(if=20any)=20that=20match=0A+=09=09= =20*=20btree=20index=20names.=0A+=09=09=20*/=0A+=09=09= appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=09=20=20",\nindex=20(oid,=20= relpages)=20AS=20("=0A+=09=09=09=09=09=09=20=20"\nSELECT=20c.oid,=20= c.relpages"=0A+=09=09=09=09=09=09=20=20"\nFROM=20relation=20r"=0A+=09=09=09= =09=09=09=20=20"\nINNER=20JOIN=20pg_catalog.pg_index=20i"=0A+=09=09=09=09= =09=09=20=20"\nON=20r.oid=20=3D=20i.indrelid"=0A+=09=09=09=09=09=09=20=20= "\nINNER=20JOIN=20pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=09=20=20= "\nON=20i.indexrelid=20=3D=20c.oid");=0A+=09=09if=20(opts.excludeidx)=0A= +=09=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=09=20= "\nINNER=20JOIN=20pg_catalog.pg_namespace=20n"=0A+=09=09=09=09=09=09=09=09= =20"\nON=20c.relnamespace=20=3D=20n.oid"=0A+=09=09=09=09=09=09=09=09=20= "\nLEFT=20OUTER=20JOIN=20exclude_pat=20ep"=0A+=09=09=09=09=09=09=09=09=20= "\nON=20(n.nspname=20~=20ep.nsp_regex=20OR=20ep.nsp_regex=20IS=20NULL)"=0A= +=09=09=09=09=09=09=09=09=20"\nAND=20(c.relname=20~=20ep.rel_regex=20OR=20= ep.rel_regex=20IS=20NULL)"=0A+=09=09=09=09=09=09=09=09=20"\nAND=20= ep.index_only"=0A+=09=09=09=09=09=09=09=09=20"\nWHERE=20ep.id=20IS=20= NULL");=0A+=09=09else=0A+=09=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09= =09=09=09=09=09=20"\nWHERE=20true");=0A+=09=09appendPQExpBuffer(&sql,=0A= +=09=09=09=09=09=09=20=20"\nAND=20c.relam=20=3D=20%u"=0A+=09=09=09=09=09=09= =20=20"\nAND=20c.relkind=20=3D=20'i'",=0A+=09=09=09=09=09=09=20=20= BTREE_AM_OID);=0A+=09=09if=20(opts.no_toast_expansion)=0A+=09=09=09= appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=09=09=20=20"\nAND=20= c.relnamespace=20!=3D=20%u",=0A+=09=09=09=09=09=09=09=20=20= PG_TOAST_NAMESPACE);=0A+=09=09appendPQExpBufferStr(&sql,=20"\n)");=0A+=09= }=0A+=0A+=09if=20(!opts.no_toast_expansion=20&&=20= !opts.no_index_expansion)=0A+=09{=0A+=09=09/*=0A+=09=09=20*=20Include=20= a=20CTE=20for=20btree=20indexes=20associated=20with=20toast=20tables=20= of=0A+=09=09=20*=20primary=20tables=20selected=20above,=20filtering=20by=20= exclusion=20patterns=20(if=0A+=09=09=20*=20any)=20that=20match=20the=20= toast=20index=20names.=0A+=09=09=20*/=0A+=09=09appendPQExpBuffer(&sql,=0A= +=09=09=09=09=09=09=20=20",\ntoast_index=20(oid,=20relpages)=20AS=20("=0A= +=09=09=09=09=09=09=20=20"\nSELECT=20c.oid,=20c.relpages"=0A+=09=09=09=09= =09=09=20=20"\nFROM=20toast=20t"=0A+=09=09=09=09=09=09=20=20"\nINNER=20= JOIN=20pg_catalog.pg_index=20i"=0A+=09=09=09=09=09=09=20=20"\nON=20t.oid=20= =3D=20i.indrelid"=0A+=09=09=09=09=09=09=20=20"\nINNER=20JOIN=20= pg_catalog.pg_class=20c"=0A+=09=09=09=09=09=09=20=20"\nON=20i.indexrelid=20= =3D=20c.oid");=0A+=09=09if=20(opts.excludeidx)=0A+=09=09=09= appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=09=20"\nLEFT=20OUTER=20= JOIN=20exclude_pat=20ep"=0A+=09=09=09=09=09=09=09=09=20"\nON=20= ('pg_toast'=20~=20ep.nsp_regex=20OR=20ep.nsp_regex=20IS=20NULL)"=0A+=09=09= =09=09=09=09=09=09=20"\nAND=20(c.relname=20~=20ep.rel_regex=20OR=20= ep.rel_regex=20IS=20NULL)"=0A+=09=09=09=09=09=09=09=09=20"\nAND=20= ep.index_only"=0A+=09=09=09=09=09=09=09=09=20"\nWHERE=20ep.id=20IS=20= NULL");=0A+=09=09else=0A+=09=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09= =09=09=09=09=09=20"\nWHERE=20true");=0A+=09=09appendPQExpBuffer(&sql,=0A= +=09=09=09=09=09=09=20=20"\nAND=20c.relam=20=3D=20%u"=0A+=09=09=09=09=09=09= =20=20"\nAND=20c.relkind=20=3D=20'i'"=0A+=09=09=09=09=09=09=20=20"\n)",=0A= +=09=09=09=09=09=09=20=20BTREE_AM_OID);=0A+=09}=0A+=0A+=09/*=0A+=09=20*=20= Roll-up=20distinct=20rows=20from=20CTEs.=0A+=09=20*=0A+=09=20*=20= Relations=20that=20match=20more=20than=20one=20pattern=20may=20occur=20= more=20than=20once=20in=0A+=09=20*=20the=20list,=20and=20indexes=20and=20= toast=20for=20primary=20relations=20may=20also=20have=0A+=09=20*=20= matched=20in=20their=20own=20right,=20so=20we=20rely=20on=20UNION=20to=20= deduplicate=20the=0A+=09=20*=20list.=0A+=09=20*/=0A+=09= appendPQExpBuffer(&sql,=0A+=09=09=09=09=09=20=20"\nSELECT=20id,=20= is_table,=20is_index,=20oid"=0A+=09=09=09=09=09=20=20"\nFROM=20(");=0A+=09= appendPQExpBufferStr(&sql,=0A+=09/*=20Inclusion=20patterns=20that=20= failed=20to=20match=20*/=0A+=09=09=09=09=09=09=20"\nSELECT=20id,=20= is_table,=20is_index,"=0A+=09=09=09=09=09=09=20"\nNULL::OID=20AS=20oid,"=0A= +=09=09=09=09=09=09=20"\nNULL::INTEGER=20AS=20relpages"=0A+=09=09=09=09=09= =09=20"\nFROM=20relation"=0A+=09=09=09=09=09=09=20"\nWHERE=20id=20IS=20= NOT=20NULL"=0A+=09=09=09=09=09=09=20"\nUNION"=0A+=09/*=20Primary=20= relations=20*/=0A+=09=09=09=09=09=09=20"\nSELECT=20NULL::INTEGER=20AS=20= id,"=0A+=09=09=09=09=09=09=20"\nis_table,=20is_index,"=0A+=09=09=09=09=09= =09=20"\noid,=20relpages"=0A+=09=09=09=09=09=09=20"\nFROM=20relation");=0A= +=09if=20(!opts.no_toast_expansion)=0A+=09=09appendPQExpBufferStr(&sql,=0A= +=09=09=09=09=09=09=09=20"\nUNION"=0A+=09=09/*=20Toast=20tables=20for=20= primary=20relations=20*/=0A+=09=09=09=09=09=09=09=20"\nSELECT=20= NULL::INTEGER=20AS=20id,=20TRUE=20AS=20is_table,"=0A+=09=09=09=09=09=09=09= =20"\nFALSE=20AS=20is_index,=20oid,=20relpages"=0A+=09=09=09=09=09=09=09=20= "\nFROM=20toast");=0A+=09if=20(!opts.no_index_expansion)=0A+=09=09= appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=09=20"\nUNION"=0A+=09=09= /*=20Indexes=20for=20primary=20relations=20*/=0A+=09=09=09=09=09=09=09=20= "\nSELECT=20NULL::INTEGER=20AS=20id,=20FALSE=20AS=20is_table,"=0A+=09=09=09= =09=09=09=09=20"\nTRUE=20AS=20is_index,=20oid,=20relpages"=0A+=09=09=09=09= =09=09=09=20"\nFROM=20index");=0A+=09if=20(!opts.no_toast_expansion=20&&=20= !opts.no_index_expansion)=0A+=09=09appendPQExpBufferStr(&sql,=0A+=09=09=09= =09=09=09=09=20"\nUNION"=0A+=09=09/*=20Indexes=20for=20toast=20relations=20= */=0A+=09=09=09=09=09=09=09=20"\nSELECT=20NULL::INTEGER=20AS=20id,=20= FALSE=20AS=20is_table,"=0A+=09=09=09=09=09=09=09=20"\nTRUE=20AS=20= is_index,=20oid,=20relpages"=0A+=09=09=09=09=09=09=09=20"\nFROM=20= toast_index");=0A+=09appendPQExpBufferStr(&sql,=0A+=09=09=09=09=09=09=20= "\n)=20AS=20combined_records"=0A+=09=09=09=09=09=09=20"\nORDER=20BY=20= relpages=20DESC=20NULLS=20FIRST,=20oid");=0A+=0A+=09res=20=3D=20= executeQuery(conn,=20sql.data,=20opts.echo);=0A+=09if=20= (PQresultStatus(res)=20!=3D=20PGRES_TUPLES_OK)=0A+=09{=0A+=09=09= pg_log_error("query=20failed:=20%s",=20PQerrorMessage(conn));=0A+=09=09= pg_log_error("query=20was:=20%s",=20sql.data);=0A+=09=09= disconnectDatabase(conn);=0A+=09=09exit(1);=0A+=09}=0A+=09= termPQExpBuffer(&sql);=0A+=0A+=09/*=0A+=09=20*=20Allocate=20a=20single=20= copy=20of=20the=20database=20name=20to=20be=20shared=20by=20all=20nodes=0A= +=09=20*=20in=20the=20object=20list,=20constructed=20below.=0A+=09=20*/=0A= +=09datname=20=3D=20pstrdup(PQdb(conn));=0A+=0A+=09ntups=20=3D=20= PQntuples(res);=0A+=09for=20(i=20=3D=200;=20i=20<=20ntups;=20i++)=0A+=09= {=0A+=09=09int=09=09=09pattern_id=20=3D=200;=0A+=09=09bool=09=09is_table=20= =3D=20false;=0A+=09=09bool=09=09is_index=20=3D=20false;=0A+=09=09Oid=09=09= =09oid=20=3D=20InvalidOid;=0A+=0A+=09=09if=20(!PQgetisnull(res,=20i,=20= 0))=0A+=09=09=09pattern_id=20=3D=20atoi(PQgetvalue(res,=20i,=200));=0A+=09= =09if=20(!PQgetisnull(res,=20i,=201))=0A+=09=09=09is_table=20=3D=20= (PQgetvalue(res,=20i,=201)[0]=20=3D=3D=20't');=0A+=09=09if=20= (!PQgetisnull(res,=20i,=202))=0A+=09=09=09is_index=20=3D=20= (PQgetvalue(res,=20i,=202)[0]=20=3D=3D=20't');=0A+=09=09if=20= (!PQgetisnull(res,=20i,=203))=0A+=09=09=09oid=20=3D=20= atooid(PQgetvalue(res,=20i,=203));=0A+=0A+=09=09if=20(pattern_id=20>=20= 0)=0A+=09=09{=0A+=09=09=09/*=0A+=09=09=09=20*=20Current=20record=20= pertains=20to=20an=20inclusion=20pattern.=20=20Find=20the=0A+=09=09=09=20= *=20pattern=20in=20the=20list=20and=20record=20that=20it=20matched.=20=20= If=20we=20expected=0A+=09=09=09=20*=20a=20large=20number=20of=20= command-line=20inclusion=20pattern=20arguments,=20the=0A+=09=09=09=20*=20= datastructure=20here=20might=20need=20to=20be=20more=20efficient,=20but=20= we=0A+=09=09=09=20*=20expect=20the=20list=20to=20be=20short.=0A+=09=09=09= =20*/=0A+=0A+=09=09=09SimplePtrListCell=20*cell;=0A+=09=09=09bool=09=09= found;=0A+=0A+=09=09=09for=20(found=20=3D=20false,=20cell=20=3D=20= opts.include.head;=20cell;=20cell=20=3D=20cell->next)=0A+=09=09=09{=0A+=09= =09=09=09PatternInfo=20*info=20=3D=20(PatternInfo=20*)=20cell->ptr;=0A+=0A= +=09=09=09=09if=20(info->pattern_id=20=3D=3D=20pattern_id)=0A+=09=09=09=09= {=0A+=09=09=09=09=09info->matched=20=3D=20true;=0A+=09=09=09=09=09found=20= =3D=20true;=0A+=09=09=09=09=09break;=0A+=09=09=09=09}=0A+=09=09=09}=0A+=09= =09=09if=20(!found)=0A+=09=09=09{=0A+=09=09=09=09pg_log_error("internal=20= error:=20received=20unexpected=20pattern_id=20%d",=0A+=09=09=09=09=09=09=09= =20pattern_id);=0A+=09=09=09=09exit(1);=0A+=09=09=09}=0A+=09=09}=0A+=09=09= else=0A+=09=09{=0A+=09=09=09/*=20Current=20record=20pertains=20to=20a=20= relation=20*/=0A+=0A+=09=09=09RelationInfo=20*rel=20=3D=20(RelationInfo=20= *)=20palloc0(sizeof(RelationInfo));=0A+=0A+=09=09=09= Assert(OidIsValid(oid));=0A+=09=09=09Assert(is_table=20^=20is_index);=0A= +=0A+=09=09=09rel->datinfo=20=3D=20dat;=0A+=09=09=09rel->reloid=20=3D=20= oid;=0A+=09=09=09rel->is_table=20=3D=20is_table;=0A+=0A+=09=09=09= simple_ptr_list_append(relations,=20rel);=0A+=09=09}=0A+=09}=0A+=09= PQclear(res);=0A+}=0Adiff=20--git=20a/contrib/pg_amcheck/t/001_basic.pl=20= b/contrib/pg_amcheck/t/001_basic.pl=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..dfa0ae9e06=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/001_basic.pl=0A@@=20-0,0=20+1,9=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20TestLib;=0A+use=20Test::More=20= tests=20=3D>=208;=0A+=0A+program_help_ok('pg_amcheck');=0A= +program_version_ok('pg_amcheck');=0A= +program_options_handling_ok('pg_amcheck');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/002_nonesuch.pl=20= b/contrib/pg_amcheck/t/002_nonesuch.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..8c6e267ee9=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/002_nonesuch.pl=0A@@=20-0,0=20+1,213=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20TestLib;=0A= +use=20Test::More=20tests=20=3D>=2060;=0A+=0A+#=20Test=20set-up=0A+my=20= ($node,=20$port);=0A+$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+$port=20=3D=20$node->port;=0A+=0A+#=20Load=20the=20= amcheck=20extension,=20upon=20which=20pg_amcheck=20depends=0A= +$node->safe_psql('postgres',=20q(CREATE=20EXTENSION=20amcheck));=0A+=0A= +#########################################=0A+#=20Test=20connecting=20to=20= a=20non-existent=20database=0A+=0A+#=20Failing=20to=20connect=20to=20the=20= initial=20database=20is=20an=20error.=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'qqq'=20],=0A+=09qr/database=20"qqq"=20= does=20not=20exist/,=0A+=09'checking=20a=20non-existent=20database');=0A= +=0A+#=20Failing=20to=20resolve=20a=20secondary=20database=20name=20is=20= also=20an=20error,=20though=20since=0A+#=20the=20string=20is=20treated=20= as=20a=20pattern,=20the=20error=20message=20looks=20different.=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'qqq'=20],=0A+=09qr/pg_amcheck:=20no=20checkable=20= database:=20"qqq"/,=0A+=09'checking=20a=20non-existent=20database');=0A+=0A= +#=20Failing=20to=20connect=20to=20the=20initial=20database=20is=20still=20= an=20error=20when=20using=0A+#=20--no-strict-names.=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'--no-strict-names',=20= '-p',=20$port,=20'qqq'=20],=0A+=09qr/database=20"qqq"=20does=20not=20= exist/,=0A+=09'checking=20a=20non-existent=20database=20with=20= --no-strict-names');=0A+=0A+#=20But=20failing=20to=20resolve=20secondary=20= database=20names=20is=20not=20an=20error=20when=20using=0A+#=20= --no-strict-names.=20=20We=20should=20still=20see=20the=20message,=20but=20= as=20a=20non-fatal=0A+#=20warning=0A+$node->command_checks_all(=0A+=09[=20= 'pg_amcheck',=20'--no-strict-names',=20'-p',=20$port,=20'-d',=20= 'no_such_database',=20'postgres',=20'qqq'=20],=0A+=090,=0A+=09[=20],=0A+=09= [=20qr/no=20checkable=20database:=20"qqq"/=20],=0A+=09'checking=20a=20= non-existent=20secondary=20database=20with=20--no-strict-names');=0A+=0A= +#=20Check=20that=20a=20substring=20of=20an=20existent=20database=20name=20= does=20not=20get=20interpreted=0A+#=20as=20a=20matching=20pattern.=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20'post'=20= ],=0A+=09qr/database=20"post"=20does=20not=20exist/,=0A+=09'checking=20a=20= non-existent=20primary=20database=20(substring=20of=20existent=20= database)');=0A+=0A+#=20And=20again,=20but=20testing=20the=20secondary=20= database=20name=20rather=20than=20the=20primary=0A+command_fails_like(=0A= +=09[=20'pg_amcheck',=20'-p',=20$port,=20'postgres',=20'post'=20],=0A+=09= qr/pg_amcheck:=20no=20checkable=20database:=20"post"/,=0A+=09'checking=20= a=20non-existent=20secondary=20database=20(substring=20of=20existent=20= database)');=0A+=0A+#=20Likewise,=20check=20that=20a=20superstring=20of=20= an=20existent=20database=20name=20does=20not=20get=0A+#=20interpreted=20= as=20a=20matching=20pattern.=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'postresql'=20],=0A+=09qr/database=20= "postresql"=20does=20not=20exist/,=0A+=09'checking=20a=20non-existent=20= primary=20database=20(superstring=20of=20existent=20database)');=0A+=0A= +#=20And=20again,=20but=20testing=20the=20secondary=20database=20name=20= rather=20than=20the=20primary=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'postgres',=20'postgresql'=20],=0A+=09= qr/pg_amcheck:=20no=20checkable=20database:=20"postgresql"/,=0A+=09= 'checking=20a=20non-existent=20secondary=20database=20(superstring=20of=20= existent=20database)');=0A+=0A+#########################################=0A= +#=20Test=20connecting=20with=20a=20non-existent=20user=0A+=0A+#=20= Failing=20to=20connect=20to=20the=20initial=20database=20due=20to=20bad=20= username=20is=20an=20error.=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'-U=3Dno_such_user',=20'postgres'=20],=0A= +=09qr/role=20"=3Dno_such_user"=20does=20not=20exist/,=0A+=09'checking=20= with=20a=20non-existent=20user');=0A+=0A+#=20Failing=20to=20connect=20to=20= the=20initial=20database=20due=20to=20bad=20username=20is=20still=20an=0A= +#=20error=20when=20using=20--no-strict-names.=0A+command_fails_like(=0A= +=09[=20'pg_amcheck',=20'--no-strict-names',=20'-p',=20$port,=20= '-U=3Dno_such_user',=20'postgres'=20],=0A+=09qr/role=20"=3Dno_such_user"=20= does=20not=20exist/,=0A+=09'checking=20with=20a=20non-existent=20user,=20= --no-strict-names');=0A+=0A+#########################################=0A= +#=20Test=20checking=20databases=20without=20amcheck=20installed=0A+=0A= +#=20Attempting=20to=20check=20a=20database=20by=20name=20where=20= amcheck=20is=20not=20installed=20should=0A+#=20raise=20a=20warning.=20=20= If=20all=20databases=20are=20skipped,=20having=20no=20relations=20to=20= check=0A+#=20raises=20an=20error.=0A+$node->command_checks_all(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'template1'=20],=0A+=091,=0A+=09[],=0A+=09= [=20qr/pg_amcheck:=20skipping=20database=20"template1":=20amcheck=20is=20= not=20installed/,=0A+=09=20=20qr/pg_amcheck:=20no=20relations=20to=20= check/=20],=0A+=09'checking=20a=20database=20by=20name=20without=20= amcheck=20installed');=0A+=0A+#=20Likewise,=20but=20by=20database=20= pattern=20rather=20than=20by=20name,=20such=20that=20some=0A+#=20= databases=20with=20amcheck=20installed=20are=20included,=20and=20so=20= checking=20occurs=20and=0A+#=20only=20a=20warning=20is=20raised.=0A= +$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= '-d',=20'*',=20'postgres'=20],=0A+=090,=0A+=09[],=0A+=09[=20= qr/pg_amcheck:=20skipping=20database=20"template1":=20amcheck=20is=20not=20= installed/=20],=0A+=09'checking=20a=20database=20by=20dbname=20= implication=20without=20amcheck=20installed');=0A+=0A+#=20And=20again,=20= but=20by=20checking=20all=20databases.=0A+$node->command_checks_all(=0A+=09= [=20'pg_amcheck',=20'-p',=20$port,=20'--all',=20'postgres'=20],=0A+=090,=0A= +=09[],=0A+=09[=20qr/pg_amcheck:=20skipping=20database=20"template1":=20= amcheck=20is=20not=20installed/=20],=0A+=09'checking=20a=20database=20by=20= --all=20implication=20without=20amcheck=20installed');=0A+=0A= +#########################################=0A+#=20Test=20unreasonable=20= patterns=0A+=0A+#=20Check=20three-part=20unreasonable=20pattern=20that=20= has=20zero-length=20names=0A+$node->command_checks_all(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'postgres',=20'-t',=20'..'=20],=0A+=09= 1,=0A+=09[=20qr/^$/=20],=0A+=09[=20qr/pg_amcheck:=20no=20checkable=20= database:=20"\.\."/=20],=0A+=09'checking=20table=20pattern=20".."');=0A+=0A= +#=20Again,=20but=20with=20non-trivial=20schema=20and=20relation=20parts=0A= +$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-t',=20'.foo.bar'=20],=0A+=091,=0A+=09[=20qr/^$/=20],=0A+=09= [=20qr/pg_amcheck:=20no=20checkable=20database:=20"\.foo\.bar"/=20],=0A+=09= 'checking=20table=20pattern=20".foo.bar"');=0A+=0A+#=20Check=20two-part=20= unreasonable=20pattern=20that=20has=20zero-length=20names=0A= +$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20= 'postgres',=20'-t',=20'.'=20],=0A+=091,=0A+=09[=20qr/^$/=20],=0A+=09[=20= qr/pg_amcheck:=20no=20tables=20to=20check=20for=20"\."/=20],=0A+=09= 'checking=20table=20pattern=20"."');=0A+=0A= +#########################################=0A+#=20Test=20checking=20= non-existent=20schemas,=20tables,=20and=20indexes=0A+=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20'-s',=20= 'no_such_schema'=20],=0A+=09qr/pg_amcheck:=20no=20relations=20to=20check=20= in=20schemas=20for=20"no_such_schema"/,=0A+=09'checking=20a=20= non-existent=20schema');=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'--no-strict-names',=20'-v',=20'-p',=20$port,=20'-s',=20= 'no_such_schema'=20],=0A+=09qr/pg_amcheck:=20no=20relations=20to=20= check/,=0A+=09'checking=20a=20non-existent=20schema=20with=20= --no-strict-names=20-v');=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'-t',=20'no_such_table'=20],=0A+=09= qr/pg_amcheck:=20no=20tables=20to=20check=20for=20"no_such_table"/,=0A+=09= 'checking=20a=20non-existent=20table');=0A+=0A+command_fails_like(=0A+=09= [=20'pg_amcheck',=20'--no-strict-names',=20'-v',=20'-p',=20$port,=20= '-t',=20'no_such_table'=20],=0A+=09qr/pg_amcheck:=20no=20relations=20to=20= check/,=0A+=09'checking=20a=20non-existent=20table=20with=20= --no-strict-names=20-v');=0A+=0A+command_fails_like(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$port,=20'-i',=20'no_such_index'=20],=0A+=09= qr/pg_amcheck:=20no=20btree=20indexes=20to=20check=20for=20= "no_such_index"/,=0A+=09'checking=20a=20non-existent=20index');=0A+=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'--no-strict-names',=20= '-v',=20'-p',=20$port,=20'-i',=20'no_such_index'=20],=0A+=09= qr/pg_amcheck:=20no=20relations=20to=20check/,=0A+=09'checking=20a=20= non-existent=20index=20with=20--no-strict-names=20-v');=0A+=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20'-s',=20= 'no*such*schema*'=20],=0A+=09qr/pg_amcheck:=20no=20relations=20to=20= check=20in=20schemas=20for=20"no\*such\*schema\*"/,=0A+=09'no=20matching=20= schemas');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20= '--no-strict-names',=20'-v',=20'-p',=20$port,=20'-s',=20= 'no*such*schema*'=20],=0A+=09qr/pg_amcheck:=20no=20relations=20to=20= check/,=0A+=09'no=20matching=20schemas=20with=20--no-strict-names=20= -v');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20= $port,=20'-t',=20'no*such*table*'=20],=0A+=09qr/pg_amcheck:=20no=20= tables=20to=20check=20for=20"no\*such\*table\*"/,=0A+=09'no=20matching=20= tables');=0A+=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20= '--no-strict-names',=20'-v',=20'-p',=20$port,=20'-t',=20'no*such*table*'=20= ],=0A+=09qr/pg_amcheck:=20no=20relations=20to=20check/,=0A+=09'no=20= matching=20tables=20with=20--no-strict-names=20-v');=0A+=0A= +command_fails_like(=0A+=09[=20'pg_amcheck',=20'-p',=20$port,=20'-i',=20= 'no*such*index*'=20],=0A+=09qr/pg_amcheck:=20no=20btree=20indexes=20to=20= check=20for=20"no\*such\*index\*"/,=0A+=09'no=20matching=20indexes');=0A= +=0A+command_fails_like(=0A+=09[=20'pg_amcheck',=20'--no-strict-names',=20= '-v',=20'-p',=20$port,=20'-i',=20'no*such*index*'=20],=0A+=09= qr/pg_amcheck:=20no=20relations=20to=20check/,=0A+=09'no=20matching=20= indexes=20with=20--no-strict-names=20-v');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/003_check.pl=20= b/contrib/pg_amcheck/t/003_check.pl=0Anew=20file=20mode=20100644=0Aindex=20= 0000000000..502b599fcd=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/003_check.pl=0A@@=20-0,0=20+1,497=20@@=0A+use=20= strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20TestLib;=0A= +use=20Test::More=20tests=20=3D>=2057;=0A+=0A+my=20($node,=20$port,=20= %corrupt_page,=20%remove_relation);=0A+=0A+#=20Returns=20the=20= filesystem=20path=20for=20the=20named=20relation.=0A+#=0A+#=20Assumes=20= the=20test=20node=20is=20running=0A+sub=20relation_filepath($$)=0A+{=0A+=09= my=20($dbname,=20$relname)=20=3D=20@_;=0A+=0A+=09my=20$pgdata=20=3D=20= $node->data_dir;=0A+=09my=20$rel=20=3D=20$node->safe_psql($dbname,=0A+=09= =09=09=09=09=09=09=20=20=20qq(SELECT=20= pg_relation_filepath('$relname')));=0A+=09die=20"path=20not=20found=20= for=20relation=20$relname"=20unless=20defined=20$rel;=0A+=09return=20= "$pgdata/$rel";=0A+}=0A+=0A+#=20Returns=20the=20name=20of=20the=20toast=20= relation=20associated=20with=20the=20named=20relation.=0A+#=0A+#=20= Assumes=20the=20test=20node=20is=20running=0A+sub=20relation_toast($$)=0A= +{=0A+=09my=20($dbname,=20$relname)=20=3D=20@_;=0A+=0A+=09my=20$rel=20=3D=20= $node->safe_psql($dbname,=20qq(=0A+=09=09SELECT=20ct.relname=0A+=09=09=09= FROM=20pg_catalog.pg_class=20cr,=20pg_catalog.pg_class=20ct=0A+=09=09=09= WHERE=20cr.oid=20=3D=20'$relname'::regclass=0A+=09=09=09=20=20AND=20= cr.reltoastrelid=20=3D=20ct.oid=0A+=09=09=09));=0A+=09return=20undef=20= unless=20defined=20$rel;=0A+=09return=20"pg_toast.$rel";=0A+}=0A+=0A+#=20= Adds=20the=20relation=20file=20for=20the=20given=20(dbname,=20relname)=20= to=20the=20list=0A+#=20to=20be=20corrupted=20by=20means=20of=20= overwriting=20junk=20in=20the=20first=20page.=0A+#=0A+#=20Assumes=20the=20= test=20node=20is=20running.=0A+sub=20plan_to_corrupt_first_page($$)=0A+{=0A= +=09my=20($dbname,=20$relname)=20=3D=20@_;=0A+=09my=20$relpath=20=3D=20= relation_filepath($dbname,=20$relname);=0A+=09$corrupt_page{$relpath}=20= =3D=201;=0A+}=0A+=0A+#=20Adds=20the=20relation=20file=20for=20the=20= given=20(dbname,=20relname)=20to=20the=20list=0A+#=20to=20be=20corrupted=20= by=20means=20of=20removing=20the=20file..=0A+#=0A+#=20Assumes=20the=20= test=20node=20is=20running=0A+sub=20plan_to_remove_relation_file($$)=0A= +{=0A+=09my=20($dbname,=20$relname)=20=3D=20@_;=0A+=09my=20$relpath=20=3D=20= relation_filepath($dbname,=20$relname);=0A+=09$remove_relation{$relpath}=20= =3D=201;=0A+}=0A+=0A+#=20For=20the=20given=20(dbname,=20relname),=20if=20= a=20corresponding=20toast=20table=0A+#=20exists,=20adds=20that=20toast=20= table's=20relation=20file=20to=20the=20list=20to=20be=0A+#=20corrupted=20= by=20means=20of=20removing=20the=20file.=0A+#=0A+#=20Assumes=20the=20= test=20node=20is=20running.=0A+sub=20plan_to_remove_toast_file($$)=0A+{=0A= +=09my=20($dbname,=20$relname)=20=3D=20@_;=0A+=09my=20$toastname=20=3D=20= relation_toast($dbname,=20$relname);=0A+=09= plan_to_remove_relation_file($dbname,=20$toastname)=20if=20($toastname);=0A= +}=0A+=0A+#=20Corrupts=20the=20first=20page=20of=20the=20given=20file=20= path=0A+sub=20corrupt_first_page($)=0A+{=0A+=09my=20($relpath)=20=3D=20= @_;=0A+=0A+=09my=20$fh;=0A+=09open($fh,=20'+<',=20$relpath)=0A+=09=20=20= or=20BAIL_OUT("open=20failed:=20$!");=0A+=09binmode=20$fh;=0A+=0A+=09#=20= Corrupt=20some=20line=20pointers.=20=20The=20values=20are=20chosen=20to=20= hit=20the=0A+=09#=20various=20line-pointer-corruption=20checks=20in=20= verify_heapam.c=0A+=09#=20on=20both=20little-endian=20and=20big-endian=20= architectures.=0A+=09seek($fh,=2032,=200)=0A+=09=20=20or=20= BAIL_OUT("seek=20failed:=20$!");=0A+=09syswrite(=0A+=09=09$fh,=0A+=09=09= pack("L*",=0A+=09=09=090xAAA15550,=200xAAA0D550,=200x00010000,=0A+=09=09=09= 0x00008000,=200x0000800F,=200x001e8000,=0A+=09=09=090xFFFFFFFF)=0A+=09)=20= or=20BAIL_OUT("syswrite=20failed:=20$!");=0A+=09close($fh)=0A+=09=20=20= or=20BAIL_OUT("close=20failed:=20$!");=0A+}=0A+=0A+#=20Stops=20the=20= node,=20performs=20all=20the=20corruptions=20previously=20planned,=20and=0A= +#=20starts=20the=20node=20again.=0A+#=0A+sub=20= perform_all_corruptions()=0A+{=0A+=09$node->stop();=0A+=09for=20my=20= $relpath=20(keys=20%corrupt_page)=0A+=09{=0A+=09=09= corrupt_first_page($relpath);=0A+=09}=0A+=09for=20my=20$relpath=20(keys=20= %remove_relation)=0A+=09{=0A+=09=09unlink($relpath);=0A+=09}=0A+=09= $node->start;=0A+}=0A+=0A+#=20Test=20set-up=0A+$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A+$node->start;=0A+$port=20=3D=20= $node->port;=0A+=0A+for=20my=20$dbname=20(qw(db1=20db2=20db3))=0A+{=0A+=09= #=20Create=20the=20database=0A+=09$node->safe_psql('postgres',=20= qq(CREATE=20DATABASE=20$dbname));=0A+=0A+=09#=20Load=20the=20amcheck=20= extension,=20upon=20which=20pg_amcheck=20depends.=20=20Put=20the=0A+=09#=20= extension=20in=20an=20unexpected=20location=20to=20test=20that=20= pg_amcheck=20finds=20it=0A+=09#=20correctly.=20=20Create=20tables=20with=20= names=20that=20look=20like=20pg_catalog=20names=20to=0A+=09#=20check=20= that=20pg_amcheck=20does=20not=20get=20confused=20by=20them.=20=20Create=20= functions=20in=0A+=09#=20schema=20public=20that=20look=20like=20amcheck=20= functions=20to=20check=20that=20pg_amcheck=0A+=09#=20does=20not=20use=20= them.=0A+=09$node->safe_psql($dbname,=20q(=0A+=09=09CREATE=20SCHEMA=20= amcheck_schema;=0A+=09=09CREATE=20EXTENSION=20amcheck=20WITH=20SCHEMA=20= amcheck_schema;=0A+=09=09CREATE=20TABLE=20amcheck_schema.pg_database=20= (junk=20text);=0A+=09=09CREATE=20TABLE=20amcheck_schema.pg_namespace=20= (junk=20text);=0A+=09=09CREATE=20TABLE=20amcheck_schema.pg_class=20(junk=20= text);=0A+=09=09CREATE=20TABLE=20amcheck_schema.pg_operator=20(junk=20= text);=0A+=09=09CREATE=20TABLE=20amcheck_schema.pg_proc=20(junk=20text);=0A= +=09=09CREATE=20TABLE=20amcheck_schema.pg_tablespace=20(junk=20text);=0A= +=0A+=09=09CREATE=20FUNCTION=20public.bt_index_check(index=20regclass,=0A= +=09=09=09=09=09=09=09=09=09=09=09=20=20heapallindexed=20boolean=20= default=20false)=0A+=09=09RETURNS=20VOID=20AS=20$$=0A+=09=09BEGIN=0A+=09=09= =09RAISE=20EXCEPTION=20'Invoked=20wrong=20bt_index_check!';=0A+=09=09= END;=0A+=09=09$$=20LANGUAGE=20plpgsql;=0A+=0A+=09=09CREATE=20FUNCTION=20= public.bt_index_parent_check(index=20regclass,=0A+=09=09=09=09=09=09=09=09= =09=09=09=09=09=20heapallindexed=20boolean=20default=20false,=0A+=09=09=09= =09=09=09=09=09=09=09=09=09=09=20rootdescend=20boolean=20default=20= false)=0A+=09=09RETURNS=20VOID=20AS=20$$=0A+=09=09BEGIN=0A+=09=09=09= RAISE=20EXCEPTION=20'Invoked=20wrong=20bt_index_parent_check!';=0A+=09=09= END;=0A+=09=09$$=20LANGUAGE=20plpgsql;=0A+=0A+=09=09CREATE=20FUNCTION=20= public.verify_heapam(relation=20regclass,=0A+=09=09=09=09=09=09=09=09=09=09= =09=20on_error_stop=20boolean=20default=20false,=0A+=09=09=09=09=09=09=09= =09=09=09=09=20check_toast=20boolean=20default=20false,=0A+=09=09=09=09=09= =09=09=09=09=09=09=20skip=20text=20default=20'none',=0A+=09=09=09=09=09=09= =09=09=09=09=09=20startblock=20bigint=20default=20null,=0A+=09=09=09=09=09= =09=09=09=09=09=09=20endblock=20bigint=20default=20null,=0A+=09=09=09=09=09= =09=09=09=09=09=09=20blkno=20OUT=20bigint,=0A+=09=09=09=09=09=09=09=09=09= =09=09=20offnum=20OUT=20integer,=0A+=09=09=09=09=09=09=09=09=09=09=09=20= attnum=20OUT=20integer,=0A+=09=09=09=09=09=09=09=09=09=09=09=20msg=20OUT=20= text)=0A+=09=09RETURNS=20SETOF=20record=20AS=20$$=0A+=09=09BEGIN=0A+=09=09= =09RAISE=20EXCEPTION=20'Invoked=20wrong=20verify_heapam!';=0A+=09=09END;=0A= +=09=09$$=20LANGUAGE=20plpgsql;=0A+=09));=0A+=0A+=09#=20Create=20= schemas,=20tables=20and=20indexes=20in=20five=20separate=0A+=09#=20= schemas.=20=20The=20schemas=20are=20all=20identical=20to=20start,=20but=0A= +=09#=20we=20will=20corrupt=20them=20differently=20later.=0A+=09#=0A+=09= for=20my=20$schema=20(qw(s1=20s2=20s3=20s4=20s5))=0A+=09{=0A+=09=09= $node->safe_psql($dbname,=20qq(=0A+=09=09=09CREATE=20SCHEMA=20$schema;=0A= +=09=09=09CREATE=20SEQUENCE=20$schema.seq1;=0A+=09=09=09CREATE=20= SEQUENCE=20$schema.seq2;=0A+=09=09=09CREATE=20TABLE=20$schema.t1=20(=0A+=09= =09=09=09i=20INTEGER,=0A+=09=09=09=09b=20BOX,=0A+=09=09=09=09ia=20= int4[],=0A+=09=09=09=09ir=20int4range,=0A+=09=09=09=09t=20TEXT=0A+=09=09=09= );=0A+=09=09=09CREATE=20TABLE=20$schema.t2=20(=0A+=09=09=09=09i=20= INTEGER,=0A+=09=09=09=09b=20BOX,=0A+=09=09=09=09ia=20int4[],=0A+=09=09=09= =09ir=20int4range,=0A+=09=09=09=09t=20TEXT=0A+=09=09=09);=0A+=09=09=09= CREATE=20VIEW=20$schema.t2_view=20AS=20(=0A+=09=09=09=09SELECT=20i*2,=20= t=20FROM=20$schema.t2=0A+=09=09=09);=0A+=09=09=09ALTER=20TABLE=20= $schema.t2=0A+=09=09=09=09ALTER=20COLUMN=20t=0A+=09=09=09=09SET=20= STORAGE=20EXTERNAL;=0A+=0A+=09=09=09INSERT=20INTO=20$schema.t1=20(i,=20= b,=20ia,=20ir,=20t)=0A+=09=09=09=09(SELECT=20gs::INTEGER=20AS=20i,=0A+=09= =09=09=09=09=09box(point(gs,gs+5),point(gs*2,gs*3))=20AS=20b,=0A+=09=09=09= =09=09=09array[gs,=20gs=20+=201]::int4[]=20AS=20ia,=0A+=09=09=09=09=09=09= int4range(gs,=20gs+100)=20AS=20ir,=0A+=09=09=09=09=09=09repeat('foo',=20= gs)=20AS=20t=0A+=09=09=09=09=09=20FROM=20generate_series(1,10000,3000)=20= AS=20gs);=0A+=0A+=09=09=09INSERT=20INTO=20$schema.t2=20(i,=20b,=20ia,=20= ir,=20t)=0A+=09=09=09=09(SELECT=20gs::INTEGER=20AS=20i,=0A+=09=09=09=09=09= =09box(point(gs,gs+5),point(gs*2,gs*3))=20AS=20b,=0A+=09=09=09=09=09=09= array[gs,=20gs=20+=201]::int4[]=20AS=20ia,=0A+=09=09=09=09=09=09= int4range(gs,=20gs+100)=20AS=20ir,=0A+=09=09=09=09=09=09repeat('foo',=20= gs)=20AS=20t=0A+=09=09=09=09=09=20FROM=20generate_series(1,10000,3000)=20= AS=20gs);=0A+=0A+=09=09=09CREATE=20MATERIALIZED=20VIEW=20$schema.t1_mv=20= AS=20SELECT=20*=20FROM=20$schema.t1;=0A+=09=09=09CREATE=20MATERIALIZED=20= VIEW=20$schema.t2_mv=20AS=20SELECT=20*=20FROM=20$schema.t2;=0A+=0A+=09=09= =09create=20table=20$schema.p1=20(a=20int,=20b=20int)=20PARTITION=20BY=20= list=20(a);=0A+=09=09=09create=20table=20$schema.p2=20(a=20int,=20b=20= int)=20PARTITION=20BY=20list=20(a);=0A+=0A+=09=09=09create=20table=20= $schema.p1_1=20partition=20of=20$schema.p1=20for=20values=20in=20(1,=20= 2,=203);=0A+=09=09=09create=20table=20$schema.p1_2=20partition=20of=20= $schema.p1=20for=20values=20in=20(4,=205,=206);=0A+=09=09=09create=20= table=20$schema.p2_1=20partition=20of=20$schema.p2=20for=20values=20in=20= (1,=202,=203);=0A+=09=09=09create=20table=20$schema.p2_2=20partition=20= of=20$schema.p2=20for=20values=20in=20(4,=205,=206);=0A+=0A+=09=09=09= CREATE=20INDEX=20t1_btree=20ON=20$schema.t1=20USING=20BTREE=20(i);=0A+=09= =09=09CREATE=20INDEX=20t2_btree=20ON=20$schema.t2=20USING=20BTREE=20(i);=0A= +=0A+=09=09=09CREATE=20INDEX=20t1_hash=20ON=20$schema.t1=20USING=20HASH=20= (i);=0A+=09=09=09CREATE=20INDEX=20t2_hash=20ON=20$schema.t2=20USING=20= HASH=20(i);=0A+=0A+=09=09=09CREATE=20INDEX=20t1_brin=20ON=20$schema.t1=20= USING=20BRIN=20(i);=0A+=09=09=09CREATE=20INDEX=20t2_brin=20ON=20= $schema.t2=20USING=20BRIN=20(i);=0A+=0A+=09=09=09CREATE=20INDEX=20= t1_gist=20ON=20$schema.t1=20USING=20GIST=20(b);=0A+=09=09=09CREATE=20= INDEX=20t2_gist=20ON=20$schema.t2=20USING=20GIST=20(b);=0A+=0A+=09=09=09= CREATE=20INDEX=20t1_gin=20ON=20$schema.t1=20USING=20GIN=20(ia);=0A+=09=09= =09CREATE=20INDEX=20t2_gin=20ON=20$schema.t2=20USING=20GIN=20(ia);=0A+=0A= +=09=09=09CREATE=20INDEX=20t1_spgist=20ON=20$schema.t1=20USING=20SPGIST=20= (ir);=0A+=09=09=09CREATE=20INDEX=20t2_spgist=20ON=20$schema.t2=20USING=20= SPGIST=20(ir);=0A+=09=09));=0A+=09}=0A+}=0A+=0A+#=20Database=20'db1'=20= corruptions=0A+#=0A+=0A+#=20Corrupt=20indexes=20in=20schema=20"s1"=0A= +plan_to_remove_relation_file('db1',=20's1.t1_btree');=0A= +plan_to_corrupt_first_page('db1',=20's1.t2_btree');=0A+=0A+#=20Corrupt=20= tables=20in=20schema=20"s2"=0A+plan_to_remove_relation_file('db1',=20= 's2.t1');=0A+plan_to_corrupt_first_page('db1',=20's2.t2');=0A+=0A+#=20= Corrupt=20tables,=20partitions,=20matviews,=20and=20btrees=20in=20schema=20= "s3"=0A+plan_to_remove_relation_file('db1',=20's3.t1');=0A= +plan_to_corrupt_first_page('db1',=20's3.t2');=0A+=0A= +plan_to_remove_relation_file('db1',=20's3.t1_mv');=0A= +plan_to_remove_relation_file('db1',=20's3.p1_1');=0A+=0A= +plan_to_corrupt_first_page('db1',=20's3.t2_mv');=0A= +plan_to_corrupt_first_page('db1',=20's3.p2_1');=0A+=0A= +plan_to_remove_relation_file('db1',=20's3.t1_btree');=0A= +plan_to_corrupt_first_page('db1',=20's3.t2_btree');=0A+=0A+#=20Corrupt=20= toast=20table,=20partitions,=20and=20materialized=20views=20in=20schema=20= "s4"=0A+plan_to_remove_toast_file('db1',=20's4.t2');=0A+=0A+#=20Corrupt=20= all=20other=20object=20types=20in=20schema=20"s5".=20=20We=20don't=20= have=20amcheck=20support=0A+#=20for=20these=20types,=20but=20we=20check=20= that=20their=20corruption=20does=20not=20trigger=20any=0A+#=20errors=20= in=20pg_amcheck=0A+plan_to_remove_relation_file('db1',=20's5.seq1');=0A= +plan_to_remove_relation_file('db1',=20's5.t1_hash');=0A= +plan_to_remove_relation_file('db1',=20's5.t1_gist');=0A= +plan_to_remove_relation_file('db1',=20's5.t1_gin');=0A= +plan_to_remove_relation_file('db1',=20's5.t1_brin');=0A= +plan_to_remove_relation_file('db1',=20's5.t1_spgist');=0A+=0A= +plan_to_corrupt_first_page('db1',=20's5.seq2');=0A= +plan_to_corrupt_first_page('db1',=20's5.t2_hash');=0A= +plan_to_corrupt_first_page('db1',=20's5.t2_gist');=0A= +plan_to_corrupt_first_page('db1',=20's5.t2_gin');=0A= +plan_to_corrupt_first_page('db1',=20's5.t2_brin');=0A= +plan_to_corrupt_first_page('db1',=20's5.t2_spgist');=0A+=0A+=0A+#=20= Database=20'db2'=20corruptions=0A+#=0A= +plan_to_remove_relation_file('db2',=20's1.t1');=0A= +plan_to_remove_relation_file('db2',=20's1.t1_btree');=0A+=0A+=0A+#=20= Leave=20'db3'=20uncorrupted=0A+#=0A+=0A+#=20Perform=20the=20corruptions=20= we=20planned=20above=20using=20only=20a=20single=20database=20restart.=0A= +#=0A+perform_all_corruptions();=0A+=0A+=0A+#=20Standard=20first=20= arguments=20to=20TestLib=20functions=0A+my=20@cmd=20=3D=20('pg_amcheck',=20= '--quiet',=20'-p',=20$port);=0A+=0A+#=20Regular=20expressions=20to=20= match=20various=20expected=20output=0A+my=20$no_output_re=20=3D=20= qr/^$/;=0A+my=20$line_pointer_corruption_re=20=3D=20qr/line=20pointer/;=0A= +my=20$missing_file_re=20=3D=20qr/could=20not=20open=20file=20".*":=20No=20= such=20file=20or=20directory/;=0A+my=20$index_missing_relation_fork_re=20= =3D=20qr/index=20".*"=20lacks=20a=20main=20relation=20fork/;=0A+=0A+#=20= Checking=20databases=20with=20amcheck=20installed=20and=20corrupt=20= relations,=20pg_amcheck=0A+#=20command=20itself=20should=20return=20exit=20= status=20=3D=202,=20because=20tables=20and=20indexes=20are=0A+#=20= corrupt,=20not=20exit=20status=20=3D=201,=20which=20would=20mean=20the=20= pg_amcheck=20command=20itself=0A+#=20failed.=20=20Corruption=20messages=20= should=20go=20to=20stdout,=20and=20nothing=20to=20stderr.=0A+#=0A= +$node->command_checks_all(=0A+=09[=20@cmd,=20'db1'=20],=0A+=092,=0A+=09= [=20$index_missing_relation_fork_re,=0A+=09=20=20= $line_pointer_corruption_re,=0A+=09=20=20$missing_file_re,=0A+=09],=0A+=09= [=20$no_output_re=20],=0A+=09'pg_amcheck=20all=20schemas,=20tables=20and=20= indexes=20in=20database=20db1');=0A+=0A+$node->command_checks_all(=0A+=09= [=20@cmd,=20'db1',=20'db2',=20'db3'=20],=0A+=092,=0A+=09[=20= $index_missing_relation_fork_re,=0A+=09=20=20= $line_pointer_corruption_re,=0A+=09=20=20$missing_file_re,=0A+=09],=0A+=09= [=20$no_output_re=20],=0A+=09'pg_amcheck=20all=20schemas,=20tables=20and=20= indexes=20in=20databases=20db1,=20db2,=20and=20db3');=0A+=0A+#=20Scans=20= of=20indexes=20in=20s1=20should=20detect=20the=20specific=20corruption=20= that=20we=20created=0A+#=20above.=20=20For=20missing=20relation=20forks,=20= we=20know=20what=20the=20error=20message=20looks=0A+#=20like.=20=20For=20= corrupted=20index=20pages,=20the=20error=20might=20vary=20depending=20on=20= how=20the=0A+#=20page=20was=20formatted=20on=20disk,=20including=20= variations=20due=20to=20alignment=20differences=0A+#=20between=20= platforms,=20so=20we=20accept=20any=20non-empty=20error=20message.=0A+#=0A= +#=20If=20we=20don't=20limit=20the=20check=20to=20databases=20with=20= amcheck=20installed,=20we=20expect=0A+#=20complaint=20on=20stderr,=20but=20= otherwise=20stderr=20should=20be=20quiet.=0A+#=0A= +$node->command_checks_all(=0A+=09[=20@cmd,=20'--all',=20'-s',=20's1',=20= '-i',=20't1_btree'=20],=0A+=092,=0A+=09[=20= $index_missing_relation_fork_re=20],=0A+=09[=20qr/pg_amcheck:=20skipping=20= database=20"postgres":=20amcheck=20is=20not=20installed/=20],=0A+=09= 'pg_amcheck=20index=20s1.t1_btree=20reports=20missing=20main=20relation=20= fork');=0A+=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20'db1',=20= '-s',=20's1',=20'-i',=20't2_btree'=20],=0A+=092,=0A+=09[=20qr/.+/=20],=09= =09=09#=20Any=20non-empty=20error=20message=20is=20acceptable=0A+=09[=20= $no_output_re=20],=0A+=09'pg_amcheck=20index=20s1.s2=20reports=20index=20= corruption');=0A+=0A+#=20Checking=20db1.s1=20with=20indexes=20excluded=20= should=20show=20no=20corruptions=20because=20we=0A+#=20did=20not=20= corrupt=20any=20tables=20in=20db1.s1.=20=20Verify=20that=20both=20stdout=20= and=20stderr=0A+#=20are=20quiet.=0A+#=0A+$node->command_checks_all(=0A+=09= [=20@cmd,=20'db1',=20'-t',=20's1.*',=20'--no-index-expansion'=20],=0A+=09= 0,=0A+=09[=20$no_output_re=20],=0A+=09[=20$no_output_re=20],=0A+=09= 'pg_amcheck=20of=20db1.s1=20excluding=20indexes');=0A+=0A+#=20Checking=20= db2.s1=20should=20show=20table=20corruptions=20if=20indexes=20are=20= excluded=0A+#=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20'db2',=20= '-t',=20's1.*',=20'--no-index-expansion'=20],=0A+=092,=0A+=09[=20= $missing_file_re=20],=0A+=09[=20$no_output_re=20],=0A+=09'pg_amcheck=20= of=20db2.s1=20excluding=20indexes');=0A+=0A+#=20In=20schema=20db1.s3,=20= the=20tables=20and=20indexes=20are=20both=20corrupt.=20=20We=20should=20= see=0A+#=20corruption=20messages=20on=20stdout,=20and=20nothing=20on=20= stderr.=0A+#=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20'db1',=20= '-s',=20's3'=20],=0A+=092,=0A+=09[=20$index_missing_relation_fork_re,=0A= +=09=20=20$line_pointer_corruption_re,=0A+=09=20=20$missing_file_re,=0A+=09= ],=0A+=09[=20$no_output_re=20],=0A+=09'pg_amcheck=20schema=20s3=20= reports=20table=20and=20index=20errors');=0A+=0A+#=20In=20schema=20= db1.s4,=20only=20toast=20tables=20are=20corrupt.=20=20Check=20that=20= under=20default=0A+#=20options=20the=20toast=20corruption=20is=20= reported,=20but=20when=20excluding=20toast=20we=20get=20no=0A+#=20error=20= reports.=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20'db1',=20'-s',=20= 's4'=20],=0A+=092,=0A+=09[=20$missing_file_re=20],=0A+=09[=20= $no_output_re=20],=0A+=09'pg_amcheck=20in=20schema=20s4=20reports=20= toast=20corruption');=0A+=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20= '--no-toast-expansion',=20'--exclude-toast-pointers',=20'db1',=20'-s',=20= 's4'=20],=0A+=090,=0A+=09[=20$no_output_re=20],=0A+=09[=20$no_output_re=20= ],=0A+=09'pg_amcheck=20in=20schema=20s4=20excluding=20toast=20reports=20= no=20corruption');=0A+=0A+#=20Check=20that=20no=20corruption=20is=20= reported=20in=20schema=20db1.s5=0A+$node->command_checks_all(=0A+=09[=20= @cmd,=20'db1',=20'-s',=20's5'=20],=0A+=090,=0A+=09[=20$no_output_re=20],=0A= +=09[=20$no_output_re=20],=0A+=09'pg_amcheck=20over=20schema=20s5=20= reports=20no=20corruption');=0A+=0A+#=20In=20schema=20db1.s1,=20only=20= indexes=20are=20corrupt.=20=20Verify=20that=20when=20we=20exclude=0A+#=20= the=20indexes,=20no=20corruption=20is=20reported=20about=20the=20schema.=0A= +#=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20'db1',=20'-s',=20= 's1',=20'-I',=20't1_btree',=20'-I',=20't2_btree'=20],=0A+=090,=0A+=09[=20= $no_output_re=20],=0A+=09[=20$no_output_re=20],=0A+=09'pg_amcheck=20over=20= schema=20s1=20with=20corrupt=20indexes=20excluded=20reports=20no=20= corruption');=0A+=0A+#=20In=20schema=20db1.s1,=20only=20indexes=20are=20= corrupt.=20=20Verify=20that=20when=20we=20provide=20only=0A+#=20table=20= inclusions,=20and=20disable=20index=20expansion,=20no=20corruption=20is=20= reported=0A+#=20about=20the=20schema.=0A+#=0A+$node->command_checks_all(=0A= +=09[=20@cmd,=20'db1',=20'-t',=20's1.*',=20'--no-index-expansion'=20],=0A= +=090,=0A+=09[=20$no_output_re=20],=0A+=09[=20$no_output_re=20],=0A+=09= 'pg_amcheck=20over=20schema=20s1=20with=20all=20indexes=20excluded=20= reports=20no=20corruption');=0A+=0A+#=20In=20schema=20db1.s2,=20only=20= tables=20are=20corrupt.=20=20Verify=20that=20when=20we=20exclude=20those=0A= +#=20tables=20that=20no=20corruption=20is=20reported.=0A+#=0A= +$node->command_checks_all(=0A+=09[=20@cmd,=20'db1',=20'-s',=20's2',=20= '-T',=20't1',=20'-T',=20't2'=20],=0A+=090,=0A+=09[=20$no_output_re=20],=0A= +=09[=20$no_output_re=20],=0A+=09'pg_amcheck=20over=20schema=20s2=20with=20= corrupt=20tables=20excluded=20reports=20no=20corruption');=0A+=0A+#=20= Check=20errors=20about=20bad=20block=20range=20command=20line=20= arguments.=20=20We=20use=20schema=20s5=0A+#=20to=20avoid=20getting=20= messages=20about=20corrupt=20tables=20or=20indexes.=0A+#=0A= +command_fails_like(=0A+=09[=20@cmd,=20'db1',=20'-s',=20's5',=20= '--startblock',=20'junk'=20],=0A+=09qr/relation=20starting=20block=20= argument=20contains=20garbage=20characters/,=0A+=09'pg_amcheck=20rejects=20= garbage=20startblock');=0A+=0A+command_fails_like(=0A+=09[=20@cmd,=20= 'db1',=20'-s',=20's5',=20'--endblock',=20'1234junk'=20],=0A+=09= qr/relation=20ending=20block=20argument=20contains=20garbage=20= characters/,=0A+=09'pg_amcheck=20rejects=20garbage=20endblock');=0A+=0A= +command_fails_like(=0A+=09[=20@cmd,=20'db1',=20'-s',=20's5',=20= '--startblock',=20'5',=20'--endblock',=20'4'=20],=0A+=09qr/relation=20= ending=20block=20argument=20precedes=20starting=20block=20argument/,=0A+=09= 'pg_amcheck=20rejects=20invalid=20block=20range');=0A+=0A+#=20Check=20= bt_index_parent_check=20alternates.=20=20We=20don't=20create=20any=20= index=20corruption=0A+#=20that=20would=20behave=20differently=20under=20= these=20modes,=20so=20just=20smoke=20test=20that=20the=0A+#=20arguments=20= are=20handled=20sensibly.=0A+#=0A+$node->command_checks_all(=0A+=09[=20= @cmd,=20'db1',=20'-s',=20's1',=20'-i',=20't1_btree',=20'--parent-check'=20= ],=0A+=092,=0A+=09[=20$index_missing_relation_fork_re=20],=0A+=09[=20= $no_output_re=20],=0A+=09'pg_amcheck=20smoke=20test=20--parent-check');=0A= +=0A+$node->command_checks_all(=0A+=09[=20@cmd,=20'db1',=20'-s',=20's1',=20= '-i',=20't1_btree',=20'--heapallindexed',=20'--rootdescend'=20],=0A+=09= 2,=0A+=09[=20$index_missing_relation_fork_re=20],=0A+=09[=20= $no_output_re=20],=0A+=09'pg_amcheck=20smoke=20test=20--heapallindexed=20= --rootdescend');=0Adiff=20--git=20= a/contrib/pg_amcheck/t/004_verify_heapam.pl=20= b/contrib/pg_amcheck/t/004_verify_heapam.pl=0Anew=20file=20mode=20100644=0A= index=200000000000..d5537a5b37=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/004_verify_heapam.pl=0A@@=20-0,0=20+1,487=20@@=0A= +use=20strict;=0A+use=20warnings;=0A+=0A+use=20PostgresNode;=0A+use=20= TestLib;=0A+=0A+use=20Test::More=20tests=20=3D>=2020;=0A+=0A+#=20This=20= regression=20test=20demonstrates=20that=20the=20pg_amcheck=20binary=20= supplied=20with=0A+#=20the=20pg_amcheck=20contrib=20module=20correctly=20= identifies=20specific=20kinds=20of=0A+#=20corruption=20within=20pages.=20= =20To=20test=20this,=20we=20need=20a=20mechanism=20to=20create=20corrupt=0A= +#=20pages=20with=20predictable,=20repeatable=20corruption.=20=20The=20= postgres=20backend=20cannot=0A+#=20be=20expected=20to=20help=20us=20with=20= this,=20as=20its=20design=20is=20not=20consistent=20with=20the=0A+#=20= goal=20of=20intentionally=20corrupting=20pages.=0A+#=0A+#=20Instead,=20= we=20create=20a=20table=20to=20corrupt,=20and=20with=20careful=20= consideration=20of=20how=0A+#=20postgresql=20lays=20out=20heap=20pages,=20= we=20seek=20to=20offsets=20within=20the=20page=20and=0A+#=20overwrite=20= deliberately=20chosen=20bytes=20with=20specific=20values=20calculated=20= to=0A+#=20corrupt=20the=20page=20in=20expected=20ways.=20=20We=20then=20= verify=20that=20pg_amcheck=20reports=0A+#=20the=20corruption,=20and=20= that=20it=20runs=20without=20crashing.=20=20Note=20that=20the=20backend=0A= +#=20cannot=20simply=20be=20started=20to=20run=20queries=20against=20the=20= corrupt=20table,=20as=20the=0A+#=20backend=20will=20crash,=20at=20least=20= for=20some=20of=20the=20corruption=20types=20we=20generate.=0A+#=0A+#=20= Autovacuum=20potentially=20touching=20the=20table=20in=20the=20= background=20makes=20the=20exact=0A+#=20behavior=20of=20this=20test=20= harder=20to=20reason=20about.=20=20We=20turn=20it=20off=20to=20keep=20= things=0A+#=20simpler.=20=20We=20use=20a=20"belt=20and=20suspenders"=20= approach,=20turning=20it=20off=20for=20the=0A+#=20system=20generally=20= in=20postgresql.conf,=20and=20turning=20it=20off=20specifically=20for=20= the=0A+#=20test=20table.=0A+#=0A+#=20This=20test=20depends=20on=20the=20= table=20being=20written=20to=20the=20heap=20file=20exactly=20as=20we=0A= +#=20expect=20it=20to=20be,=20so=20we=20take=20care=20to=20arrange=20the=20= columns=20of=20the=20table,=20and=0A+#=20insert=20rows=20of=20the=20= table,=20that=20give=20predictable=20sizes=20and=20locations=20within=0A= +#=20the=20table=20page.=0A+#=0A+#=20The=20HeapTupleHeaderData=20has=20= 23=20bytes=20of=20fixed=20size=20fields=20before=20the=20variable=0A+#=20= length=20t_bits[]=20array.=20=20We=20have=20exactly=203=20columns=20in=20= the=20table,=20so=20natts=20=3D=203,=0A+#=20t_bits=20is=201=20byte=20= long,=20and=20t_hoff=20=3D=20MAXALIGN(23=20+=201)=20=3D=2024.=0A+#=0A+#=20= We're=20not=20too=20fussy=20about=20which=20datatypes=20we=20use=20for=20= the=20test,=20but=20we=20do=20care=0A+#=20about=20some=20specific=20= properties.=20=20We'd=20like=20to=20test=20both=20fixed=20size=20and=0A= +#=20varlena=20types.=20=20We'd=20like=20some=20varlena=20data=20inline=20= and=20some=20toasted.=20=20And=0A+#=20we'd=20like=20the=20layout=20of=20= the=20table=20such=20that=20the=20datums=20land=20at=20predictable=0A+#=20= offsets=20within=20the=20tuple.=20=20We=20choose=20a=20structure=20= without=20padding=20on=20all=0A+#=20supported=20architectures:=0A+#=0A+#=20= =09a=20BIGINT=0A+#=09b=20TEXT=0A+#=09c=20TEXT=0A+#=0A+#=20We=20always=20= insert=20a=207-ascii=20character=20string=20into=20field=20'b',=20which=20= with=20a=0A+#=201-byte=20varlena=20header=20gives=20an=208=20byte=20= inline=20value.=20=20We=20always=20insert=20a=20long=0A+#=20text=20= string=20in=20field=20'c',=20long=20enough=20to=20force=20toast=20= storage.=0A+#=0A+#=20We=20choose=20to=20read=20and=20write=20binary=20= copies=20of=20our=20table's=20tuples,=20using=20perl's=0A+#=20pack()=20= and=20unpack()=20functions.=20=20Perl=20uses=20a=20packing=20code=20= system=20in=20which:=0A+#=0A+#=09L=20=3D=20"Unsigned=2032-bit=20Long",=0A= +#=09S=20=3D=20"Unsigned=2016-bit=20Short",=0A+#=09C=20=3D=20"Unsigned=20= 8-bit=20Octet",=0A+#=09c=20=3D=20"signed=208-bit=20octet",=0A+#=09q=20=3D=20= "signed=2064-bit=20quadword"=0A+#=0A+#=20Each=20tuple=20in=20our=20table=20= has=20a=20layout=20as=20follows:=0A+#=0A+#=20=20=20=20xx=20xx=20xx=20xx=20= =20=20=20=20=20=20=20=20=20=20=20t_xmin:=20xxxx=09=09offset=20=3D=200=09=09= L=0A+#=20=20=20=20xx=20xx=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20= t_xmax:=20xxxx=09=09offset=20=3D=204=09=09L=0A+#=20=20=20=20xx=20xx=20xx=20= xx=20=20=20=20=20=20=20=20=20=20t_field3:=20xxxx=09=09offset=20=3D=208=09= =09L=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20bi_hi:=20xx=09=09=09offset=20=3D=2012=09=09S=0A+#=20=20=20=20= xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20bi_lo:=20= xx=09=09=09offset=20=3D=2014=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20ip_posid:=20xx=09=09=09offset=20=3D=20= 16=09=09S=0A+#=20=20=20=20xx=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20= t_infomask2:=20xx=09=09=09offset=20=3D=2018=09=09S=0A+#=20=20=20=20xx=20= xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20t_infomask:=20xx=09=09=09= offset=20=3D=2020=09=09S=0A+#=20=20=20=20xx=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20t_hoff:=20x=09=09=09offset=20=3D=2022=09= =09C=0A+#=20=20=20=20xx=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20t_bits:=20x=09=09=09offset=20=3D=2023=09=09C=0A+#=20=20=20=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'a':=20xxxxxxxx=09offset=20= =3D=2024=09=09q=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20= =20'b':=20xxxxxxxx=09offset=20=3D=2032=09=09Cccccccc=0A+#=20=20=20=20xx=20= xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20'c':=20xxxxxxxx=09offset=20=3D=20= 40=09=09SSSS=0A+#=20=20=20=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20xx=20=20=20= =20=20=20:=20xxxxxxxx=09=20...continued=09SSSS=0A+#=20=20=20=20xx=20xx=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20:=20= xx=20=20=20=20=20=20=09=20...continued=09S=0A+#=0A+#=20We=20could=20= choose=20to=20read=20and=20write=20columns=20'b'=20and=20'c'=20in=20= other=20ways,=20but=0A+#=20it=20is=20convenient=20enough=20to=20do=20it=20= this=20way.=20=20We=20define=20packing=20code=0A+#=20constants=20here,=20= where=20they=20can=20be=20compared=20easily=20against=20the=20layout.=0A= +=0A+use=20constant=20HEAPTUPLE_PACK_CODE=20=3D>=20= 'LLLSSSSSCCqCcccccccSSSSSSSSS';=0A+use=20constant=20= HEAPTUPLE_PACK_LENGTH=20=3D>=2058;=20=20=20=20=20#=20Total=20size=0A+=0A= +#=20Read=20a=20tuple=20of=20our=20table=20from=20a=20heap=20page.=0A+#=0A= +#=20Takes=20an=20open=20filehandle=20to=20the=20heap=20file,=20and=20= the=20offset=20of=20the=20tuple.=0A+#=0A+#=20Rather=20than=20returning=20= the=20binary=20data=20from=20the=20file,=20unpacks=20the=20data=20into=20= a=0A+#=20perl=20hash=20with=20named=20fields.=20=20These=20fields=20= exactly=20match=20the=20ones=20understood=0A+#=20by=20write_tuple(),=20= below.=20=20Returns=20a=20reference=20to=20this=20hash.=0A+#=0A+sub=20= read_tuple=20($$)=0A+{=0A+=09my=20($fh,=20$offset)=20=3D=20@_;=0A+=09my=20= ($buffer,=20%tup);=0A+=09seek($fh,=20$offset,=200);=0A+=09sysread($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=0A+=09@_=20=3D=20= unpack(HEAPTUPLE_PACK_CODE,=20$buffer);=0A+=09%tup=20=3D=20(t_xmin=20=3D>=20= shift,=0A+=09=09=09t_xmax=20=3D>=20shift,=0A+=09=09=09t_field3=20=3D>=20= shift,=0A+=09=09=09bi_hi=20=3D>=20shift,=0A+=09=09=09bi_lo=20=3D>=20= shift,=0A+=09=09=09ip_posid=20=3D>=20shift,=0A+=09=09=09t_infomask2=20=3D>= =20shift,=0A+=09=09=09t_infomask=20=3D>=20shift,=0A+=09=09=09t_hoff=20=3D>= =20shift,=0A+=09=09=09t_bits=20=3D>=20shift,=0A+=09=09=09a=20=3D>=20= shift,=0A+=09=09=09b_header=20=3D>=20shift,=0A+=09=09=09b_body1=20=3D>=20= shift,=0A+=09=09=09b_body2=20=3D>=20shift,=0A+=09=09=09b_body3=20=3D>=20= shift,=0A+=09=09=09b_body4=20=3D>=20shift,=0A+=09=09=09b_body5=20=3D>=20= shift,=0A+=09=09=09b_body6=20=3D>=20shift,=0A+=09=09=09b_body7=20=3D>=20= shift,=0A+=09=09=09c1=20=3D>=20shift,=0A+=09=09=09c2=20=3D>=20shift,=0A+=09= =09=09c3=20=3D>=20shift,=0A+=09=09=09c4=20=3D>=20shift,=0A+=09=09=09c5=20= =3D>=20shift,=0A+=09=09=09c6=20=3D>=20shift,=0A+=09=09=09c7=20=3D>=20= shift,=0A+=09=09=09c8=20=3D>=20shift,=0A+=09=09=09c9=20=3D>=20shift);=0A= +=09#=20Stitch=20together=20the=20text=20for=20column=20'b'=0A+=09= $tup{b}=20=3D=20join('',=20map=20{=20chr($tup{"b_body$_"})=20}=20= (1..7));=0A+=09return=20\%tup;=0A+}=0A+=0A+#=20Write=20a=20tuple=20of=20= our=20table=20to=20a=20heap=20page.=0A+#=0A+#=20Takes=20an=20open=20= filehandle=20to=20the=20heap=20file,=20the=20offset=20of=20the=20tuple,=20= and=20a=0A+#=20reference=20to=20a=20hash=20with=20the=20tuple=20values,=20= as=20returned=20by=20read_tuple().=0A+#=20Writes=20the=20tuple=20fields=20= from=20the=20hash=20into=20the=20heap=20file.=0A+#=0A+#=20The=20purpose=20= of=20this=20function=20is=20to=20write=20a=20tuple=20back=20to=20disk=20= with=20some=0A+#=20subset=20of=20fields=20modified.=20=20The=20function=20= does=20no=20error=20checking.=20=20Use=0A+#=20cautiously.=0A+#=0A+sub=20= write_tuple($$$)=0A+{=0A+=09my=20($fh,=20$offset,=20$tup)=20=3D=20@_;=0A= +=09my=20$buffer=20=3D=20pack(HEAPTUPLE_PACK_CODE,=0A+=09=09=09=09=09= $tup->{t_xmin},=0A+=09=09=09=09=09$tup->{t_xmax},=0A+=09=09=09=09=09= $tup->{t_field3},=0A+=09=09=09=09=09$tup->{bi_hi},=0A+=09=09=09=09=09= $tup->{bi_lo},=0A+=09=09=09=09=09$tup->{ip_posid},=0A+=09=09=09=09=09= $tup->{t_infomask2},=0A+=09=09=09=09=09$tup->{t_infomask},=0A+=09=09=09=09= =09$tup->{t_hoff},=0A+=09=09=09=09=09$tup->{t_bits},=0A+=09=09=09=09=09= $tup->{a},=0A+=09=09=09=09=09$tup->{b_header},=0A+=09=09=09=09=09= $tup->{b_body1},=0A+=09=09=09=09=09$tup->{b_body2},=0A+=09=09=09=09=09= $tup->{b_body3},=0A+=09=09=09=09=09$tup->{b_body4},=0A+=09=09=09=09=09= $tup->{b_body5},=0A+=09=09=09=09=09$tup->{b_body6},=0A+=09=09=09=09=09= $tup->{b_body7},=0A+=09=09=09=09=09$tup->{c1},=0A+=09=09=09=09=09= $tup->{c2},=0A+=09=09=09=09=09$tup->{c3},=0A+=09=09=09=09=09$tup->{c4},=0A= +=09=09=09=09=09$tup->{c5},=0A+=09=09=09=09=09$tup->{c6},=0A+=09=09=09=09= =09$tup->{c7},=0A+=09=09=09=09=09$tup->{c8},=0A+=09=09=09=09=09= $tup->{c9});=0A+=09seek($fh,=20$offset,=200);=0A+=09syswrite($fh,=20= $buffer,=20HEAPTUPLE_PACK_LENGTH);=0A+=09return;=0A+}=0A+=0A+#=20Set=20= umask=20so=20test=20directories=20and=20files=20are=20created=20with=20= default=20permissions=0A+umask(0077);=0A+=0A+#=20Set=20up=20the=20node.=20= =20Once=20we=20create=20and=20corrupt=20the=20table,=0A+#=20autovacuum=20= workers=20visiting=20the=20table=20could=20crash=20the=20backend.=0A+#=20= Disable=20autovacuum=20so=20that=20won't=20happen.=0A+my=20$node=20=3D=20= get_new_node('test');=0A+$node->init;=0A= +$node->append_conf('postgresql.conf',=20'autovacuum=3Doff');=0A+=0A+#=20= Start=20the=20node=20and=20load=20the=20extensions.=20=20We=20depend=20= on=20both=0A+#=20amcheck=20and=20pageinspect=20for=20this=20test.=0A= +$node->start;=0A+my=20$port=20=3D=20$node->port;=0A+my=20$pgdata=20=3D=20= $node->data_dir;=0A+$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20= amcheck");=0A+$node->safe_psql('postgres',=20"CREATE=20EXTENSION=20= pageinspect");=0A+=0A+#=20Get=20a=20non-zero=20datfrozenxid=0A= +$node->safe_psql('postgres',=20qq(VACUUM=20FREEZE));=0A+=0A+#=20Create=20= the=20test=20table=20with=20precisely=20the=20schema=20that=20our=20= corruption=20function=0A+#=20expects.=0A+$node->safe_psql(=0A+=09= 'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20public.test=20(a=20BIGINT,=20= b=20TEXT,=20c=20TEXT);=0A+=09=09ALTER=20TABLE=20public.test=20SET=20= (autovacuum_enabled=3Dfalse);=0A+=09=09ALTER=20TABLE=20public.test=20= ALTER=20COLUMN=20c=20SET=20STORAGE=20EXTERNAL;=0A+=09=09CREATE=20INDEX=20= test_idx=20ON=20public.test(a,=20b);=0A+=09));=0A+=0A+#=20We=20want=20(0=20= <=20datfrozenxid=20<=20test.relfrozenxid).=20=20To=20achieve=20this,=20= we=20freeze=0A+#=20an=20otherwise=20unused=20table,=20public.junk,=20= prior=20to=20inserting=20data=20and=20freezing=0A+#=20public.test=0A= +$node->safe_psql(=0A+=09'postgres',=20qq(=0A+=09=09CREATE=20TABLE=20= public.junk=20AS=20SELECT=20'junk'::TEXT=20AS=20junk_column;=0A+=09=09= ALTER=20TABLE=20public.junk=20SET=20(autovacuum_enabled=3Dfalse);=0A+=09=09= VACUUM=20FREEZE=20public.junk=0A+=09));=0A+=0A+my=20$rel=20=3D=20= $node->safe_psql('postgres',=20qq(SELECT=20= pg_relation_filepath('public.test')));=0A+my=20$relpath=20=3D=20= "$pgdata/$rel";=0A+=0A+#=20Insert=20data=20and=20freeze=20public.test=0A= +use=20constant=20ROWCOUNT=20=3D>=2016;=0A+$node->safe_psql('postgres',=20= qq(=0A+=09INSERT=20INTO=20public.test=20(a,=20b,=20c)=0A+=09=09VALUES=20= (=0A+=09=09=0912345678,=0A+=09=09=09'abcdefg',=0A+=09=09=09repeat('w',=20= 10000)=0A+=09=09);=0A+=09VACUUM=20FREEZE=20public.test=0A+=09))=20for=20= (1..ROWCOUNT);=0A+=0A+my=20$relfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A+=09q(select=20relfrozenxid=20from=20= pg_class=20where=20relname=20=3D=20'test'));=0A+my=20$datfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A+=09q(select=20datfrozenxid=20from=20= pg_database=20where=20datname=20=3D=20'postgres'));=0A+=0A+#=20Find=20= where=20each=20of=20the=20tuples=20is=20located=20on=20the=20page.=0A+my=20= @lp_off;=0A+for=20my=20$tup=20(0..ROWCOUNT-1)=0A+{=0A+=09push=20= (@lp_off,=20$node->safe_psql('postgres',=20qq(=0A+select=20lp_off=20from=20= heap_page_items(get_raw_page('test',=20'main',=200))=0A+=09offset=20$tup=20= limit=201)));=0A+}=0A+=0A+#=20Check=20that=20pg_amcheck=20runs=20against=20= the=20uncorrupted=20table=20without=20error.=0A= +$node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A+=09=09= =09=09=20=20'pg_amcheck=20test=20table,=20prior=20to=20corruption');=0A+=0A= +#=20Check=20that=20pg_amcheck=20runs=20against=20the=20uncorrupted=20= table=20and=20index=20without=20error.=0A= +$node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A+=09=09= =09=09=20=20'pg_amcheck=20test=20table=20and=20index,=20prior=20to=20= corruption');=0A+=0A+$node->stop;=0A+=0A+#=20Sanity=20check=20that=20our=20= 'test'=20table=20has=20a=20relfrozenxid=20newer=20than=20the=0A+#=20= datfrozenxid=20for=20the=20database,=20and=20that=20the=20datfrozenxid=20= is=20greater=20than=20the=0A+#=20first=20normal=20xid.=20=20We=20rely=20= on=20these=20invariants=20in=20some=20of=20our=20tests.=0A+if=20= ($datfrozenxid=20<=3D=203=20||=20$datfrozenxid=20>=3D=20$relfrozenxid)=0A= +{=0A+=09fail('Xid=20thresholds=20not=20as=20expected');=0A+=09= $node->clean_node;=0A+=09exit;=0A+}=0A+=0A+#=20Some=20#define=20= constants=20from=20access/htup_details.h=20for=20use=20while=20= corrupting.=0A+use=20constant=20HEAP_HASNULL=20=20=20=20=20=20=20=20=20=20= =20=20=3D>=200x0001;=0A+use=20constant=20HEAP_XMAX_LOCK_ONLY=20=20=20=20=20= =3D>=200x0080;=0A+use=20constant=20HEAP_XMIN_COMMITTED=20=20=20=20=20=3D>=20= 0x0100;=0A+use=20constant=20HEAP_XMIN_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0200;=0A+use=20constant=20HEAP_XMAX_COMMITTED=20=20=20=20=20=3D>=20= 0x0400;=0A+use=20constant=20HEAP_XMAX_INVALID=20=20=20=20=20=20=20=3D>=20= 0x0800;=0A+use=20constant=20HEAP_NATTS_MASK=20=20=20=20=20=20=20=20=20=3D>= =200x07FF;=0A+use=20constant=20HEAP_XMAX_IS_MULTI=20=20=20=20=20=20=3D>=20= 0x1000;=0A+use=20constant=20HEAP_KEYS_UPDATED=20=20=20=20=20=20=20=3D>=20= 0x2000;=0A+=0A+#=20Helper=20function=20to=20generate=20a=20regular=20= expression=20matching=20the=20header=20we=0A+#=20expect=20= verify_heapam()=20to=20return=20given=20which=20fields=20we=20expect=20= to=20be=20non-null.=0A+sub=20header=0A+{=0A+=09my=20($blkno,=20$offnum,=20= $attnum)=20=3D=20@_;=0A+=09return=20qr/relation=20= postgres\.public\.test,=20block=20$blkno,=20offset=20$offnum,=20= attribute=20$attnum\s+/ms=0A+=09=09if=20(defined=20$attnum);=0A+=09= return=20qr/relation=20postgres\.public\.test,=20block=20$blkno,=20= offset=20$offnum\s+/ms=0A+=09=09if=20(defined=20$offnum);=0A+=09return=20= qr/relation=20postgres\.public\.test\s+/ms=0A+=09=09if=20(defined=20= $blkno);=0A+=09return=20qr/relation=20postgres\.public\.test\s+/ms;=0A+}=0A= +=0A+#=20Corrupt=20the=20tuples,=20one=20type=20of=20corruption=20per=20= tuple.=20=20Some=20types=20of=0A+#=20corruption=20cause=20verify_heapam=20= to=20skip=20to=20the=20next=20tuple=20without=0A+#=20performing=20any=20= remaining=20checks,=20so=20we=20can't=20exercise=20the=20system=20= properly=20if=0A+#=20we=20focus=20all=20our=20corruption=20on=20a=20= single=20tuple.=0A+#=0A+my=20@expected;=0A+my=20$file;=0A+open($file,=20= '+<',=20$relpath);=0A+binmode=20$file;=0A+=0A+for=20(my=20$tupidx=20=3D=20= 0;=20$tupidx=20<=20ROWCOUNT;=20$tupidx++)=0A+{=0A+=09my=20$offnum=20=3D=20= $tupidx=20+=201;=20=20#=20offnum=20is=201-based,=20not=20zero-based=0A+=09= my=20$offset=20=3D=20$lp_off[$tupidx];=0A+=09my=20$tup=20=3D=20= read_tuple($file,=20$offset);=0A+=0A+=09#=20Sanity-check=20that=20the=20= data=20appears=20on=20the=20page=20where=20we=20expect.=0A+=09if=20= ($tup->{a}=20ne=20'12345678'=20||=20$tup->{b}=20ne=20'abcdefg')=0A+=09{=0A= +=09=09fail('Page=20layout=20differs=20from=20our=20expectations');=0A+=09= =09$node->clean_node;=0A+=09=09exit;=0A+=09}=0A+=0A+=09my=20$header=20=3D=20= header(0,=20$offnum,=20undef);=0A+=09if=20($offnum=20=3D=3D=201)=0A+=09{=0A= +=09=09#=20Corruptly=20set=20xmin=20<=20relfrozenxid=0A+=09=09my=20$xmin=20= =3D=20$relfrozenxid=20-=201;=0A+=09=09$tup->{t_xmin}=20=3D=20$xmin;=0A+=09= =09$tup->{t_infomask}=20&=3D=20~HEAP_XMIN_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20&=3D=20~HEAP_XMIN_INVALID;=0A+=0A+=09=09#=20= Expected=20corruption=20report=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}xmin=20$xmin=20precedes=20relation=20freeze=20threshold=20= 0:\d+/;=0A+=09}=0A+=09if=20($offnum=20=3D=3D=202)=0A+=09{=0A+=09=09#=20= Corruptly=20set=20xmin=20<=20datfrozenxid=0A+=09=09my=20$xmin=20=3D=203;=0A= +=09=09$tup->{t_xmin}=20=3D=20$xmin;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmin=20$xmin=20precedes=20oldest=20valid=20transaction=20ID=20= 0:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=203)=0A+=09{=0A+=09=09#=20= Corruptly=20set=20xmin=20<=20datfrozenxid,=20further=20back,=20noting=20= circularity=0A+=09=09#=20of=20xid=20comparison.=20=20For=20a=20new=20= cluster=20with=20epoch=20=3D=200,=20the=20corrupt=0A+=09=09#=20xmin=20= will=20be=20interpreted=20as=20in=20the=20future=0A+=09=09$tup->{t_xmin}=20= =3D=204026531839;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_COMMITTED;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMIN_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmin=204026531839=20equals=20or=20exceeds=20next=20valid=20= transaction=20ID=200:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=204)=0A= +=09{=0A+=09=09#=20Corruptly=20set=20xmax=20<=20relminmxid;=0A+=09=09= $tup->{t_xmax}=20=3D=204026531839;=0A+=09=09$tup->{t_infomask}=20&=3D=20= ~HEAP_XMAX_INVALID;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${$header}xmax=204026531839=20equals=20or=20exceeds=20next=20valid=20= transaction=20ID=200:\d+/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=205)=0A= +=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20but=20keep=20it=20= aligned=20properly=0A+=09=09$tup->{t_hoff}=20+=3D=20128;=0A+=0A+=09=09= push=20@expected,=0A+=09=09=09qr/${$header}data=20begins=20at=20offset=20= 152=20beyond=20the=20tuple=20length=2058/,=0A+=09=09=09= qr/${$header}tuple=20data=20should=20begin=20at=20byte=2024,=20but=20= actually=20begins=20at=20byte=20152=20\(3=20attributes,=20no=20nulls\)/;=0A= +=09}=0A+=09elsif=20($offnum=20=3D=3D=206)=0A+=09{=0A+=09=09#=20Corrupt=20= the=20tuple=20t_hoff,=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20+=3D=20= 3;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 27=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=207)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20= underflow=20but=20correct=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=20= 8;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 16=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=208)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20t_hoff,=20= underflow=20and=20wrong=20alignment=0A+=09=09$tup->{t_hoff}=20-=3D=203;=0A= +=0A+=09=09push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20= should=20begin=20at=20byte=2024,=20but=20actually=20begins=20at=20byte=20= 21=20\(3=20attributes,=20no=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20= =3D=3D=209)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20to=20look=20= like=20it=20has=20lots=20of=20attributes,=20not=20just=203=0A+=09=09= $tup->{t_infomask2}=20|=3D=20HEAP_NATTS_MASK;=0A+=0A+=09=09push=20= @expected,=0A+=09=09=09qr/${$header}number=20of=20attributes=202047=20= exceeds=20maximum=20expected=20for=20table=203/;=0A+=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2010)=0A+=09{=0A+=09=09#=20Corrupt=20the=20tuple=20to=20= look=20like=20it=20has=20lots=20of=20attributes,=20some=20of=0A+=09=09#=20= them=20null.=20=20This=20falsely=20creates=20the=20impression=20that=20= the=20t_bits=0A+=09=09#=20array=20is=20longer=20than=20just=20one=20= byte,=20but=20t_hoff=20still=20says=20otherwise.=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09$tup->{t_infomask2}=20= |=3D=20HEAP_NATTS_MASK;=0A+=09=09$tup->{t_bits}=20=3D=200xAA;=0A+=0A+=09=09= push=20@expected,=0A+=09=09=09qr/${$header}tuple=20data=20should=20begin=20= at=20byte=20280,=20but=20actually=20begins=20at=20byte=2024=20\(2047=20= attributes,=20has=20nulls\)/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=20= 11)=0A+=09{=0A+=09=09#=20Same=20as=20above,=20but=20this=20time=20t_hoff=20= plays=20along=0A+=09=09$tup->{t_infomask}=20|=3D=20HEAP_HASNULL;=0A+=09=09= $tup->{t_infomask2}=20|=3D=20(HEAP_NATTS_MASK=20&=200x40);=0A+=09=09= $tup->{t_bits}=20=3D=200xAA;=0A+=09=09$tup->{t_hoff}=20=3D=2032;=0A+=0A+=09= =09push=20@expected,=0A+=09=09=09qr/${$header}number=20of=20attributes=20= 67=20exceeds=20maximum=20expected=20for=20table=203/;=0A+=09}=0A+=09= elsif=20($offnum=20=3D=3D=2012)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= bits=20in=20column=20'b'=201-byte=20varlena=20header=0A+=09=09= $tup->{b_header}=20=3D=200x80;=0A+=0A+=09=09$header=20=3D=20header(0,=20= $offnum,=201);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}attribute=201=20with=20length=204294967295=20ends=20at=20= offset=20416848000=20beyond=20total=20tuple=20length=2058/;=0A+=09}=0A+=09= elsif=20($offnum=20=3D=3D=2013)=0A+=09{=0A+=09=09#=20Corrupt=20the=20= bits=20in=20column=20'c'=20toast=20pointer=0A+=09=09$tup->{c6}=20=3D=20= 41;=0A+=09=09$tup->{c7}=20=3D=2041;=0A+=0A+=09=09$header=20=3D=20= header(0,=20$offnum,=202);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}final=20toast=20chunk=20number=200=20differs=20from=20= expected=20value=206/,=0A+=09=09=09qr/${header}toasted=20value=20for=20= attribute=202=20missing=20from=20toast=20table/;=0A+=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2014)=0A+=09{=0A+=09=09#=20Set=20both=20= HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_IS_MULTI;=0A+=09=09$tup->{t_xmax}=20= =3D=204;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}multitransaction=20ID=204=20equals=20or=20exceeds=20next=20= valid=20multitransaction=20ID=201/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D= =2015)=09#=20Last=20offnum=20must=20equal=20ROWCOUNT=0A+=09{=0A+=09=09#=20= Set=20both=20HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A+=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_IS_MULTI;=0A+=09=09$tup->{t_xmax}=20= =3D=204000000000;=0A+=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}multitransaction=20ID=204000000000=20precedes=20relation=20= minimum=20multitransaction=20ID=20threshold=201/;=0A+=09}=0A+=09= write_tuple($file,=20$offset,=20$tup);=0A+}=0A+close($file);=0A= +$node->start;=0A+=0A+#=20Run=20pg_amcheck=20against=20the=20corrupt=20= table=20with=20epoch=3D0,=20comparing=20actual=0A+#=20corruption=20= messages=20against=20the=20expected=20messages=0A= +$node->command_checks_all(=0A+=09['pg_amcheck',=20= '--no-index-expansion',=20'-p',=20$port,=20'postgres'],=0A+=092,=0A+=09[=20= @expected=20],=0A+=09[=20],=0A+=09'Expected=20corruption=20message=20= output');=0A+=0A+$node->teardown_node;=0A+$node->clean_node;=0Adiff=20= --git=20a/contrib/pg_amcheck/t/005_opclass_damage.pl=20= b/contrib/pg_amcheck/t/005_opclass_damage.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..eba8ea9cae=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/005_opclass_damage.pl=0A@@=20-0,0=20+1,54=20@@=0A= +#=20This=20regression=20test=20checks=20the=20behavior=20of=20the=20= btree=20validation=20in=20the=0A+#=20presence=20of=20breaking=20sort=20= order=20changes.=0A+#=0A+use=20strict;=0A+use=20warnings;=0A+use=20= PostgresNode;=0A+use=20TestLib;=0A+use=20Test::More=20tests=20=3D>=205;=0A= +=0A+my=20$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+=0A+#=20Create=20a=20custom=20operator=20class=20and=20= an=20index=20which=20uses=20it.=0A+$node->safe_psql('postgres',=20q(=0A+=09= CREATE=20EXTENSION=20amcheck;=0A+=0A+=09CREATE=20FUNCTION=20int4_asc_cmp=20= (a=20int4,=20b=20int4)=20RETURNS=20int=20LANGUAGE=20sql=20AS=20$$=0A+=09=09= SELECT=20CASE=20WHEN=20$1=20=3D=20$2=20THEN=200=20WHEN=20$1=20>=20$2=20= THEN=201=20ELSE=20-1=20END;=20$$;=0A+=0A+=09CREATE=20OPERATOR=20CLASS=20= int4_fickle_ops=20FOR=20TYPE=20int4=20USING=20btree=20AS=0A+=09=20=20=20=20= OPERATOR=201=20<=20(int4,=20int4),=20OPERATOR=202=20<=3D=20(int4,=20= int4),=0A+=09=20=20=20=20OPERATOR=203=20=3D=20(int4,=20int4),=20OPERATOR=20= 4=20>=3D=20(int4,=20int4),=0A+=09=20=20=20=20OPERATOR=205=20>=20(int4,=20= int4),=20FUNCTION=201=20int4_asc_cmp(int4,=20int4);=0A+=0A+=09CREATE=20= TABLE=20int4tbl=20(i=20int4);=0A+=09INSERT=20INTO=20int4tbl=20(SELECT=20= *=20FROM=20generate_series(1,1000)=20gs);=0A+=09CREATE=20INDEX=20= fickleidx=20ON=20int4tbl=20USING=20btree=20(i=20int4_fickle_ops);=0A+));=0A= +=0A+#=20We=20have=20not=20yet=20broken=20the=20index,=20so=20we=20= should=20get=20no=20corruption=0A+$node->command_like(=0A+=09[=20= 'pg_amcheck',=20'--quiet',=20'-p',=20$node->port,=20'postgres'=20],=0A+=09= qr/^$/,=0A+=09'pg_amcheck=20all=20schemas,=20tables=20and=20indexes=20= reports=20no=20corruption');=0A+=0A+#=20Change=20the=20operator=20class=20= to=20use=20a=20function=20which=20sorts=20in=20a=20different=0A+#=20= order=20to=20corrupt=20the=20btree=20index=0A= +$node->safe_psql('postgres',=20q(=0A+=09CREATE=20FUNCTION=20= int4_desc_cmp=20(int4,=20int4)=20RETURNS=20int=20LANGUAGE=20sql=20AS=20= $$=0A+=09=09SELECT=20CASE=20WHEN=20$1=20=3D=20$2=20THEN=200=20WHEN=20$1=20= >=20$2=20THEN=20-1=20ELSE=201=20END;=20$$;=0A+=09UPDATE=20= pg_catalog.pg_amproc=0A+=09=09SET=20amproc=20=3D=20= 'int4_desc_cmp'::regproc=0A+=09=09WHERE=20amproc=20=3D=20= 'int4_asc_cmp'::regproc=0A+));=0A+=0A+#=20Index=20corruption=20should=20= now=20be=20reported=0A+$node->command_checks_all(=0A+=09[=20= 'pg_amcheck',=20'-p',=20$node->port,=20'postgres'=20],=0A+=092,=0A+=09[=20= qr/item=20order=20invariant=20violated=20for=20index=20"fickleidx"/=20],=0A= +=09[=20],=0A+=09'pg_amcheck=20all=20schemas,=20tables=20and=20indexes=20= reports=20fickleidx=20corruption'=0A+);=0Adiff=20--git=20= a/doc/src/sgml/contrib.sgml=20b/doc/src/sgml/contrib.sgml=0Aindex=20= d3ca4b6932..7e101f7c11=20100644=0A---=20a/doc/src/sgml/contrib.sgml=0A= +++=20b/doc/src/sgml/contrib.sgml=0A@@=20-185,6=20+185,7=20@@=20pages.=0A= =20=20=20=0A=20=0A=20=20&oid2name;=0A+=20&pgamcheck;=0A=20=20= &vacuumlo;=0A=20=20=0A=20=0Adiff=20--git=20= a/doc/src/sgml/filelist.sgml=20b/doc/src/sgml/filelist.sgml=0Aindex=20= db1d369743..5115cb03d0=20100644=0A---=20a/doc/src/sgml/filelist.sgml=0A= +++=20b/doc/src/sgml/filelist.sgml=0A@@=20-133,6=20+133,7=20@@=0A=20= =0A=20= =0A=20= =0A= +=0A=20= =0A=20= =0A=20= =0Adiff=20= --git=20a/doc/src/sgml/pgamcheck.sgml=20b/doc/src/sgml/pgamcheck.sgml=0A= new=20file=20mode=20100644=0Aindex=200000000000..9bee92c30a=0A---=20= /dev/null=0A+++=20b/doc/src/sgml/pgamcheck.sgml=0A@@=20-0,0=20+1,668=20= @@=0A+=0A+=0A+=0A+=20=0A+=20=20= pg_amcheck=0A+=20=0A+=0A+=20=0A+=20= =20pg_amcheck=0A= +=20=201=0A+=20=20= Application=0A+=20=0A+=0A+=20= =0A+=20=20pg_amcheck=0A+=20=20= checks=20for=20corruption=20in=20one=20or=20more=0A+=20=20= PostgreSQL=20databases=0A+=20= =0A+=0A+=20=0A+=20=20=0A+=20=20= =20pg_amcheck=0A+=20=20=20option=0A+=20=20=20dbname=0A+=20=20= =0A+=20=0A+=0A+=20=0A+=20=20= Description=0A+=0A+=20=20=0A+=20=20=20= pg_amcheck=20supports=20running=0A+=20=20=20= 's=20corruption=20checking=20functions=20= against=20one=20or=0A+=20=20=20more=20databases,=20with=20options=20to=20= select=20which=20schemas,=20tables=20and=20indexes=20to=0A+=20=20=20= check,=20which=20kinds=20of=20checking=20to=20perform,=20and=20whether=20= to=20perform=20the=20checks=0A+=20=20=20in=20parallel,=20and=20if=20so,=20= the=20number=20of=20parallel=20connections=20to=20establish=20and=0A+=20=20= =20use.=0A+=20=20=0A+=0A+=20=20=0A+=20=20=20Only=20table=20= relations=20and=20btree=20indexes=20are=20currently=20supported.=20=20= Other=0A+=20=20=20relation=20types=20are=20silently=20skipped.=0A+=20=20= =0A+=0A+=20=0A+=0A+=20=0A+=20=20= Options=0A+=0A+=20=20=0A+=20=20=20pg_amcheck=20= accepts=20the=20following=20command-line=20arguments:=0A+=0A+=20=20=20= =0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Perform=20checking=20= in=20all=20databases.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20In=20the=20absence=20of=20any=20other=20= options,=20selects=20all=20objects=20across=20all=0A+=20=20=20=20=20=20=20= schemas=20and=20databases.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=20=20Option=20=20= =20takes=0A+=20=20=20=20=20=20=20= precedence=20over=20.=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20= =20=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Perform=20= checking=20in=20the=20specified=20database.=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20This=20option=20may=20= be=20specified=20multiple=20times=20to=20list=20more=20than=20one=0A+=20=20= =20=20=20=20=20database=20(or=20database=20pattern)=20for=20checking.=20=20= By=20default,=20all=20objects=0A+=20=20=20=20=20=20=20in=20the=20= matching=20database(s)=20will=20be=20checked.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20If=20no=20= =20argument=20is=20given=20nor=20is=20any=0A= +=20=20=20=20=20=20=20database=20name=20given=20as=20a=20command=20line=20= argument,=20the=20first=20argument=0A+=20=20=20=20=20=20=20specified=20= with=20=20=20will=20be=0A+=20= =20=20=20=20=20=20used=20for=20the=20initial=20connection.=20=20If=20= that=20argument=20is=20not=20a=20literal=0A+=20=20=20=20=20=20=20= database=20name,=20the=20attempt=20to=20connect=20will=20fail.=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20If=20= =20is=20also=20specified,=20=0A= +=20=20=20=20=20=20=20=20does=20not=20affect=20= which=20databases=20are=20checked,=0A+=20=20=20=20=20=20=20but=20may=20= be=20used=20to=20specify=20the=20database=20for=20the=20initial=20= connection.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=20=20Option=20=20= =20takes=0A+=20=20=20=20=20=20=20= precedence=20over=20=20.=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= Examples:=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= --dbname=3Dafrica=0A+=20=20=20=20=20=20= =20=20--dbname=3D"a*"=0A+=20=20=20=20= =20=20=20=20= --dbname=3D"africa|asia|europe"=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Do=20not=20= perform=20checking=20in=20the=20specified=20database.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20This=20= option=20may=20be=20specified=20multiple=20times=20to=20list=20more=20= than=20one=0A+=20=20=20=20=20=20=20database=20(or=20database=20pattern)=20= for=20exclusion.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20If=20a=20database=20which=20is=20included=20= using=20=20or=0A+=20=20=20=20=20=20=20= =20=20is=20also=20excluded=20= using=0A+=20=20=20=20=20=20=20=20= ,=20the=20database=20will=0A+=20=20=20=20= =20=20=20be=20excluded.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20Examples:=0A+=20=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20=20= --exclude-dbname=3Damerica=0A+=20=20=20= =20=20=20=20=20= --exclude-dbname=3D"*pacific*"=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Print=20= to=20stdout=20all=20commands=20and=20queries=20being=20executed=20= against=20the=0A+=20=20=20=20=20=20=20server.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=0A+=0A= +=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Skip=20= (do=20not=20check)=20all=20pages=20after=20the=20given=20ending=20block.=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20By=20default,=20no=20pages=20are=20skipped.=20=20This=20option=20= will=20be=20applied=20to=20all=0A+=20=20=20=20=20=20=20table=20relations=20= that=20are=20checked,=20including=20toast=20tables,=20but=20note=20that=0A= +=20=20=20=20=20=20=20unless=20=20= is=20given,=20toast=0A+=20=20=20=20=20=20=20pointers=20found=20in=20the=20= main=20table=20will=20be=20followed=20into=20the=20toast=20table=0A+=20=20= =20=20=20=20=20without=20regard=20for=20the=20location=20in=20the=20= toast=20table.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20When=20= checking=20main=20relations,=20do=20not=20look=20up=20entries=20in=20= toast=20tables=0A+=20=20=20=20=20=20=20corresponding=20to=20toast=20= pointers=20in=20the=20main=20relation.=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20The=20default=20behavior=20= checks=20each=20toast=20pointer=20encountered=20in=20the=20main=0A+=20=20= =20=20=20=20=20table=20to=20verify,=20as=20much=20as=20possible,=20that=20= the=20pointer=20points=20at=0A+=20=20=20=20=20=20=20something=20in=20the=20= toast=20table=20that=20is=20reasonable.=20=20Toast=20pointers=20which=0A= +=20=20=20=20=20=20=20point=20beyond=20the=20end=20of=20the=20toast=20= table,=20or=20to=20the=20middle=20(rather=20than=0A+=20=20=20=20=20=20=20= the=20beginning)=20of=20a=20toast=20entry,=20are=20identified=20as=20= corrupt.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20The=20process=20by=20which=20's=0A+=20=20=20=20=20=20=20= verify_heapam=20function=20checks=20each=20toast=20= pointer=20is=0A+=20=20=20=20=20=20=20slow=20and=20may=20be=20improved=20= in=20a=20future=20release.=20=20Some=20users=20may=20wish=20to=0A+=20=20=20= =20=20=20=20disable=20this=20check=20to=20save=20time.=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=0A= +=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20For=20= each=20index=20checked,=20verify=20the=20presence=20of=20all=20heap=20= tuples=20as=20index=0A+=20=20=20=20=20=20=20tuples=20in=20the=20index=20= using=20's=0A+=20=20=20=20=20=20=20= =20option.=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20= =20=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Show=20= help=20about=20pg_amcheck=20command=20line=0A= +=20=20=20=20=20=20=20arguments,=20and=20exit.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=0A+=0A= +=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Specifies=20= the=20host=20name=20of=20the=20machine=20on=20which=20the=20server=20is=20= running.=0A+=20=20=20=20=20=20=20If=20the=20value=20begins=20with=20a=20= slash,=20it=20is=20used=20as=20the=20directory=20for=20the=0A+=20=20=20=20= =20=20=20Unix=20domain=20socket.=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Perform=20= checks=20on=20the=20specified=20index(es).=20=20This=20is=20an=20alias=20= for=20the=0A+=20=20=20=20=20=20=20=20= =20option,=20except=20that=20it=0A+=20=20=20=20= =20=20=20applies=20only=20to=20indexes,=20not=20tables.=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=0A= +=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Exclude=20= checks=20on=20the=20specified=20index(es).=20=20This=20is=20an=20alias=20= for=20the=0A+=20=20=20=20=20=20=20=20= =20option,=20except=0A+=20=20=20=20=20= =20=20that=20it=20applies=20only=20to=20indexes,=20not=20tables.=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Use=20the=20specified=20= number=20of=20concurrent=20connections=20to=20the=20server,=20or=0A+=20=20= =20=20=20=20=20one=20per=20object=20to=20be=20checked,=20whichever=20= number=20is=20smaller.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20The=20default=20is=20to=20use=20a=20= single=20connection.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Specifies=20= the=20name=20of=20the=20database=20to=20connect=20to=20when=20querying=20= the=0A+=20=20=20=20=20=20=20list=20of=20all=20databases.=20=20If=20not=20= specified,=20the=0A+=20=20=20=20=20=20=20postgres=20= database=20will=20be=20used;=20if=20that=20does=20not=0A+=20=20=20=20=20=20= =20exist=20template1=20will=20be=20used.=20=20This=20= can=20be=20a=0A+=20=20=20=20=20=20=20connection=20string.=20=20If=20so,=0A= +=20=20=20=20=20=20=20connection=20string=20parameters=20will=20override=20= any=20conflicting=20command=0A+=20=20=20=20=20=20=20line=20options.=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20When=20= including=20a=20table=20relation=20in=20the=20list=20of=20relations=20to=20= check,=20do=0A+=20=20=20=20=20=20=20not=20automatically=20include=20= btree=20indexes=20associated=20with=20table.=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20By=20= default,=20all=20tables=20to=20be=20checked=20will=20also=20have=20= checks=20performed=20on=0A+=20=20=20=20=20=20=20their=20associated=20= btree=20indexes,=20if=20any.=20=20If=20this=20option=20is=20given,=20= only=0A+=20=20=20=20=20=20=20those=20indexes=20which=20match=20a=20= =20or=0A+=20=20=20=20=20=20=20= =20pattern=20will=20be=20checked.=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20When=20= calculating=20the=20list=20of=20databases=20to=20check,=20and=20the=20= objects=20within=0A+=20=20=20=20=20=20=20those=20databases=20to=20be=20= checked,=20do=20not=20raise=20an=20error=20for=20database,=0A+=20=20=20=20= =20=20=20schema,=20relation,=20table,=20nor=20index=20inclusion=20= patterns=20which=20match=20no=0A+=20=20=20=20=20=20=20corresponding=20= objects.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20Exclusion=20patterns=20are=20not=20required=20to=20match=20= any=20objects,=20but=20by=0A+=20=20=20=20=20=20=20default=20unmatched=20= inclusion=20patterns=20raise=20an=20error,=20including=20when=0A+=20=20=20= =20=20=20=20they=20fail=20to=20match=20as=20a=20result=20of=20an=20= exclusion=20pattern=20having=0A+=20=20=20=20=20=20=20prohibited=20them=20= matching=20an=20existent=20object,=20and=20when=20they=20fail=20to=0A+=20= =20=20=20=20=20=20match=20a=20database=20because=20it=20is=20= unconnectable=20(datallowconn=20is=20false).=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20=20=20=0A+=0A= +=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20When=20= including=20a=20table=20relation=20in=20the=20list=20of=20relations=20to=20= check,=20do=0A+=20=20=20=20=20=20=20not=20automatically=20include=20= toast=20tables=20associated=20with=20table.=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20By=20= default,=20all=20tables=20to=20be=20checked=20will=20also=20have=20= checks=20performed=20on=0A+=20=20=20=20=20=20=20their=20associated=20= toast=20tables,=20if=20any.=20=20If=20this=20option=20is=20given,=20only=0A= +=20=20=20=20=20=20=20those=20toast=20tables=20which=20match=20a=20= =20or=0A+=20=20=20=20=20=20=20= =20pattern=20will=20be=20checked.=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20After=20= reporting=20all=20corruptions=20on=20the=20first=20page=20of=20a=20table=20= where=0A+=20=20=20=20=20=20=20corruptions=20are=20found,=20stop=20= processing=20that=20table=20relation=20and=20move=20on=0A+=20=20=20=20=20= =20=20to=20the=20next=20table=20or=20index.=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Note=20that=20index=20= checking=20always=20stops=20after=20the=20first=20corrupt=20page.=0A+=20=20= =20=20=20=20=20This=20option=20only=20has=20meaning=20relative=20to=20= table=20relations.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20For=20= each=20btree=20index=20checked,=20use=20's=0A= +=20=20=20=20=20=20=20bt_index_parent_check=20= function,=20which=20performs=0A+=20=20=20=20=20=20=20additional=20checks=20= of=20parent/child=20relationships=20during=20index=20checking.=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20The=20= default=20is=20to=20use=20amcheck's=0A+=20=20=20= =20=20=20=20bt_index_check=20function,=20but=20note=20= that=20use=20of=20the=0A+=20=20=20=20=20=20=20= =20option=20implicitly=20selects=0A+=20=20=20= =20=20=20=20bt_index_parent_check.=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=0A= +=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Specifies=20the=20TCP=20= port=20or=20local=20Unix=20domain=20socket=20file=20extension=20on=0A+=20= =20=20=20=20=20=20which=20the=20server=20is=20listening=20for=20= connections.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=20=20=20=20=0A+=0A+=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Show=20= progress=20information=20about=20how=20many=20relations=20have=20been=20= checked.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A+=20= =20=20=20=0A+=0A+=20=20=20=20=0A+=20=20=20=20= =20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20Do=20not=20write=20= additional=20messages=20beyond=20those=20about=20corruption.=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= This=20option=20does=20not=20quiet=20any=20output=20specifically=20due=20= to=20the=20use=20of=0A+=20=20=20=20=20=20=20the=20=20= =20option.=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Perform=20= checking=20on=20the=20specified=20relation(s).=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20This=20= option=20may=20be=20specified=20multiple=20times=20to=20list=20more=20= than=20one=0A+=20=20=20=20=20=20=20relation=20(or=20relation=20pattern)=20= for=20checking.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=20=20Option=20=20= =20takes=0A+=20=20=20=20=20=20=20= precedence=20over=20=20.=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =20=20Examples:=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =20=20--relation=3Daccounts_table=0A= +=20=20=20=20=20=20=20=20= --relation=3Daccounting_department.accounts_table=0A+=20=20=20=20=20=20=20=20= --relation=3Dcorporate_database.accounting_department.*_t= able=0A+=20=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Exclude=20= checks=20on=20the=20specified=20relation(s).=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Option=20= =20=20takes=0A+=20=20= =20=20=20=20=20precedence=20over=20=20= ,=0A+=20=20=20=20=20=20=20=20= =20and=20=0A+=20=20=20=20=20=20= =20.=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20For=20each=20index=20= checked,=20re-find=20tuples=20on=20the=20leaf=20level=20by=20performing=20= a=0A+=20=20=20=20=20=20=20new=20search=20from=20the=20root=20page=20for=20= each=20tuple=20using=0A+=20=20=20=20=20=20=20's=20=20option.=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= Use=20of=20this=20option=20implicitly=20also=20selects=20the=0A+=20=20=20= =20=20=20=20=20option.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20This=20form=20= of=20verification=20was=20originally=20written=20to=20help=20in=20the=0A= +=20=20=20=20=20=20=20development=20of=20btree=20index=20features.=20=20= It=20may=20be=20of=20limited=20use=20or=20even=0A+=20=20=20=20=20=20=20= of=20no=20use=20in=20helping=20detect=20the=20kinds=20of=20corruption=20= that=20occur=20in=0A+=20=20=20=20=20=20=20practice.=20=20It=20may=20also=20= cause=20corruption=20checking=20to=20take=20considerably=0A+=20=20=20=20=20= =20=20longer=20and=20consume=20considerably=20more=20resources=20on=20= the=20server.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=20=20=20=20=0A+=0A+=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20Perform=20checking=20in=20= the=20specified=20schema(s).=0A+=20=20=20=20=20=20=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=20=20This=20option=20may=20be=20= specified=20multiple=20times=20to=20list=20more=20than=20one=0A+=20=20=20= =20=20=20=20schema=20(or=20schema=20pattern)=20for=20checking.=20=20By=20= default,=20all=20objects=20in=0A+=20=20=20=20=20=20=20the=20matching=20= schema(s)=20will=20be=20checked.=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=20Option=20=20= =20takes=0A+=20=20=20=20=20=20=20= precedence=20over=20=20.=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= Examples:=0A+=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= --schema=3Dcorp=0A+=20=20=20=20=20=20= =20=20--schema=3D"corp|llc|npo"=0A+=20= =20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=20Note=20that=20both=20tables=20= and=20indexes=20are=20included=20using=20this=20option,=20which=0A+=20=20= =20=20=20=20=20might=20not=20be=20what=20you=20want=20if=20you=20are=20= also=20using=0A+=20=20=20=20=20=20=20= .=20=20To=20specify=20all=20tables=20= in=20a=20schema=0A+=20=20=20=20=20=20=20without=20also=20specifying=20= all=20indexes,=20=20can=20be=0A+=20=20=20=20=20=20= =20used=20with=20a=20pattern=20that=20specifies=20the=20schema.=20=20For=20= example,=20to=20check=0A+=20=20=20=20=20=20=20all=20tables=20in=20schema=20= corp,=20the=20option=0A+=20=20=20=20=20=20=20= --table=3D"corp.*"=20may=20be=20used.=0A+=20=20=20=20=20= =20=0A+=20=20=20=20=20=0A+=20=20=20=20=0A= +=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Do=20not=20= perform=20checking=20in=20the=20specified=20schema.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20This=20= option=20may=20be=20specified=20multiple=20times=20to=20list=20more=20= than=20one=0A+=20=20=20=20=20=20=20schema=20(or=20schema=20pattern)=20= for=20exclusion.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=20If=20a=20schema=20which=20is=20included=20= using=0A+=20=20=20=20=20=20=20=20= =20is=20also=20excluded=20using=0A+=20=20=20=20=20= =20=20=20,=20the=20= schema=20will=0A+=20=20=20=20=20=20=20be=20excluded.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Examples:=0A= +=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20=20= -S=20corp=20-S=20llc=0A+=20=20=20=20=20= =20=20=20--exclude-schema=3D"*c*"=0A= +=20=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20If=20= "all-frozen"=20is=20given,=20table=20corruption=20= checks=0A+=20=20=20=20=20=20=20will=20skip=20over=20pages=20in=20all=20= tables=20that=20are=20marked=20as=20all=20frozen.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20If=20= "all-visible"=20is=20given,=20table=20corruption=20= checks=0A+=20=20=20=20=20=20=20will=20skip=20over=20pages=20in=20all=20= tables=20that=20are=20marked=20as=20all=20visible.=0A+=20=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20By=20= default,=20no=20pages=20are=20skipped.=20=20This=20can=20be=20specified=20= as=0A+=20=20=20=20=20=20=20"none",=20but=20since=20= this=20is=20the=20default,=20it=20need=20not=20be=0A+=20=20=20=20=20=20=20= mentioned.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=0A= +=20=20=20=20=0A+=0A+=20=20=20=20=0A+=20=20=20= =20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Skip=20= (do=20not=20check)=20pages=20prior=20to=20the=20given=20starting=20= block.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=0A+=20=20= =20=20=20=20=20By=20default,=20no=20pages=20are=20skipped.=20=20This=20= option=20will=20be=20applied=20to=20all=0A+=20=20=20=20=20=20=20table=20= relations=20that=20are=20checked,=20including=20toast=20tables,=20but=20= note=0A+=20=20=20=20=20=20=20that=20unless=20= =20is=20given,=20toast=0A+=20=20= =20=20=20=20=20pointers=20found=20in=20the=20main=20table=20will=20be=20= followed=20into=20the=20toast=20table=0A+=20=20=20=20=20=20=20without=20= regard=20for=20the=20location=20in=20the=20toast=20table.=0A+=20=20=20=20= =20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=20=20Perform=20checks=20on=20= the=20specified=20tables(s).=20=20This=20is=20an=20alias=20for=20the=0A+=20= =20=20=20=20=20=20=20=20= option,=20except=20that=20it=0A+=20=20=20=20=20=20=20applies=20only=20to=20= tables,=20not=20indexes.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Exclude=20= checks=20on=20the=20specified=20tables(s).=20=20This=20is=20an=20alias=20= for=20the=0A+=20=20=20=20=20=20=20=20= =20option,=20except=0A+=20=20=20=20=20= =20=20that=20it=20applies=20only=20to=20tables,=20not=20indexes.=0A+=20=20= =20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20User=20= name=20to=20connect=20as.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Increases=20= the=20log=20level=20verbosity.=20=20This=20option=20may=20be=20given=20= more=20than=0A+=20=20=20=20=20=20=20once.=0A+=20=20=20=20=20=20=0A= +=20=20=20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20= =20=0A+=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Print=20= the=20pg_amcheck=20version=20and=20exit.=0A+=20= =20=20=20=20=20=0A+=20=20=20=20=20=0A+=20=20=20=20= =0A+=0A+=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20= =0A+=20=20=20=20=20=0A= +=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Never=20issue=20a=20= password=20prompt.=20=20If=20the=20server=20requires=20password=0A+=20=20= =20=20=20=20=20authentication=20and=20a=20password=20is=20not=20= available=20by=20other=20means=20such=20as=0A+=20=20=20=20=20=20=20a=20= .pgpass=20file,=20the=20connection=20attempt=20will=20= fail.=0A+=20=20=20=20=20=20=20This=20option=20can=20be=20useful=20in=20= batch=20jobs=20and=20scripts=20where=20no=20user=20is=0A+=20=20=20=20=20=20= =20present=20to=20enter=20a=20password.=0A+=20=20=20=20=20=20=0A+=20= =20=20=20=20=0A+=20=20=20=20=0A+=0A+=20=20=20=20= =0A+=20=20=20=20=20=0A+=20=20= =20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20Force=20= pg_amcheck=20to=20prompt=20for=20a=20password=0A= +=20=20=20=20=20=20=20before=20connecting=20to=20a=20database.=0A+=20=20=20= =20=20=20=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20=20=20= This=20option=20is=20never=20essential,=20since=0A+=20=20=20=20=20=20=20= pg_amcheck=20will=20automatically=20prompt=20= for=20a=0A+=20=20=20=20=20=20=20password=20if=20the=20server=20demands=20= password=20authentication.=20=20However,=0A+=20=20=20=20=20=20=20= pg_amcheck=20will=20waste=20a=20connection=20= attempt=0A+=20=20=20=20=20=20=20finding=20out=20that=20the=20server=20= wants=20a=20password.=20=20In=20some=20cases=20it=20is=0A+=20=20=20=20=20= =20=20worth=20typing=20=20to=20avoid=20the=20extra=20= connection=20attempt.=0A+=20=20=20=20=20=20=0A+=20=20=20=20=20= =0A+=20=20=20=20=0A+=0A+=20=20=20= =0A+=20=20=0A+=20=0A+=0A+=20=0A= +=20=20Notes=0A+=0A+=20=20=0A+=20=20=20= pg_amcheck=20is=20designed=20to=20work=20with=0A= +=20=20=20PostgreSQL=2014.0=20and=20later.=0A= +=20=20=0A+=20=0A+=0A+=20=0A+=20=20= Author=0A+=0A+=20=20=0A+=20=20=20Mark=20Dilger=20= mark.dilger@enterprisedb.com=0A+=20=20=0A+=20= =0A+=0A+=20=0A+=20=20See=20Also=0A+=0A= +=20=20=0A+=20=20=20=0A+=20=20=0A+=20=0A= +=0Adiff=20--git=20a/src/tools/msvc/Install.pm=20= b/src/tools/msvc/Install.pm=0Aindex=20ea3af48777..49ad558b74=20100644=0A= ---=20a/src/tools/msvc/Install.pm=0A+++=20b/src/tools/msvc/Install.pm=0A= @@=20-18,7=20+18,7=20@@=20our=20(@ISA,=20@EXPORT_OK);=0A=20@EXPORT_OK=20= =3D=20qw(Install);=0A=20=0A=20my=20$insttype;=0A-my=20@client_contribs=20= =3D=20('oid2name',=20'pgbench',=20'vacuumlo');=0A+my=20@client_contribs=20= =3D=20('oid2name',=20'pg_amcheck',=20'pgbench',=20'vacuumlo');=0A=20my=20= @client_program_files=20=3D=20(=0A=20=09'clusterdb',=20=20=20=20=20=20= 'createdb',=20=20=20'createuser',=20=20=20=20'dropdb',=0A=20=09= 'dropuser',=20=20=20=20=20=20=20'ecpg',=20=20=20=20=20=20=20'libecpg',=20= =20=20=20=20=20=20'libecpg_compat',=0Adiff=20--git=20= a/src/tools/msvc/Mkvcbuild.pm=20b/src/tools/msvc/Mkvcbuild.pm=0Aindex=20= 49614106dc..f680544e07=20100644=0A---=20a/src/tools/msvc/Mkvcbuild.pm=0A= +++=20b/src/tools/msvc/Mkvcbuild.pm=0A@@=20-33,9=20+33,9=20@@=20my=20= @unlink_on_exit;=0A=20=0A=20#=20Set=20of=20variables=20for=20modules=20= in=20contrib/=20and=20src/test/modules/=0A=20my=20$contrib_defines=20=3D=20= {=20'refint'=20=3D>=20'REFINT_VERBOSE'=20};=0A-my=20@contrib_uselibpq=20= =3D=20('dblink',=20'oid2name',=20'postgres_fdw',=20'vacuumlo');=0A-my=20= @contrib_uselibpgport=20=20=20=3D=20('oid2name',=20'vacuumlo');=0A-my=20= @contrib_uselibpgcommon=20=3D=20('oid2name',=20'vacuumlo');=0A+my=20= @contrib_uselibpq=20=3D=20('dblink',=20'oid2name',=20'pg_amcheck',=20= 'postgres_fdw',=20'vacuumlo');=0A+my=20@contrib_uselibpgport=20=20=20=3D=20= ('oid2name',=20'pg_amcheck',=20'vacuumlo');=0A+my=20= @contrib_uselibpgcommon=20=3D=20('oid2name',=20'pg_amcheck',=20= 'vacuumlo');=0A=20my=20$contrib_extralibs=20=20=20=20=20=20=3D=20undef;=0A= =20my=20$contrib_extraincludes=20=3D=20{=20'dblink'=20=3D>=20= ['src/backend']=20};=0A=20my=20$contrib_extrasource=20=3D=20{=0Adiff=20= --git=20a/src/tools/pgindent/typedefs.list=20= b/src/tools/pgindent/typedefs.list=0Aindex=20b1dec43f9d..a0dfe164cd=20= 100644=0A---=20a/src/tools/pgindent/typedefs.list=0A+++=20= b/src/tools/pgindent/typedefs.list=0A@@=20-101,6=20+101,7=20@@=20= AlterUserMappingStmt=0A=20AlteredTableInfo=0A=20AlternativeSubPlan=0A=20= AlternativeSubPlanState=0A+AmcheckOptions=0A=20= AnalyzeAttrComputeStatsFunc=0A=20AnalyzeAttrFetchFunc=0A=20= AnalyzeForeignTable_function=0A@@=20-499,6=20+500,7=20@@=20DSA=0A=20= DWORD=0A=20DataDumperPtr=0A=20DataPageDeleteStack=0A+DatabaseInfo=0A=20= DateADT=0A=20Datum=0A=20DatumTupleFields=0A@@=20-2084,6=20+2086,7=20@@=20= RelToCluster=0A=20RelabelType=0A=20Relation=0A=20RelationData=0A= +RelationInfo=0A=20RelationPtr=0A=20RelationSyncEntry=0A=20= RelcacheCallbackFunction=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A Content-Disposition: attachment; filename=v42-0003-Extending-PostgresNode-to-test-corruption.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v42-0003-Extending-PostgresNode-to-test-corruption.patch" Content-Transfer-Encoding: quoted-printable =46rom=20cd0e6068ce7e3d523050467fdf9bc0f21c02c7d8=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Tue,=202=20Feb=202021=2012:37:58=20-0800=0ASubject:=20[PATCH=20= v42=203/3]=20Extending=20PostgresNode=20to=20test=20corruption.=0A=0A= PostgresNode=20now=20has=20functions=20for=20overwriting=20relation=20= files=0Awith=20full=20or=20partial=20prior=20versions=20of=20those=20= files,=20creating=0Acorruption=20beyond=20merely=20twiddling=20the=20= bits=20of=20a=20heap=20relation=0Afile.=0A=0AAdding=20a=20regression=20= test=20for=20pg_amcheck=20based=20on=20this=20new=0Afunctionality.=0A---=0A= =20contrib/pg_amcheck/t/006_relfile_damage.pl=20=20=20=20|=20145=20= ++++++++++=0A=20src/test/modules/Makefile=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20|=20=20=201=20+=0A=20= src/test/modules/corruption/Makefile=20=20=20=20=20=20=20=20=20=20|=20=20= 16=20++=0A=20.../modules/corruption/t/001_corruption.pl=20=20=20=20|=20=20= 83=20++++++=0A=20src/test/perl/PostgresNode.pm=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20|=20261=20++++++++++++++++++=0A=205=20files=20= changed,=20506=20insertions(+)=0A=20create=20mode=20100644=20= contrib/pg_amcheck/t/006_relfile_damage.pl=0A=20create=20mode=20100644=20= src/test/modules/corruption/Makefile=0A=20create=20mode=20100644=20= src/test/modules/corruption/t/001_corruption.pl=0A=0Adiff=20--git=20= a/contrib/pg_amcheck/t/006_relfile_damage.pl=20= b/contrib/pg_amcheck/t/006_relfile_damage.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..45ad223531=0A---=20/dev/null=0A+++=20= b/contrib/pg_amcheck/t/006_relfile_damage.pl=0A@@=20-0,0=20+1,145=20@@=0A= +use=20strict;=0A+use=20warnings;=0A+=0A+use=20TestLib;=0A+use=20= Test::More=20tests=20=3D>=2022;=0A+use=20PostgresNode;=0A+=0A+my=20= ($node,=20$port);=0A+=0A+#=20Returns=20the=20name=20of=20the=20toast=20= relation=20associated=20with=20the=20named=20relation.=0A+#=0A+#=20= Assumes=20the=20test=20node=20is=20running=0A+sub=20relation_toast($$)=0A= +{=0A+=09my=20($dbname,=20$relname)=20=3D=20@_;=0A+=0A+=09my=20$rel=20=3D=20= $node->safe_psql($dbname,=20qq(=0A+=09=09SELECT=20ct.relname=0A+=09=09=09= FROM=20pg_catalog.pg_class=20cr,=20pg_catalog.pg_class=20ct=0A+=09=09=09= WHERE=20cr.oid=20=3D=20'$relname'::regclass=0A+=09=09=09=20=20AND=20= cr.reltoastrelid=20=3D=20ct.oid=0A+=09=09=09));=0A+=09return=20undef=20= unless=20defined=20$rel;=0A+=09return=20"pg_toast.$rel";=0A+}=0A+=0A+#=20= Test=20set-up=0A+$node=20=3D=20get_new_node('test');=0A+$node->init;=0A= +$node->start;=0A+$port=20=3D=20$node->port;=0A+=0A+#=20Load=20the=20= amcheck=20extension,=20upon=20which=20pg_amcheck=20depends=0A= +$node->safe_psql('postgres',=20q(CREATE=20EXTENSION=20amcheck));=0A+=0A= +#=20Create=20a=20table=20with=20a=20btree=20index.=20=20Use=20a=20= fillfactor=20for=20the=20table=20and=20index=0A+#=20that=20will=20allow=20= some=20fraction=20of=20updates=20to=20be=20on=20the=20original=20pages=20= and=20some=0A+#=20on=20new=20pages.=0A+#=0A+$node->safe_psql('postgres',=20= qq(=0A+create=20schema=20t;=0A+create=20table=20t.t1=20(id=20integer,=20= t=20text)=20with=20(fillfactor=3D75);=0A+alter=20table=20t.t1=20alter=20= column=20t=20set=20storage=20external;=0A+insert=20into=20t.t1=20select=20= gs,=20repeat('x',gs)=20from=20generate_series(9990,10000)=20gs;=0A= +create=20index=20t1_idx=20on=20t.t1=20(id)=20with=20(fillfactor=3D75);=0A= +));=0A+=0A+my=20$toastrel=20=3D=20relation_toast('postgres',=20't.t1');=0A= +=0A+#=20Flush=20relation=20files=20to=20disk=20and=20take=20snapshots=20= of=20the=20toast=20and=20index=0A+#=0A+$node->restart;=0A= +$node->take_relfile_snapshot_minimal('postgres',=20'idx',=20= 't.t1_idx');=0A+$node->take_relfile_snapshot_minimal('postgres',=20= 'toast',=20$toastrel);=0A+=0A+#=20Insert=20new=20data=20into=20the=20= table=20and=20index=0A+#=0A+$node->safe_psql('postgres',=20qq(=0A+insert=20= into=20t.t1=20select=20gs,=20repeat('y',gs)=20from=20= generate_series(10001,10100)=20gs;=0A+));=0A+=0A+#=20Revert=20index.=20=20= The=20reverted=20snapshot=20file=20is=20not=20corrupt,=20but=20it=20also=0A= +#=20does=20not=20match=20the=20current=20contents=20of=20the=20table.=0A= +#=0A+$node->stop;=0A+$node->revert_to_snapshot('idx');=0A+=0A+#=20= Restart=20the=20node=20and=20check=20table=20and=20index=20with=20= varying=20options.=0A+#=0A+$node->start;=0A+=0A+#=20Checks=20which=20do=20= not=20reconcile=20the=20index=20and=20table=20via=20--heapallindexed=20= will=0A+#=20not=20notice=20any=20problems=0A+#=0A+$node->command_like(=0A= +=09[=20'pg_amcheck',=20'--quiet',=20'-p',=20$port,=20'-r',=20= 'postgres.t.*'=20],=0A+=09qr/^$/,=0A+=09'pg_amcheck=20reverted=20index=20= at=20default=20checking=20level');=0A+=0A+$node->command_like(=0A+=09[=20= 'pg_amcheck',=20'--quiet',=20'-p',=20$port,=20'-r',=20'postgres.t.*'=20= ],=0A+=09qr/^$/,=0A+=09'pg_amcheck=20reverted=20index=20at=20default=20= checking=20level');=0A+=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20= '--quiet',=20'-p',=20$port,=20'-r',=20'postgres.t.*',=20'--parent-check'=20= ],=0A+=09qr/^$/,=0A+=09'pg_amcheck=20reverted=20index=20with=20= --parent-check');=0A+=0A+$node->command_like(=0A+=09[=20'pg_amcheck',=20= '--quiet',=20'-p',=20$port,=20'-r',=20'postgres.t.*',=20'--rootdescend'=20= ],=0A+=09qr/^$/,=0A+=09'pg_amcheck=20reverted=20index=20with=20= --rootdescend');=0A+=0A+#=20Checks=20which=20do=20reconcile=20the=20= index=20and=20table=20via=20--heapallindexed=20will=0A+#=20notice=20the=20= mismatch=20in=20their=20contents=0A+#=0A+$node->command_checks_all(=0A+=09= [=20'pg_amcheck',=20'--quiet',=20'-p',=20$port,=20'-r',=20= 'postgres.t.*',=20'--heapallindexed'=20],=0A+=092,=0A+=09[=20qr/heap=20= tuple=20.*=20from=20table=20"t1"=20lacks=20matching=20index=20tuple=20= within=20index=20"t1_idx"/=20],=0A+=09[=20],=0A+=09'pg_amcheck=20= reverted=20index=20with=20--heapallindexed');=0A+=0A= +$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20'--quiet',=20'-p',=20= $port,=20'-r',=20'postgres.t.*',=20'--heapallindexed',=20'--rootdescend'=20= ],=0A+=092,=0A+=09[=20qr/heap=20tuple=20.*=20from=20table=20"t1"=20lacks=20= matching=20index=20tuple=20within=20index=20"t1_idx"/=20],=0A+=09[=20],=0A= +=09'pg_amcheck=20reverted=20index=20with=20--heapallindexed=20= --rootdescend');=0A+=0A+#=20Revert=20the=20toast.=20=20The=20reverted=20= toast=20table=20is=20not=20corrupt,=20but=20it=20does=20not=0A+#=20have=20= entries=20for=20all=20toast=20pointers=20in=20the=20main=20table=0A+#=0A= +$node->stop;=0A+$node->revert_to_snapshot('toast');=0A+=0A+#=20Restart=20= the=20node=20and=20check=20table=20and=20toast=20with=20varying=20= options.=20=20When=0A+#=20checking=20the=20toast=20pointers,=20we=20may=20= get=20errors=20produced=20by=20verify_heapam,=20but=0A+#=20we=20may=20= also=20get=20errors=20from=20failure=20to=20read=20toast=20blocks=20that=20= are=20beyond=20the=0A+#=20end=20of=20the=20toast=20table,=20of=20the=20= form=20/ERROR:=20=20could=20not=20read=20block/.=20=20To=20avoid=0A+#=20= having=20a=20brittle=20test,=20we=20accept=20any=20error=20message.=0A+#=0A= +$node->start;=0A+=0A+$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20= '--quiet',=20'-p',=20$port,=20'-r',=20$toastrel=20],=0A+=090,=0A+=09[=20= qr/^$/=20],=0A+=09[=20],=0A+=09'pg_amcheck=20reverted=20toast=20table');=0A= +=0A+$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20'--quiet',=20= '-p',=20$port,=20'-r',=20'postgres.t.*',=20'--exclude-toast-pointers'=20= ],=0A+=090,=0A+=09[=20qr/^$/=20],=0A+=09[=20],=0A+=09'pg_amcheck=20with=20= reverted=20toast=20using=20--exclude-toast-pointers');=0A+=0A= +$node->command_checks_all(=0A+=09[=20'pg_amcheck',=20'--quiet',=20'-p',=20= $port,=20'-r',=20'postgres.t.*'=20],=0A+=092,=0A+=09[=20qr/.+/=20],=09=09= =09#=20Any=20non-empty=20error=20message=20is=20acceptable=0A+=09[=20],=0A= +=09'pg_amcheck=20with=20reverted=20toast=20and=20default=20checking');=0A= diff=20--git=20a/src/test/modules/Makefile=20b/src/test/modules/Makefile=0A= index=205391f461a2..c92d1702b4=20100644=0A---=20= a/src/test/modules/Makefile=0A+++=20b/src/test/modules/Makefile=0A@@=20= -7,6=20+7,7=20@@=20include=20$(top_builddir)/src/Makefile.global=0A=20= SUBDIRS=20=3D=20\=0A=20=09=09=20=20brin=20\=0A=20=09=09=20=20commit_ts=20= \=0A+=09=09=20=20corruption=20\=0A=20=09=09=20=20delay_execution=20\=0A=20= =09=09=20=20dummy_index_am=20\=0A=20=09=09=20=20dummy_seclabel=20\=0A= diff=20--git=20a/src/test/modules/corruption/Makefile=20= b/src/test/modules/corruption/Makefile=0Anew=20file=20mode=20100644=0A= index=200000000000..ba461c645d=0A---=20/dev/null=0A+++=20= b/src/test/modules/corruption/Makefile=0A@@=20-0,0=20+1,16=20@@=0A+#=20= src/test/modules/corruption/Makefile=0A+=0A+#=20EXTRA_INSTALL=20=3D=20= contrib/pg_amcheck=0A+=0A+TAP_TESTS=20=3D=201=0A+=0A+ifdef=20USE_PGXS=0A= +PG_CONFIG=20=3D=20pg_config=0A+PGXS=20:=3D=20$(shell=20$(PG_CONFIG)=20= --pgxs)=0A+include=20$(PGXS)=0A+else=0A+subdir=20=3D=20= src/test/modules/corruption=0A+top_builddir=20=3D=20../../../..=0A= +include=20$(top_builddir)/src/Makefile.global=0A+include=20= $(top_srcdir)/contrib/contrib-global.mk=0A+endif=0Adiff=20--git=20= a/src/test/modules/corruption/t/001_corruption.pl=20= b/src/test/modules/corruption/t/001_corruption.pl=0Anew=20file=20mode=20= 100644=0Aindex=200000000000..ae4a262e06=0A---=20/dev/null=0A+++=20= b/src/test/modules/corruption/t/001_corruption.pl=0A@@=20-0,0=20+1,83=20= @@=0A+use=20strict;=0A+use=20warnings;=0A+=0A+use=20TestLib;=0A+use=20= Test::More=20tests=20=3D>=2010;=0A+use=20PostgresNode;=0A+=0A+my=20$node=20= =3D=20get_new_node('test');=0A+$node->init;=0A+$node->start;=0A+=0A+#=20= Create=20something=20non-trivial=20for=20the=20first=20snapshot=0A= +$node->safe_psql('postgres',=20qq(=0A+create=20table=20t1=20(id=20= integer,=20short_text=20text,=20long_text=20text);=0A+insert=20into=20t1=20= (id,=20short_text,=20long_text)=0A+=09(select=20gs,=20'foo',=20= repeat('x',=20gs)=0A+=09=09from=20generate_series(1,10000)=20gs);=0A= +create=20unique=20index=20idx1=20on=20t1=20(id,=20short_text);=0A= +vacuum=20freeze;=0A+));=0A+=0A+#=20Flush=20relation=20files=20to=20disk=20= and=20take=20snapshot=20of=20them=0A+$node->restart;=0A= +$node->take_relfile_snapshot('postgres',=20'snap1',=20'public.t1');=0A+=0A= +#=20Update=20data=20in=20the=20table,=20toast=20table,=20and=20index=0A= +$node->safe_psql('postgres',=20qq(=0A+update=20t1=20set=0A+=09= short_text=20=3D=20'bar',=0A+=09long_text=20=3D=20repeat('y',=20id);=0A= +));=0A+=0A+#=20Flush=20relation=20files=20to=20disk=20and=20take=20= second=20snapshot=0A+$node->restart;=0A= +$node->take_relfile_snapshot('postgres',=20'snap2',=20'public.t1');=0A+=0A= +#=20Revert=20the=20first=20page=20of=20t1=20using=20a=20torn=20= snapshot.=20=20This=20should=20be=20a=20partial=0A+#=20and=20corrupt=20= reverting=20of=20the=20update.=0A+$node->stop;=0A= +$node->revert_to_torn_relfile_snapshot('snap1',=208192);=0A+=0A+#=20= Restart=20the=20node=20and=20count=20the=20number=20of=20rows=20in=20t1=20= with=20the=20original=0A+#=20(pre-update)=20values.=20=20It=20should=20= not=20be=20zero,=20but=20nor=20will=20it=20be=20the=20full=0A+#=2010000.=0A= +$node->start;=0A+my=20($old,=20$new,=20$oldtoast,=20$newtoast)=20=3D=20= counts();=0A+ok($old=20>=200=20&&=20$old=20<=2010000,=20"Torn=20snapshot=20= reverts=20some=20of=20the=20main=20updates");=0A+ok($new=20>=200=20&&=20= $new=20<=3D=2010000,=20"Torn=20snapshot=20retains=20some=20of=20the=20= main=20updates");=0A+=0A+#=20Revert=20t1=20fully=20to=20the=20first=20= snapshot.=20=20This=20should=20fully=20restore=20the=0A+#=20original=20= (pre-update)=20values.=0A+$node->stop;=0A= +$node->revert_to_snapshot('snap1');=0A+=0A+#=20Restart=20the=20node=20= and=20verify=20only=20old=20values=20remain=0A+$node->start;=0A+($old,=20= $new,=20$oldtoast,=20$newtoast)=20=3D=20counts();=0A+is($old,=2010000,=20= "Full=20snapshot=20restores=20all=20the=20old=20main=20values");=0A= +is($oldtoast,=2010000,=20"Full=20snapshot=20restores=20all=20the=20old=20= toast=20values");=0A+is($new,=200,=20"Full=20snapshot=20reverts=20all=20= the=20new=20main=20values");=0A+is($newtoast,=200,=20"Full=20snapshot=20= reverts=20all=20the=20new=20toast=20values");=0A+=0A+#=20Restore=20t1=20= fully=20to=20the=20second=20snapshot.=20=20This=20should=20fully=20= restore=20the=0A+#=20new=20(post-update)=20values.=0A+$node->stop;=0A= +$node->revert_to_snapshot('snap2');=0A+=0A+#=20Restart=20the=20node=20= and=20verify=20only=20new=20values=20remain=0A+$node->start;=0A+($old,=20= $new,=20$oldtoast,=20$newtoast)=20=3D=20counts();=0A+is($old,=200,=20= "Full=20snapshot=20reverts=20all=20the=20old=20main=20values");=0A= +is($oldtoast,=200,=20"Full=20snapshot=20reverts=20all=20the=20old=20= toast=20values");=0A+is($new,=2010000,=20"Full=20snapshot=20restores=20= all=20the=20new=20main=20values");=0A+is($newtoast,=2010000,=20"Full=20= snapshot=20restores=20all=20the=20new=20toast=20values");=0A+=0A+sub=20= counts=20{=0A+=09return=20map=20{=0A+=09=09$node->safe_psql('postgres',=20= qq(select=20count(*)=20from=20t1=20where=20$_))=0A+=09}=20("short_text=20= =3D=20'foo'",=0A+=09=20=20=20"short_text=20=3D=20'bar'",=0A+=09=20=20=20= "long_text=20~=20'x'",=0A+=09=20=20=20"long_text=20~=20'y'");=0A+}=0A= diff=20--git=20a/src/test/perl/PostgresNode.pm=20= b/src/test/perl/PostgresNode.pm=0Aindex=209667f7667e..d470af93c5=20= 100644=0A---=20a/src/test/perl/PostgresNode.pm=0A+++=20= b/src/test/perl/PostgresNode.pm=0A@@=20-2225,6=20+2225,267=20@@=20sub=20= pg_recvlogical_upto=0A=20=0A=20=3Dback=0A=20=0A+=3Dhead1=20DATABASE=20= CORRUPTION=20METHODS=0A+=0A+=3Dover=0A+=0A+=3Ditem=20= $node->relfile_snapshot_repository()=0A+=0A+The=20path=20to=20the=20= parent=20directory=20of=20all=20directories=20storing=20snapshots=20of=0A= +relation=20backing=20files.=0A+=0A+=3Dcut=0A+=0A+sub=20= relfile_snapshot_repository=0A+{=0A+=09my=20($self)=20=3D=20@_;=0A+=09my=20= $snaprepo=20=3D=20join('/',=20$self->basedir,=20'snapshot');=0A+=09= unless=20(-d=20$snaprepo)=0A+=09{=0A+=09=09mkdir=20$snaprepo=0A+=09=09=09= or=20$!{EEXIST}=0A+=09=09=09or=20BAIL_OUT("could=20not=20create=20= snapshot=20repository=20directory=20\"$snaprepo\":=20$!");=0A+=09}=0A+=09= return=20$snaprepo;=0A+}=0A+=0A+=3Dpod=0A+=0A+=3Ditem=20= $node->relfile_snapshot_directory(snapname)=0A+=0A+The=20path=20to=20the=20= directory=20for=20storing=20the=20named=20snapshot.=0A+=0A+=3Dcut=0A+=0A= +sub=20relfile_snapshot_directory=0A+{=0A+=09my=20($self,=20$snapname)=20= =3D=20@_;=0A+=0A+=09join("/",=20$self->relfile_snapshot_repository(),=20= $snapname);=0A+}=0A+=0A+=3Dpod=0A+=0A+=3Ditem=20= $node->take_relfile_snapshot($self,=20$dbname,=20$snapname,=20@relnames)=0A= +=0A+Makes=20a=20copy=20of=20the=20files=20backing=20the=20relations=20= B<@relname>,=20the=20associated=0A+toast=20relations=20(if=20any),=20and=20= all=20associated=20indexes=20(if=20any).=20=20No=20attempt=20is=0A+made=20= to=20flush=20these=20files=20to=20disk,=20meaning=20the=20snapshot=20= taken=20could=20be=20stale=0A+unless=20the=20caller=20ensures=20these=20= files=20have=20been=20flushed=20prior=20to=20calling.=0A+=0A+Dies=20on=20= failure=20to=20invoke=20psql.=0A+=0A+Dies=20on=20missing=20relations.=0A= +=0A+Dies=20if=20the=20given=20B<$snapname>=20is=20already=20in=20use.=0A= +=0A+=3Dcut=0A+=0A+=3Dpod=0A+=0A+=3Ditem=20= $node->take_relfile_snapshot_minimal($self,=20$dbname,=20$snapname,=20= @relnames)=0A+=0A+Makes=20a=20copy=20of=20the=20files=20backing=20the=20= relations=20B<@relnames>.=20=20No=20attempt=20is=20made=0A+to=20flush=20= these=20files=20to=20disk,=20meaning=20the=20snapshot=20taken=20could=20= be=20stale=20unless=20the=0A+caller=20ensures=20these=20files=20have=20= been=20flushed=20prior=20to=20calling.=0A+=0A+Dies=20on=20failure=20to=20= invoke=20psql.=0A+=0A+Dies=20on=20missing=20relation.=0A+=0A+Dies=20if=20= the=20given=20B<$snapname>=20is=20already=20in=20use.=0A+=0A+=3Dcut=0A+=0A= +sub=20take_relfile_snapshot=0A+{=0A+=09my=20($self,=20$dbname,=20= $snapname,=20@relnames)=20=3D=20@_;=0A+=09= $self->take_relfile_snapshot_helper($dbname,=20$snapname,=201,=20= @relnames);=0A+}=0A+=0A+sub=20take_relfile_snapshot_minimal=0A+{=0A+=09= my=20($self,=20$dbname,=20$snapname,=20@relnames)=20=3D=20@_;=0A+=09= $self->take_relfile_snapshot_helper($dbname,=20$snapname,=200,=20= @relnames);=0A+}=0A+=0A+sub=20take_relfile_snapshot_helper=0A+{=0A+=09my=20= ($self,=20$dbname,=20$snapname,=20$extended,=20@relnames)=20=3D=20@_;=0A= +=0A+=09croak=20"dbname=20must=20be=20specified"=20unless=20defined=20= $dbname;=0A+=09croak=20"relnames=20must=20be=20defined"=20unless=20= scalar(grep=20{=20defined=20$_=20}=20@relnames);=0A+=09croak=20"snapname=20= must=20be=20specified"=20unless=20defined=20$snapname;=0A+=09croak=20= "snapname=20must=20be=20unique"=20if=20exists=20= $self->{snapshot}->{$snapname};=0A+=0A+=09my=20$pgdata=20=3D=20= $self->data_dir;=0A+=09my=20$snapdir=20=3D=20= $self->relfile_snapshot_directory($snapname);=0A+=09croak=20"snapname=20= directory=20name=20already=20in=20use:=20$snapdir"=20if=20(-e=20= $snapdir);=0A+=09mkdir=20$snapdir=0A+=09=09or=20BAIL_OUT("could=20not=20= create=20snapshot=20directory=20\"$snapdir\":=20$!");=0A+=0A+=09my=20= @relpaths=20=3D=20map=20{=0A+=09=09$self->safe_psql($dbname,=0A+=09=09=09= qq(SELECT=20pg_relation_filepath('$_')));=0A+=09}=20@relnames;=0A+=0A+=09= my=20(@toastpaths,=20@idxpaths);=0A+=09if=20($extended)=0A+=09{=0A+=09=09= for=20my=20$relname=20(@relnames)=0A+=09=09{=0A+=09=09=09push=20= (@toastpaths,=20grep=20/\w/,=20split(/(?:\s*\r?\n\s*)+/,=20= $self->safe_psql($dbname,=0A+=09=09=09=09qq(SELECT=20= pg_relation_filepath(c.reltoastrelid)=0A+=09=09=09=09=09FROM=20= pg_catalog.pg_class=20c=0A+=09=09=09=09=09WHERE=20c.oid=20=3D=20= '$relname'::regclass=0A+=09=09=09=09=09AND=20c.reltoastrelid=20!=3D=20= 0::oid))));=0A+=09=09=09push=20(@idxpaths,=20grep=20/\w/,=20= split(/(?:\s*\r?\n\s*)+/,=20$self->safe_psql($dbname,=0A+=09=09=09=09= qq(SELECT=20pg_relation_filepath(i.indexrelid)=0A+=09=09=09=09=09FROM=20= pg_catalog.pg_index=20i=0A+=09=09=09=09=09WHERE=20i.indrelid=20=3D=20= '$relname'::regclass))));=0A+=09=09}=0A+=09}=0A+=0A+=09= $self->{snapshot}->{$snapname}=20=3D=20{};=0A+=09for=20my=20$path=20= (@relpaths,=20grep=20{=20defined($_)=20}=20@toastpaths,=20@idxpaths)=0A+=09= {=0A+=09=09croak=20"file=20backing=20relation=20is=20missing:=20= $pgdata/$path"=20unless=20-f=20"$pgdata/$path";=0A+=09=09= copy_file($snapdir,=20$pgdata,=200,=20$path);=0A+=09=09= $self->{snapshot}->{$snapname}->{$path}=20=3D=201;=0A+=09}=0A+}=0A+=0A= +=3Dpod=0A+=0A+=3Ditem=20$node->revert_to_snapshot($self,=20$snapname)=0A= +=0A+Overwrites=20the=20database's=20relation=20files=20with=20files=20= previously=20saved=20in=0A+B<$snapname>.=0A+=0A+Dies=20if=20the=20given=20= B<$snapname>=20does=20not=20exist.=0A+=0A+=3Dcut=0A+=0A+=3Dpod=0A+=0A= +=3Ditem=20$node->revert_to_torn_relfile_snapshot($self,=20$snapname,=20= $bytes)=0A+=0A+Partially=20overwrites=20the=20database's=20relation=20= files=20using=20prefixes=20of=20the=20given=0A+number=20of=20bytes=20= from=20the=20files=20saved=20in=20B<$snapname>.=20=20If=20B<$bytes>=20is=0A= +negative,=20uses=20suffixes=20of=20the=20given=20byte=20length=20rather=20= than=20prefixes.=0A+=0A+If=20B<$bytes>=20is=20null,=20replaces=20the=20= database's=20relation=20files=20using=20the=20saved=0A+files=20in=20the=20= B<$snapname>,=20which=20unlike=20for=20non-undef=20values,=20means=20the=20= file=0A+may=20become=20shorter=20if=20the=20saved=20file=20is=20shorter=20= than=20the=20current=20file.=0A+=0A+=3Dcut=0A+=0A+sub=20= revert_to_snapshot=0A+{=0A+=09my=20($self,=20$snapname)=20=3D=20@_;=0A+=09= $self->revert_to_torn_relfile_snapshot($snapname,=20undef);=0A+}=0A+=0A= +sub=20revert_to_torn_relfile_snapshot=0A+{=0A+=09my=20($self,=20= $snapname,=20$bytes)=20=3D=20@_;=0A+=0A+=09croak=20"no=20such=20= snapshot"=20unless=20exists=20$self->{snapshot}->{$snapname};=0A+=0A+=09= my=20$pgdata=20=3D=20$self->data_dir;=0A+=09my=20$snaprepo=20=3D=20= join('/',=20$self->relfile_snapshot_repository,=20$snapname);=0A+=09= croak=20"snapname=20directory=20missing:=20$snaprepo"=20unless=20(-d=20= $snaprepo);=0A+=0A+=09if=20(defined=20$bytes)=0A+=09{=0A+=09=09= tear_file($pgdata,=20$snaprepo,=20$bytes,=20$_)=0A+=09=09=09for=20(keys=20= %{$self->{snapshot}->{$snapname}});=0A+=09}=0A+=09else=0A+=09{=0A+=09=09= copy_file($pgdata,=20$snaprepo,=201,=20$_)=0A+=09=09=09for=20(keys=20= %{$self->{snapshot}->{$snapname}});=0A+=09}=0A+}=0A+=0A+sub=20copy_file=0A= +{=0A+=09my=20($dstdir,=20$srcdir,=20$overwrite,=20$path)=20=3D=20@_;=0A= +=0A+=09croak=20"No=20such=20directory:=20$dstdir"=20unless=20-d=20= $dstdir;=0A+=09croak=20"No=20such=20directory:=20$srcdir"=20unless=20-d=20= $srcdir;=0A+=0A+=09foreach=20my=20$part=20(split(m{/},=20$path))=0A+=09{=0A= +=09=09my=20$srcpart=20=3D=20"$srcdir/$part";=0A+=09=09my=20$dstpart=20=3D= =20"$dstdir/$part";=0A+=0A+=09=09if=20(-d=20$srcpart)=0A+=09=09{=0A+=09=09= =09$srcdir=20=3D=20$srcpart;=0A+=09=09=09$dstdir=20=3D=20$dstpart;=0A+=09= =09=09die=20"$dstdir=20is=20in=20the=20way"=20if=20(-e=20$dstdir=20&&=20= !=20-d=20$dstdir);=0A+=09=09=09unless=20(-d=20$dstdir)=0A+=09=09=09{=0A+=09= =09=09=09mkdir=20$dstdir=0A+=09=09=09=09=09or=20BAIL_OUT("could=20not=20= create=20directory=20\"$dstdir\":=20$!");=0A+=09=09=09}=0A+=09=09}=0A+=09= =09elsif=20(-f=20$srcpart)=0A+=09=09{=0A+=09=09=09die=20"$dstdir/$part=20= is=20in=20the=20way"=20if=20(!$overwrite=20&&=20-e=20"$dstdir/$part");=0A= +=0A+=09=09=09File::Copy::copy($srcpart,=20"$dstdir/$part");=0A+=09=09}=0A= +=09}=0A+}=0A+=0A+sub=20tear_file=0A+{=0A+=09my=20($dstdir,=20$srcdir,=20= $bytes,=20$path)=20=3D=20@_;=0A+=0A+=09croak=20"No=20such=20directory:=20= $dstdir"=20unless=20-d=20$dstdir;=0A+=09croak=20"No=20such=20directory:=20= $srcdir"=20unless=20-d=20$srcdir;=0A+=0A+=09my=20$srcfile=20=3D=20= "$srcdir/$path";=0A+=09my=20$dstfile=20=3D=20"$dstdir/$path";=0A+=0A+=09= croak=20"No=20such=20file:=20$srcfile"=20unless=20-f=20$srcfile;=0A+=09= croak=20"No=20such=20file:=20$dstfile"=20unless=20-f=20$dstfile;=0A+=0A+=09= my=20($srcfh,=20$dstfh);=0A+=09open($srcfh,=20'<',=20$srcfile)=20or=20= die=20"Cannot=20read=20$srcfile:=20$!";=0A+=09open($dstfh,=20'+<',=20= $dstfile)=20or=20die=20"Cannot=20modify=20$dstfile:=20$!";=0A+=09= binmode($srcfh);=0A+=09binmode($dstfh);=0A+=0A+=09my=20$buffer;=0A+=09if=20= ($bytes=20<=200)=0A+=09{=0A+=09=09$bytes=20*=3D=20-1;=09=09#=20Easier=20= to=20use=20positive=20value=0A+=09=09my=20$srcsize=20=3D=20= (stat($srcfh))[7];=0A+=09=09my=20$offset=20=3D=20$srcsize=20-=20$bytes;=0A= +=09=09seek($srcfh,=20$offset,=200);=0A+=09=09seek($dstfh,=20$offset,=20= 0);=0A+=09=09sysread($srcfh,=20$buffer,=20$bytes);=0A+=09=09= syswrite($dstfh,=20$buffer,=20$bytes);=0A+=09}=0A+=09else=0A+=09{=0A+=09=09= seek($srcfh,=200,=200);=0A+=09=09seek($dstfh,=200,=200);=0A+=09=09= sysread($srcfh,=20$buffer,=20$bytes);=0A+=09=09syswrite($dstfh,=20= $buffer,=20$bytes);=0A+=09}=0A+=0A+=09close($srcfh);=0A+=09= close($dstfh);=0A+}=0A+=0A+=3Dpod=0A+=0A+=3Dback=0A+=0A=20=3Dcut=0A=20=0A= =201;=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 =E2=80=94 Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --Apple-Mail=_FA474A7B-4CCE-436F-84A2-A5706E3B0E9A--