Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lUZvW-0007z4-JN for pgsql-hackers@arkaria.postgresql.org; Thu, 08 Apr 2021 19:02:50 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lUZvV-0006jU-HF for pgsql-hackers@arkaria.postgresql.org; Thu, 08 Apr 2021 19:02:49 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lUZvV-0006gs-3C for pgsql-hackers@lists.postgresql.org; Thu, 08 Apr 2021 19:02:49 +0000 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lUZvN-00006T-3J for pgsql-hackers@postgresql.org; Thu, 08 Apr 2021 19:02:48 +0000 Received: by mail-pj1-x1031.google.com with SMTP id ha17so1656225pjb.2 for ; Thu, 08 Apr 2021 12:02:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=RzfmJH3kJfPfPQ8UVitNMfGa9QFbQe4JFRw0vrOb1j0=; b=eacMGB1tjEIhfD/EwwfXdAveUJ+Zvz4VS9uAacKXX44hIPzdz1Za7AJEisW50M+AGh 3r3aICmFazxuqSH6XyTKZd4KYKU4HR/hdHBRXFHmAqNyc2XZDsgJQAytoKSrgOKvsQp1 LJGDIV2p48dt1TqCPsrvZavqUSNoJvLuHom+7nPEsSJ3Auj341fjSb9cusH788vjx4EJ QvT2OzUdI1o5+T21LNlz5GfNTFXNpT1nbAqQC7H987X1at2WQ6MmbXJWEEd1X08fyUKd BrEGOF0S4s8qJKE7G44vrLM4vdXqGI9HNW20OVasblJltiICo7kLplJ5GK4SAsuUu/VD p3YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=RzfmJH3kJfPfPQ8UVitNMfGa9QFbQe4JFRw0vrOb1j0=; b=cADIhGkbwXppEus2HkJ6Ohq08rn9N9T1sGE5SZId3XkBnyQiTlI8Dj0kYljvpqhm0U 9DKQntdmrJsc0WOmCTCAYiAwgAvqWxFVrUvtD1EX38XrN3H6+O3VZCEn07n6ZFKXDb3l lhVlOGLLhwRfRTuG1KbVvbGecOJUc8d9NDvXG/SWAdO0KgF7YqgJTi9D3gEp1ZVnAc5o jVOZZURhgnTNgjjjK8kOA5sg0VhqbSTu8I0nKLDqKSlyL85KkGXL/eJFNOQmxyrnHKf8 Mz1XKMyYJSdz4Z7I1hbWIbk8b4WWChQ40/9scTPdmuV+Ca2WXfs/31sFzk5rDQiitf/n 5Aow== X-Gm-Message-State: AOAM5321thb7PGZ9lfcMdyfRv+1/JVzCBiRxAb4ibjHCqsPZu3fbDpd2 q0NV3lG78e/SgXclfWp1iwZUvy+1W7ST1S6pc8I0y+4nWccekee9ksWMFmAoiVhjoG41ZWuQ0Kq jFACEqW13ph2HT9Ex6911U4wOMkxC32qpHFC/dC9ohsJsTRppxXWDQpRs1pBTr+kzsfmj5A8wa4 mGWxcGyJdpQ6X1WL6Zi0eml6exzmTjt/wAVTcV X-Google-Smtp-Source: ABdhPJzKJh2JKyzZ7UpRG2dtMyNEmKrffZdjQaQjtDY8CXftBKnBq18qXT6+tPyDQLiPM52tzvAtoQ== X-Received: by 2002:a17:902:ce88:b029:e6:3a3c:2f65 with SMTP id f8-20020a170902ce88b02900e63a3c2f65mr9182169plg.66.1617908558630; Thu, 08 Apr 2021 12:02:38 -0700 (PDT) Received: from laptop280-ma-us.hitronhub.home (24-113-193-150.wavecable.com. [24.113.193.150]) by smtp.gmail.com with ESMTPSA id x69sm189962pfd.161.2021.04.08.12.02.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Apr 2021 12:02:37 -0700 (PDT) From: Mark Dilger Message-Id: Content-Type: multipart/mixed; boundary="Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: pg_amcheck contrib application Date: Thu, 8 Apr 2021 12:02:36 -0700 In-Reply-To: Cc: Tom Lane , Noah Misch , Peter Geoghegan , Thomas Munro , "Andrey M. Borodin" , Stephen Frost , Michael Paquier , Amul Sul , Dilip Kumar , Andres Freund , PostgreSQL-development To: Robert Haas References: <9E76E46A-48B2-4869-BD0C-422204C1F767@enterprisedb.com> <20210313184612.GA3195145@rfd.leadboat.com> <930621E5-5D10-456D-8CE4-CF629A6C4669@enterprisedb.com> <20210313191841.GB3195145@rfd.leadboat.com> <3012106.1615827854@sss.pgh.pa.us> <3053552.1615834640@sss.pgh.pa.us> <4F10540A-9022-49EE-A6FC-7FE90797793E@enterprisedb.com> <0D48534A-F511-405D-8645-84B0A1DA1E11@enterprisedb.com> <6915B9B2-B551-4DBC-89DB-C12D9AF985B8@enterprisedb.com> <23192440-3B8A-4D82-8C24-9D8161BAAB01@enterprisedb.com> <1F1A6246-854F-47F0-A179-A3CD9F05356B@enterprisedb.com> X-Mailer: Apple Mail (2.3608.120.23.2.4) X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Apr 7, 2021, at 1:16 PM, Robert Haas wrote: >=20 > On Sun, Apr 4, 2021 at 8:02 PM Mark Dilger = wrote: >> v18-0001 - Finishes work started in commit 3b6c1259f9 that was = overlooked owing to how I had separated the changes in v17-0002 vs. = v17-0003 >=20 > Committed. Thank you. >> v18-0002 - Postpones the toast checks for a page until after the main = table page lock is released >=20 > Committed, but I changed list_free() to list_free_deep() to avoid a > memory leak, and I revised the commit message to mention the important > point that we need to avoid following TOAST pointers from > potentially-prunable tuples. Thank you, and yes, I agree with that change. >> v18-0003 - Improves the corruption messages in ways already discussed = earlier in this thread. Changes the tests to expect the new messages, = but adds no new checks >=20 > Kibitizing your message wording: >=20 > "toast value %u chunk data is null" -> "toast value %u chunk %d has > null data". We can mention the chunk number this way. Changed. > "toast value %u corrupt extended chunk has invalid varlena header: %0x > (sequence number %d)" -> "toast value %u chunk %d has invalid varlena > header %0x". We can be more consistent about how we incorporate the > chunk number into the text, and we don't really need to include the > word corrupt, because all of these are corruption complaints, and I > think it looks better without the colon. Changed. > "toast value %u chunk sequence number %u does not match the expected > sequence number %u" -> "toast value %u contains chunk %d where chunk > %d was expected". Shorter. Uses %d for a sequence number instead of > %u, which I think is correct -- anyway we should have them all one way > or all the other. I think I'd rather ditch the "sequence number" > technology and just talk about "chunk %d" or whatever. I don't agree with this one. I do agree with changing the message, but = not to the message you suggest. Imagine a toasted attribute with 18 chunks numbered [0..17]. Then we = update the toast to have only 6 chunks numbered [0..5] except we = corruptly keep chunks numbered [12..17] in the toast table. We'd rather = see a report like this: # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 6 has sequence number 12, but expected = sequence number 6 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 7 has sequence number 13, but expected = sequence number 7 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 8 has sequence number 14, but expected = sequence number 8 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 9 has sequence number 15, but expected = sequence number 9 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 10 has sequence number 16, but expected = sequence number 10 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 11 has sequence number 17, but expected = sequence number 11 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 was expected to end at chunk 6, but ended at = chunk 12 than one like this: # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 contains chunk 12 where chunk 6 was expected # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 contains chunk 13 where chunk 7 was expected # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 contains chunk 14 where chunk 8 was expected # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 contains chunk 15 where chunk 9 was expected # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 contains chunk 16 where chunk 10 was expected # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 contains chunk 17 where chunk 11 was expected # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 was expected to end at chunk 6, but ended at = chunk 12 because saying the toast value ended at "chunk 12" after saying that it = contains "chunk 17" is contradictory. You need the distinction between = the chunk number and the chunk sequence number, since in corrupt = circumstances they may not be the same. > "toast value %u chunk sequence number %u exceeds the end chunk > sequence number %u" -> "toast value %u chunk %d follows last expected > chunk %d" Changed. > "toast value %u chunk size %u differs from the expected size %u" -> > "toast value %u chunk %d has size %u, but expected size %u" Changed. > Other complaints: >=20 > Your commit message fails to mention the addition of > VARATT_EXTERNAL_GET_POINTER, which is a significant change/bug fix > unrelated to message wording. Right you are. > It feels like we have a non-minimal number of checks/messages for the > series of toast chunks. I think that if we find a chunk after the last > chunk we were expecting to find (curchunk > endchunk) and you also get > a message if we have the wrong number of chunks in total (chunkno !=3D > (endchunk + 1)). Now maybe I'm wrong, but if the first message > triggers, it seems like the second message must also trigger. Is that > wrong? If not, maybe we can get rid of the first one entirely? That's > such a small change I think we could include it in this same patch, if > it's a correct idea. Motivated by discussions we had off-list, I dug into this one. Purely as manual testing, and not part of the patch, I hacked the = backend a bit to allow direct modification of the toast table. After = corrupting the toast with the following bit of SQL: WITH chunk_limit AS ( SELECT chunk_id, MAX(chunk_seq) AS maxseq FROM $toastname GROUP BY chunk_id) INSERT INTO $toastname (chunk_id, chunk_seq, chunk_data) (SELECT t.chunk_id, t.chunk_seq + cl.maxseq + CASE = WHEN t.chunk_seq < 3 THEN 1 ELSE 7 END, t.chunk_data FROM $toastname t INNER JOIN chunk_limit cl ON t.chunk_id =3D cl.chunk_id) pg_amcheck reports the following corruption messages: # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 6 follows last expected chunk 5 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 7 follows last expected chunk 5 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 8 follows last expected chunk 5 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 9 has sequence number 15, but expected = sequence number 9 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 10 has sequence number 16, but expected = sequence number 10 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 chunk 11 has sequence number 17, but expected = sequence number 11 # heap table "postgres"."public"."test", block 0, offset 1, attribute 2: # toast value 16444 was expected to end at chunk 6, but ended at = chunk 12 I think if we'd left out the first three messages, it would read = strangely. We would be complaining about three chunks with the wrong = sequence number, then conclude that there were six extra chunks. A = sufficiently savvy user might deduce the presence of chunks 6, 7, and 8, = but the problem is more obvious (to my eyes, at least) if we keep the = first three messages. This seems like a judgement call and not a clear = argument either way, so if you still want me to change it, I guess I = don't mind doing so. > On a related note, as I think I said before, I still think we should > be rejiggering this so that we're not testing both the size of each > individual chunk and the total size, because that ought to be > redundant. That might be better done as a separate patch but I think > we should try to clean it up. Can you point me to the exact check you are mentioning, and with which = patch applied? I don't see any examples of this after applying the = v18-0003. >> v18-0004 - Adding corruption checks of toast pointers. Extends the = regression tests to cover the new checks. >=20 > I think we could check that the result of > VARATT_EXTERNAL_GET_COMPRESS_METHOD is one of the values we expect to > see. Yes. I had that before, pulled it out along with other toast = compression checks, but have put it back in for v19. > Using AllocSizeIsValid() seems pretty vile. I know that MaxAllocSize > is 0x3FFFFFFF in no small part because that's the maximum length that > can be represented by a varlena, but I'm not sure it's a good idea to > couple the concepts so closely like this. Maybe we can just #define > VARLENA_SIZE_LIMIT in this file and use that, and a message that says > size %u exceeds limit %u. Changed. > I'm a little worried about whether the additional test cases are > Endian-dependent at all. I don't immediately know what might be wrong > with them, but I'm going to think about that some more later. Any > chance you have access to a Big-endian box where you can test this? I don't have a Big-endian box, but I think one of them may be wrong now = that you mention the issue: # Corrupt column c's toast pointer va_extinfo field The problem is that the 30-bit extsize and 2-bit cmid split is not being = handled in the perl test, and I don't see an easy way to have perl's = pack/unpack do that for us. There isn't any requirement that each = possible corruption we check actually be manifested in the regression = tests. The simplest solution is to remove this problematic test, so = that's what I did. The other two new tests corrupt c_va_toastrelid and = c_va_rawsize, both of which are read/written using unpack/pack, so perl = should handle the endianness for us (I hope). If you'd rather not commit these two extra tests, you don't have to, as = I've split them out into v19-0003. But if you do commit them, it makes = more sense to me to be one commit with 0002+0003 together, rather than = separately. Not committing the new tests just means that = verify_heapam() is able to detect additional forms of corruption that = we're not covering in the regression tests. But that's already true for = some other corruption types, such as detecting toast chunks with null = sequence numbers. --Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E Content-Disposition: attachment; filename=v19-0001-amcheck-rewording-messages-and-fixing-alignment.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v19-0001-amcheck-rewording-messages-and-fixing-alignment.patch" Content-Transfer-Encoding: quoted-printable =46rom=20b6ea48417cc99258bf2f9106feed4bfebfcf8212=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Thu,=208=20Apr=202021=2009:48:37=20-0700=0ASubject:=20[PATCH=20= v19=201/3]=20amcheck:=20rewording=20messages=20and=20fixing=20alignment=0A= =0ARemoving=20redundant=20mention=20of=20attnum=20in=20the=20corruption=20= message=20text,=0Aas=20the=20attnum=20is=20already=20its=20own=20= separate=20column.=0A=0AWhen=20reporting=20toast=20corruption,=20= mentioning=20the=20toast=20value=20in=20the=0Amessage=20since=20that=20= information=20is=20not=20otherwise=20reported.=0A=0ABeing=20more=20= careful=20about=20alignment=20when=20accessing=20a=20toast=20pointer.=0A= ---=0A=20contrib/amcheck/verify_heapam.c=20=20=20=20=20=20=20=20=20=20=20= |=2063=20+++++++++++++----------=0A=20= src/bin/pg_amcheck/t/004_verify_heapam.pl=20|=20=204=20+-=0A=202=20files=20= changed,=2038=20insertions(+),=2029=20deletions(-)=0A=0Adiff=20--git=20= a/contrib/amcheck/verify_heapam.c=20b/contrib/amcheck/verify_heapam.c=0A= index=20e8aa0d68d4..13f420d9ad=20100644=0A---=20= a/contrib/amcheck/verify_heapam.c=0A+++=20= b/contrib/amcheck/verify_heapam.c=0A@@=20-1179,7=20+1179,8=20@@=20= check_toast_tuple(HeapTuple=20toasttup,=20HeapCheckContext=20*ctx,=0A=20=09= if=20(isnull)=0A=20=09{=0A=20=09=09report_toast_corruption(ctx,=20ta,=0A= -=09=09=09=09=09=09=20=20pstrdup("toast=20chunk=20sequence=20number=20is=20= null"));=0A+=09=09=09=09=09=09=09=09psprintf("toast=20value=20%u=20has=20= toast=20chunk=20with=20null=20sequence=20number",=0A+=09=09=09=09=09=09=09= =09=09=09=20ta->toast_pointer.va_valueid));=0A=20=09=09return;=0A=20=09}=0A= =20=09chunk=20=3D=20DatumGetPointer(fastgetattr(toasttup,=203,=0A@@=20= -1187,7=20+1188,8=20@@=20check_toast_tuple(HeapTuple=20toasttup,=20= HeapCheckContext=20*ctx,=0A=20=09if=20(isnull)=0A=20=09{=0A=20=09=09= report_toast_corruption(ctx,=20ta,=0A-=09=09=09=09=09=09=20=20= pstrdup("toast=20chunk=20data=20is=20null"));=0A+=09=09=09=09=09=09=09=09= psprintf("toast=20value=20%u=20chunk=20%d=20has=20null=20data",=0A+=09=09= =09=09=09=09=09=09=09=09=20ta->toast_pointer.va_valueid,=20chunkno));=0A=20= =09=09return;=0A=20=09}=0A=20=09if=20(!VARATT_IS_EXTENDED(chunk))=0A@@=20= -1205,8=20+1207,9=20@@=20check_toast_tuple(HeapTuple=20toasttup,=20= HeapCheckContext=20*ctx,=0A=20=09=09uint32=09=09header=20=3D=20= ((varattrib_4b=20*)=20chunk)->va_4byte.va_header;=0A=20=0A=20=09=09= report_toast_corruption(ctx,=20ta,=0A-=09=09=09=09=09=09=20=20= psprintf("corrupt=20extended=20toast=20chunk=20has=20invalid=20varlena=20= header:=20%0x=20(sequence=20number=20%d)",=0A-=09=09=09=09=09=09=09=09=20= =20=20header,=20curchunk));=0A+=09=09=09=09=09=09=09=09psprintf("toast=20= value=20%u=20chunk=20%d=20has=20invalid=20varlena=20header=20%0x",=0A+=09= =09=09=09=09=09=09=09=09=09=20ta->toast_pointer.va_valueid,=0A+=09=09=09=09= =09=09=09=09=09=09=20chunkno,=20header));=0A=20=09=09return;=0A=20=09}=0A= =20=0A@@=20-1216,15=20+1219,17=20@@=20check_toast_tuple(HeapTuple=20= toasttup,=20HeapCheckContext=20*ctx,=0A=20=09if=20(curchunk=20!=3D=20= chunkno)=0A=20=09{=0A=20=09=09report_toast_corruption(ctx,=20ta,=0A-=09=09= =09=09=09=09=20=20psprintf("toast=20chunk=20sequence=20number=20%u=20= does=20not=20match=20the=20expected=20sequence=20number=20%u",=0A-=09=09=09= =09=09=09=09=09=20=20=20curchunk,=20chunkno));=0A+=09=09=09=09=09=09=09=09= psprintf("toast=20value=20%u=20chunk=20%d=20has=20sequence=20number=20= %d,=20but=20expected=20sequence=20number=20%d",=0A+=09=09=09=09=09=09=09=09= =09=09=20ta->toast_pointer.va_valueid,=0A+=09=09=09=09=09=09=09=09=09=09=20= chunkno,=20curchunk,=20chunkno));=0A=20=09=09return;=0A=20=09}=0A-=09if=20= (curchunk=20>=20endchunk)=0A+=09if=20(chunkno=20>=20endchunk)=0A=20=09{=0A= =20=09=09report_toast_corruption(ctx,=20ta,=0A-=09=09=09=09=09=09=20=20= psprintf("toast=20chunk=20sequence=20number=20%u=20exceeds=20the=20end=20= chunk=20sequence=20number=20%u",=0A-=09=09=09=09=09=09=09=09=20=20=20= curchunk,=20endchunk));=0A+=09=09=09=09=09=09=09=09psprintf("toast=20= value=20%u=20chunk=20%d=20follows=20last=20expected=20chunk=20%d",=0A+=09= =09=09=09=09=09=09=09=09=09=20ta->toast_pointer.va_valueid,=0A+=09=09=09=09= =09=09=09=09=09=09=20chunkno,=20endchunk));=0A=20=09=09return;=0A=20=09}=0A= =20=0A@@=20-1233,8=20+1238,9=20@@=20check_toast_tuple(HeapTuple=20= toasttup,=20HeapCheckContext=20*ctx,=0A=20=0A=20=09if=20(chunksize=20!=3D=20= expected_size)=0A=20=09=09report_toast_corruption(ctx,=20ta,=0A-=09=09=09= =09=09=09=20=20psprintf("toast=20chunk=20size=20%u=20differs=20from=20= the=20expected=20size=20%u",=0A-=09=09=09=09=09=09=09=09=20=20=20= chunksize,=20expected_size));=0A+=09=09=09=09=09=09=09=09psprintf("toast=20= value=20%u=20chunk=20%d=20has=20size=20%u,=20but=20expected=20size=20= %u",=0A+=09=09=09=09=09=09=09=09=09=09=20ta->toast_pointer.va_valueid,=0A= +=09=09=09=09=09=09=09=09=09=09=20chunkno,=20chunksize,=20= expected_size));=0A=20}=0A=20=0A=20/*=0A@@=20-1265,6=20+1271,7=20@@=20= check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09char=09=20=20=20= *tp;=09=09=09=09/*=20pointer=20to=20the=20tuple=20data=20*/=0A=20=09= uint16=09=09infomask;=0A=20=09Form_pg_attribute=20thisatt;=0A+=09struct=20= varatt_external=20toast_pointer;=0A=20=0A=20=09infomask=20=3D=20= ctx->tuphdr->t_infomask;=0A=20=09thisatt=20=3D=20= TupleDescAttr(RelationGetDescr(ctx->rel),=20ctx->attnum);=0A@@=20-1274,8=20= +1281,7=20@@=20check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09if=20= (ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20ctx->lp_len)=0A=20=09{=0A=20= =09=09report_corruption(ctx,=0A-=09=09=09=09=09=09=20=20= psprintf("attribute=20%u=20with=20length=20%u=20starts=20at=20offset=20= %u=20beyond=20total=20tuple=20length=20%u",=0A-=09=09=09=09=09=09=09=09=20= =20=20ctx->attnum,=0A+=09=09=09=09=09=09=20=20psprintf("attribute=20with=20= length=20%u=20starts=20at=20offset=20%u=20beyond=20total=20tuple=20= length=20%u",=0A=20=09=09=09=09=09=09=09=09=20=20=20thisatt->attlen,=0A=20= =09=09=09=09=09=09=09=09=20=20=20ctx->tuphdr->t_hoff=20+=20ctx->offset,=0A= =20=09=09=09=09=09=09=09=09=20=20=20ctx->lp_len));=0A@@=20-1295,8=20= +1301,7=20@@=20check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09=09= if=20(ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20ctx->lp_len)=0A=20=09=09= {=0A=20=09=09=09report_corruption(ctx,=0A-=09=09=09=09=09=09=09=20=20= psprintf("attribute=20%u=20with=20length=20%u=20ends=20at=20offset=20%u=20= beyond=20total=20tuple=20length=20%u",=0A-=09=09=09=09=09=09=09=09=09=20=20= =20ctx->attnum,=0A+=09=09=09=09=09=09=09=20=20psprintf("attribute=20with=20= length=20%u=20ends=20at=20offset=20%u=20beyond=20total=20tuple=20length=20= %u",=0A=20=09=09=09=09=09=09=09=09=09=20=20=20thisatt->attlen,=0A=20=09=09= =09=09=09=09=09=09=09=20=20=20ctx->tuphdr->t_hoff=20+=20ctx->offset,=0A=20= =09=09=09=09=09=09=09=09=09=20=20=20ctx->lp_len));=0A@@=20-1328,8=20= +1333,7=20@@=20check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09=09= if=20(va_tag=20!=3D=20VARTAG_ONDISK)=0A=20=09=09{=0A=20=09=09=09= report_corruption(ctx,=0A-=09=09=09=09=09=09=09=20=20psprintf("toasted=20= attribute=20%u=20has=20unexpected=20TOAST=20tag=20%u",=0A-=09=09=09=09=09= =09=09=09=09=20=20=20ctx->attnum,=0A+=09=09=09=09=09=09=09=20=20= psprintf("toasted=20attribute=20has=20unexpected=20TOAST=20tag=20%u",=0A=20= =09=09=09=09=09=09=09=09=09=20=20=20va_tag));=0A=20=09=09=09/*=20We=20= can't=20know=20where=20the=20next=20attribute=20begins=20*/=0A=20=09=09=09= return=20false;=0A@@=20-1343,8=20+1347,7=20@@=20= check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09if=20= (ctx->tuphdr->t_hoff=20+=20ctx->offset=20>=20ctx->lp_len)=0A=20=09{=0A=20= =09=09report_corruption(ctx,=0A-=09=09=09=09=09=09=20=20= psprintf("attribute=20%u=20with=20length=20%u=20ends=20at=20offset=20%u=20= beyond=20total=20tuple=20length=20%u",=0A-=09=09=09=09=09=09=09=09=20=20=20= ctx->attnum,=0A+=09=09=09=09=09=09=20=20psprintf("attribute=20with=20= length=20%u=20ends=20at=20offset=20%u=20beyond=20total=20tuple=20length=20= %u",=0A=20=09=09=09=09=09=09=09=09=20=20=20thisatt->attlen,=0A=20=09=09=09= =09=09=09=09=09=20=20=20ctx->tuphdr->t_hoff=20+=20ctx->offset,=0A=20=09=09= =09=09=09=09=09=09=20=20=20ctx->lp_len));=0A@@=20-1371,12=20+1374,17=20= @@=20check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=0A=20=09/*=20= It=20is=20external,=20and=20we're=20looking=20at=20a=20page=20on=20disk=20= */=0A=20=0A+=09/*=0A+=09=20*=20Must=20copy=20attr=20into=20toast_pointer=20= for=20alignment=20considerations=0A+=09=20*/=0A+=09= VARATT_EXTERNAL_GET_POINTER(toast_pointer,=20attr);=0A+=0A=20=09/*=20The=20= tuple=20header=20better=20claim=20to=20contain=20toasted=20values=20*/=0A= =20=09if=20(!(infomask=20&=20HEAP_HASEXTERNAL))=0A=20=09{=0A=20=09=09= report_corruption(ctx,=0A-=09=09=09=09=09=09=20=20psprintf("attribute=20= %u=20is=20external=20but=20tuple=20header=20flag=20HEAP_HASEXTERNAL=20= not=20set",=0A-=09=09=09=09=09=09=09=09=20=20=20ctx->attnum));=0A+=09=09=09= =09=09=09=20=20psprintf("toast=20value=20%u=20is=20external=20but=20= tuple=20header=20flag=20HEAP_HASEXTERNAL=20not=20set",=0A+=09=09=09=09=09= =09=09=09=20=20=20toast_pointer.va_valueid));=0A=20=09=09return=20true;=0A= =20=09}=0A=20=0A@@=20-1384,8=20+1392,8=20@@=20= check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09if=20= (!ctx->rel->rd_rel->reltoastrelid)=0A=20=09{=0A=20=09=09= report_corruption(ctx,=0A-=09=09=09=09=09=09=20=20psprintf("attribute=20= %u=20is=20external=20but=20relation=20has=20no=20toast=20relation",=0A-=09= =09=09=09=09=09=09=09=20=20=20ctx->attnum));=0A+=09=09=09=09=09=09=20=20= psprintf("toast=20value=20%u=20is=20external=20but=20relation=20has=20no=20= toast=20relation",=0A+=09=09=09=09=09=09=09=09=20=20=20= toast_pointer.va_valueid));=0A=20=09=09return=20true;=0A=20=09}=0A=20=0A= @@=20-1464,12=20+1472,13=20@@=20check_toasted_attribute(HeapCheckContext=20= *ctx,=20ToastedAttribute=20*ta)=0A=20=0A=20=09if=20(!found_toasttup)=0A=20= =09=09report_toast_corruption(ctx,=20ta,=0A-=09=09=09=09=09=09=09=09= psprintf("toasted=20value=20for=20attribute=20%u=20missing=20from=20= toast=20table",=0A-=09=09=09=09=09=09=09=09=09=09=20ta->attnum));=0A+=09=09= =09=09=09=09=09=09psprintf("toast=20value=20%u=20not=20found=20in=20= toast=20table",=0A+=09=09=09=09=09=09=09=09=09=09=20= ta->toast_pointer.va_valueid));=0A=20=09else=20if=20(chunkno=20!=3D=20= (endchunk=20+=201))=0A=20=09=09report_toast_corruption(ctx,=20ta,=0A-=09=09= =09=09=09=09=09=09psprintf("final=20toast=20chunk=20number=20%u=20= differs=20from=20expected=20value=20%u",=0A-=09=09=09=09=09=09=09=09=09=09= =20chunkno,=20(endchunk=20+=201)));=0A+=09=09=09=09=09=09=09=09= psprintf("toast=20value=20%u=20was=20expected=20to=20end=20at=20chunk=20= %u,=20but=20ended=20at=20chunk=20%u",=0A+=09=09=09=09=09=09=09=09=09=09=20= ta->toast_pointer.va_valueid,=0A+=09=09=09=09=09=09=09=09=09=09=20= (endchunk=20+=201),=20chunkno));=0A=20}=0A=20=0A=20/*=0Adiff=20--git=20= a/src/bin/pg_amcheck/t/004_verify_heapam.pl=20= b/src/bin/pg_amcheck/t/004_verify_heapam.pl=0Aindex=20= 36607596b1..307f14611c=20100644=0A---=20= a/src/bin/pg_amcheck/t/004_verify_heapam.pl=0A+++=20= b/src/bin/pg_amcheck/t/004_verify_heapam.pl=0A@@=20-480,7=20+480,7=20@@=20= for=20(my=20$tupidx=20=3D=200;=20$tupidx=20<=20ROWCOUNT;=20$tupidx++)=0A=20= =0A=20=09=09$header=20=3D=20header(0,=20$offnum,=201);=0A=20=09=09push=20= @expected,=0A-=09=09=09qr/${header}attribute=20\d+=20with=20length=20\d+=20= ends=20at=20offset=20\d+=20beyond=20total=20tuple=20length=20\d+/;=0A+=09= =09=09qr/${header}attribute=20with=20length=20\d+=20ends=20at=20offset=20= \d+=20beyond=20total=20tuple=20length=20\d+/;=0A=20=09}=0A=20=09elsif=20= ($offnum=20=3D=3D=2013)=0A=20=09{=0A@@=20-489,7=20+489,7=20@@=20for=20= (my=20$tupidx=20=3D=200;=20$tupidx=20<=20ROWCOUNT;=20$tupidx++)=0A=20=0A=20= =09=09$header=20=3D=20header(0,=20$offnum,=202);=0A=20=09=09push=20= @expected,=0A-=09=09=09qr/${header}toasted=20value=20for=20attribute=202=20= missing=20from=20toast=20table/;=0A+=09=09=09qr/${header}toast=20value=20= \d+=20not=20found=20in=20toast=20table/;=0A=20=09}=0A=20=09elsif=20= ($offnum=20=3D=3D=2014)=0A=20=09{=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A= =0A= --Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E Content-Disposition: attachment; filename=v19-0002-amcheck-adding-toast-pointer-corruption-checks.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v19-0002-amcheck-adding-toast-pointer-corruption-checks.patch" Content-Transfer-Encoding: quoted-printable =46rom=2086addc39596cf980a5850d9b1dba23fdaa927485=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Thu,=208=20Apr=202021=2010:09:59=20-0700=0ASubject:=20[PATCH=20= v19=202/3]=20amcheck:=20adding=20toast=20pointer=20corruption=20checks=0A= =0AVerifying=20that=20toast=20pointer=20va_toastrelid=20fields=20match=20= their=20heap=0Atable's=20reltoastrelid.=0A=0AChecking=20the=20extsize=20= for=20a=20toast=20pointer=20against=20the=20raw=20size.=20=20This=0A= check=20could=20fail=20if=20buggy=20compression=20logic=20fails=20to=20= notice=20that=0Acompressing=20the=20attribute=20makes=20it=20bigger.=20=20= But=20assuming=20the=20logic=0Afor=20that=20is=20correct,=20overlarge=20= extsize=20indicates=20a=20corrupted=20toast=0Apointer.=0A=0AChecking=20= if=20a=20toast=20pointer=20indicates=20the=20data=20is=20compressed,=20= that=0Athe=20toast=20pointer=20records=20a=20valid=20compression=20= method.=0A=0AChecking=20the=20toast=20is=20not=20too=20large=20to=20be=20= allocated.=20=20No=20such=0Atoasted=20value=20should=20ever=20be=20= stored,=20but=20a=20corrupted=20toast=20pointer=0Acould=20record=20an=20= unreasonbly=20large=20size,=20so=20check=20that.=0A=0AChanging=20the=20= logic=20to=20continue=20checking=20toast=20even=20after=20reporting=0A= that=20HEAP_HASEXTERNAL=20is=20false.=20=20Previously,=20the=20toast=20= checking=0Astopped=20here,=20but=20that=20wasn't=20necessary,=20and=20= subsequent=20checks=20may=0Aprovide=20additional=20useful=20diagnostic=20= information.=0A---=0A=20contrib/amcheck/verify_heapam.c=20|=2056=20= +++++++++++++++++++++++++++++++--=0A=201=20file=20changed,=2053=20= insertions(+),=203=20deletions(-)=0A=0Adiff=20--git=20= a/contrib/amcheck/verify_heapam.c=20b/contrib/amcheck/verify_heapam.c=0A= index=2013f420d9ad..7b7b40f415=20100644=0A---=20= a/contrib/amcheck/verify_heapam.c=0A+++=20= b/contrib/amcheck/verify_heapam.c=0A@@=20-30,6=20+30,8=20@@=20= PG_FUNCTION_INFO_V1(verify_heapam);=0A=20/*=20The=20number=20of=20= columns=20in=20tuples=20returned=20by=20verify_heapam=20*/=0A=20#define=20= HEAPCHECK_RELATION_COLS=204=0A=20=0A+#define=20VARLENA_SIZE_LIMIT=20= 0x3FFFFFFF=0A+=0A=20/*=0A=20=20*=20Despite=20the=20name,=20we=20use=20= this=20for=20reporting=20problems=20with=20both=20XIDs=20and=0A=20=20*=20= MXIDs.=0A@@=20-1379,14=20+1381,54=20@@=20= check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09=20*/=0A=20=09= VARATT_EXTERNAL_GET_POINTER(toast_pointer,=20attr);=0A=20=0A+=09if=20= (toast_pointer.va_rawsize=20>=20VARLENA_SIZE_LIMIT)=0A+=09=09= report_corruption(ctx,=0A+=09=09=09=09=09=09=20=20psprintf("toast=20= value=20%u=20rawsize=20%u=20exceeds=20limit=20%u",=0A+=09=09=09=09=09=09=09= =09=20=20=20toast_pointer.va_valueid,=0A+=09=09=09=09=09=09=09=09=20=20=20= toast_pointer.va_rawsize,=0A+=09=09=09=09=09=09=09=09=20=20=20= VARLENA_SIZE_LIMIT));=0A+=0A+=09/*=20Compression=20should=20never=20= expand=20the=20attribute=20*/=0A+=09if=20= (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer)=20>=20= toast_pointer.va_rawsize=20-=20VARHDRSZ)=0A+=09=09report_corruption(ctx,=0A= +=09=09=09=09=09=09=20=20psprintf("toast=20value=20%u=20external=20size=20= %u=20exceeds=20maximum=20expected=20for=20rawsize=20%u",=0A+=09=09=09=09=09= =09=09=09=20=20=20toast_pointer.va_valueid,=0A+=09=09=09=09=09=09=09=09=20= =20=20VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer),=0A+=09=09=09=09=09=09=09= =09=20=20=20toast_pointer.va_rawsize));=0A+=0A+=09/*=20Compressed=20= attributes=20should=20have=20a=20valid=20compression=20method=20*/=0A+=09= if=20(VARATT_IS_COMPRESSED(&toast_pointer))=0A+=09{=0A+=09=09= ToastCompressionId=20cmid;=0A+=09=09bool=09=09invalid=20=3D=20true;=0A+=0A= +=09=09cmid=20=3D=20TOAST_COMPRESS_METHOD(&toast_pointer);=0A+=09=09= switch=20(cmid)=0A+=09=09{=0A+=09=09=09/*=20List=20of=20all=20valid=20= compression=20method=20IDs=20*/=0A+=09=09=09case=20= TOAST_PGLZ_COMPRESSION_ID:=0A+=09=09=09case=20TOAST_LZ4_COMPRESSION_ID:=0A= +=09=09=09=09invalid=20=3D=20false;=0A+=09=09=09=09break;=0A+=0A+=09=09=09= /*=20Recognized=20but=20invalid=20compression=20method=20ID=20*/=0A+=09=09= =09case=20TOAST_INVALID_COMPRESSION_ID:=0A+=09=09=09=09break;=0A+=0A+=09=09= =09/*=20Intentionally=20no=20default=20here=20*/=0A+=09=09}=0A+=0A+=09=09= if=20(invalid)=0A+=09=09=09report_corruption(ctx,=0A+=09=09=09=09=09=09=09= =20=20psprintf("toast=20value=20%u=20has=20invalid=20compression=20= method=20id=20%d",=0A+=09=09=09=09=09=09=09=09=09=20=20=20= toast_pointer.va_valueid,=20cmid));=0A+=09}=0A+=0A=20=09/*=20The=20tuple=20= header=20better=20claim=20to=20contain=20toasted=20values=20*/=0A=20=09= if=20(!(infomask=20&=20HEAP_HASEXTERNAL))=0A-=09{=0A=20=09=09= report_corruption(ctx,=0A=20=09=09=09=09=09=09=20=20psprintf("toast=20= value=20%u=20is=20external=20but=20tuple=20header=20flag=20= HEAP_HASEXTERNAL=20not=20set",=0A=20=09=09=09=09=09=09=09=09=20=20=20= toast_pointer.va_valueid));=0A-=09=09return=20true;=0A-=09}=0A=20=0A=20=09= /*=20The=20relation=20better=20have=20a=20toast=20table=20*/=0A=20=09if=20= (!ctx->rel->rd_rel->reltoastrelid)=0A@@=20-1397,6=20+1439,14=20@@=20= check_tuple_attribute(HeapCheckContext=20*ctx)=0A=20=09=09return=20true;=0A= =20=09}=0A=20=0A+=09/*=20The=20toast=20pointer=20had=20better=20point=20= at=20the=20relation's=20toast=20table=20*/=0A+=09if=20= (toast_pointer.va_toastrelid=20!=3D=20ctx->rel->rd_rel->reltoastrelid)=0A= +=09=09report_corruption(ctx,=0A+=09=09=09=09=09=09=20=20psprintf("toast=20= value=20%u=20toast=20relation=20oid=20%u=20differs=20from=20expected=20= oid=20%u",=0A+=09=09=09=09=09=09=09=09=20=20=20toast_pointer.va_valueid,=0A= +=09=09=09=09=09=09=09=09=20=20=20toast_pointer.va_toastrelid,=0A+=09=09=09= =09=09=09=09=09=20=20=20ctx->rel->rd_rel->reltoastrelid));=0A+=0A=20=09= /*=20If=20we=20were=20told=20to=20skip=20toast=20checking,=20then=20= we're=20done.=20*/=0A=20=09if=20(ctx->toast_rel=20=3D=3D=20NULL)=0A=20=09= =09return=20true;=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E Content-Disposition: attachment; filename=v19-0003-amcheck-additional-regression-test-coverage.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v19-0003-amcheck-additional-regression-test-coverage.patch" Content-Transfer-Encoding: quoted-printable =46rom=20cd3534caf75aeb8397c7754a26a8e3722508bbd2=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Mark=20Dilger=20=0A= Date:=20Thu,=208=20Apr=202021=2010:24:40=20-0700=0ASubject:=20[PATCH=20= v19=203/3]=20amcheck:=20additional=20regression=20test=20coverage=0A=0A= Adding=20regression=20checks=20for=20two=20corruption=20checks=20added=20= in=20prior=0Acommit.=20=20We=20now=20verify=20that=20corruption=20= reports=20are=20generated=20when=20we=0Aset=20a=20toast=20pointer's=20= va_toastrelid=20to=20the=20wrong=20Oid=20or=20set=20a=20toast=0A= pointer's=20va_rawsize=20to=20an=20overlarge=20value.=0A---=0A=20= src/bin/pg_amcheck/t/004_verify_heapam.pl=20|=2031=20= ++++++++++++++++++++---=0A=201=20file=20changed,=2028=20insertions(+),=20= 3=20deletions(-)=0A=0Adiff=20--git=20= a/src/bin/pg_amcheck/t/004_verify_heapam.pl=20= b/src/bin/pg_amcheck/t/004_verify_heapam.pl=0Aindex=20= 307f14611c..e64801c7e1=20100644=0A---=20= a/src/bin/pg_amcheck/t/004_verify_heapam.pl=0A+++=20= b/src/bin/pg_amcheck/t/004_verify_heapam.pl=0A@@=20-224,7=20+224,7=20@@=20= my=20$rel=20=3D=20$node->safe_psql('postgres',=20qq(SELECT=20= pg_relation_filepath('public.te=0A=20my=20$relpath=20=3D=20= "$pgdata/$rel";=0A=20=0A=20#=20Insert=20data=20and=20freeze=20= public.test=0A-use=20constant=20ROWCOUNT=20=3D>=2016;=0A+use=20constant=20= ROWCOUNT=20=3D>=2018;=0A=20$node->safe_psql('postgres',=20qq(=0A=20=09= INSERT=20INTO=20public.test=20(a,=20b,=20c)=0A=20=09=09VALUES=20(=0A@@=20= -240,6=20+240,13=20@@=20my=20$relfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A=20my=20$datfrozenxid=20=3D=20= $node->safe_psql('postgres',=0A=20=09q(select=20datfrozenxid=20from=20= pg_database=20where=20datname=20=3D=20'postgres'));=0A=20=0A+#=20Find=20= our=20toast=20relation=20id=0A+my=20$toastrelid=20=3D=20= $node->safe_psql('postgres',=20qq(=0A+=09SELECT=20c.reltoastrelid=0A+=09=09= FROM=20pg_catalog.pg_class=20c=0A+=09=09WHERE=20c.oid=20=3D=20= 'public.test'::regclass=0A+=09=09));=0A+=0A=20#=20Sanity=20check=20that=20= our=20'test'=20table=20has=20a=20relfrozenxid=20newer=20than=20the=0A=20= #=20datfrozenxid=20for=20the=20database,=20and=20that=20the=20= datfrozenxid=20is=20greater=20than=20the=0A=20#=20first=20normal=20xid.=20= =20We=20rely=20on=20these=20invariants=20in=20some=20of=20our=20tests.=0A= @@=20-296,7=20+303,7=20@@=20close($file)=0A=20$node->start;=0A=20=0A=20#=20= Ok,=20Xids=20and=20page=20layout=20look=20ok.=20=20We=20can=20run=20= corruption=20tests.=0A-plan=20tests=20=3D>=2019;=0A+plan=20tests=20=3D>=20= 22;=0A=20=0A=20#=20Check=20that=20pg_amcheck=20runs=20against=20the=20= uncorrupted=20table=20without=20error.=0A=20= $node->command_ok(['pg_amcheck',=20'-p',=20$port,=20'postgres'],=0A@@=20= -501,7=20+508,7=20@@=20for=20(my=20$tupidx=20=3D=200;=20$tupidx=20<=20= ROWCOUNT;=20$tupidx++)=0A=20=09=09push=20@expected,=0A=20=09=09=09= qr/${header}multitransaction=20ID=204=20equals=20or=20exceeds=20next=20= valid=20multitransaction=20ID=201/;=0A=20=09}=0A-=09elsif=20($offnum=20= =3D=3D=2015)=09#=20Last=20offnum=20must=20equal=20ROWCOUNT=0A+=09elsif=20= ($offnum=20=3D=3D=2015)=0A=20=09{=0A=20=09=09#=20Set=20both=20= HEAP_XMAX_COMMITTED=20and=20HEAP_XMAX_IS_MULTI=0A=20=09=09= $tup->{t_infomask}=20|=3D=20HEAP_XMAX_COMMITTED;=0A@@=20-511,6=20+518,24=20= @@=20for=20(my=20$tupidx=20=3D=200;=20$tupidx=20<=20ROWCOUNT;=20= $tupidx++)=0A=20=09=09push=20@expected,=0A=20=09=09=09= qr/${header}multitransaction=20ID=204000000000=20precedes=20relation=20= minimum=20multitransaction=20ID=20threshold=201/;=0A=20=09}=0A+=09elsif=20= ($offnum=20=3D=3D=2016)=0A+=09{=0A+=09=09#=20Corrupt=20column=20c's=20= toast=20pointer=20va_toastrelid=20field=0A+=09=09my=20$otherid=20=3D=20= $toastrelid=20+=201;=0A+=09=09$tup->{c_va_toastrelid}=20=3D=20$otherid;=0A= +=09=09$header=20=3D=20header(0,=20$offnum,=202);=0A+=09=09push=20= @expected,=0A+=09=09=09qr/${header}toast=20value=20\d+=20toast=20= relation=20oid=20$otherid=20differs=20from=20expected=20oid=20= $toastrelid/;=0A+=09}=0A+=09elsif=20($offnum=20=3D=3D=2017)=09#=20Last=20= offnum=20should=20equal=20ROWCOUNT-1=0A+=09{=0A+=09=09#=20Corrupt=20= column=20c's=20toast=20pointer=20va_rawsize=20field=20with=20a=20value=0A= +=09=09#=20exceeding=20maximum=20allowable=20allocation=20size=0A+=09=09= $tup->{c_va_rawsize}=20=3D=200x40000000;=0A+=09=09$header=20=3D=20= header(0,=20$offnum,=202);=0A+=09=09push=20@expected,=0A+=09=09=09= qr/${header}toast=20value=20\d+=20rawsize=201073741824=20exceeds=20limit=20= 1073741823/;=0A+=09}=0A=20=09write_tuple($file,=20$offset,=20$tup);=0A=20= }=0A=20close($file)=0A--=20=0A2.21.1=20(Apple=20Git-122.3)=0A=0A= --Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 =E2=80=94 Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --Apple-Mail=_BBC12D39-5490-49B9-AE1C-AF86E954015E--