pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Teach verify_heapam() to validate update chains within a page.
4+ messages / 3 participants
[nested] [flat]

* pgsql: Teach verify_heapam() to validate update chains within a page.
@ 2023-03-22 13:17  Robert Haas <rhaas@postgresql.org>
  0 siblings, 1 reply; 4+ messages in thread

From: Robert Haas @ 2023-03-22 13:17 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Teach verify_heapam() to validate update chains within a page.

Prior to this commit, we only consider each tuple or line pointer
on the page in isolation, but now we can do some validation of a line
pointer against its successor. For example, a redirect line pointer
shouldn't point to another redirect line pointer, and if a tuple
is HOT-updated, the result should be a heap-only tuple.

Himanshu Upadhyaya and Robert Haas, reviewed by Aleksander Alekseev,
Andres Freund, and Peter Geoghegan.

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/bbc1376b39627c6bddd8a0dc0a7dda24c91a97a0

Modified Files
--------------
contrib/amcheck/verify_heapam.c           | 291 +++++++++++++++++++++++++++++-
src/bin/pg_amcheck/t/004_verify_heapam.pl | 250 +++++++++++++++++++++++--
2 files changed, 524 insertions(+), 17 deletions(-)



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: pgsql: Teach verify_heapam() to validate update chains within a page.
@ 2023-03-22 19:33  Andrew Dunstan <andrew@dunslane.net>
  parent: Robert Haas <rhaas@postgresql.org>
  0 siblings, 1 reply; 4+ messages in thread

From: Andrew Dunstan @ 2023-03-22 19:33 UTC (permalink / raw)
  To: Robert Haas <rhaas@postgresql.org>; pgsql-committers@lists.postgresql.org


On 2023-03-22 We 09:17, Robert Haas wrote:
> Teach verify_heapam() to validate update chains within a page.
>
> Prior to this commit, we only consider each tuple or line pointer
> on the page in isolation, but now we can do some validation of a line
> pointer against its successor. For example, a redirect line pointer
> shouldn't point to another redirect line pointer, and if a tuple
> is HOT-updated, the result should be a heap-only tuple.
>
> Himanshu Upadhyaya and Robert Haas, reviewed by Aleksander Alekseev,
> Andres Freund, and Peter Geoghegan.


This has apparently broken one case in the buildfarm, which now fails an 
upgrade from REL9_2_STABLE because we run amcheck after the upgrade:


heap table "regression.public.xacttest", block 0, offset 1:
     non-heap-only update produced a heap-only tuple at offset 18
heap table "regression.public.xacttest", block 0, offset 2:
     non-heap-only update produced a heap-only tuple at offset 19
heap table "regression.public.xacttest", block 0, offset 4:
     non-heap-only update produced a heap-only tuple at offset 20
heap table "regression.public.xacttest", block 0, offset 5:
     non-heap-only update produced a heap-only tuple at offset 21
heap table "regression.public.wslot", block 0, offset 89:
     non-heap-only update produced a heap-only tuple at offset 95
heap table "regression.public.pslot", block 0, offset 95:
     non-heap-only update produced a heap-only tuple at offset 100
heap table "regression.public.pslot", block 0, offset 96:
     non-heap-only update produced a heap-only tuple at offset 101


I don't know if this is a bug in this commit or if the REL9_2_STABLE 
repo is really broken.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: pgsql: Teach verify_heapam() to validate update chains within a page.
@ 2023-03-22 20:05  Andres Freund <andres@anarazel.de>
  parent: Andrew Dunstan <andrew@dunslane.net>
  0 siblings, 1 reply; 4+ messages in thread

From: Andres Freund @ 2023-03-22 20:05 UTC (permalink / raw)
  To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Robert Haas <rhaas@postgresql.org>; pgsql-committers@lists.postgresql.org

Hi,

On 2023-03-22 15:33:15 -0400, Andrew Dunstan wrote:
> On 2023-03-22 We 09:17, Robert Haas wrote:
> > Teach verify_heapam() to validate update chains within a page.
> > 
> > Prior to this commit, we only consider each tuple or line pointer
> > on the page in isolation, but now we can do some validation of a line
> > pointer against its successor. For example, a redirect line pointer
> > shouldn't point to another redirect line pointer, and if a tuple
> > is HOT-updated, the result should be a heap-only tuple.
> > 
> > Himanshu Upadhyaya and Robert Haas, reviewed by Aleksander Alekseev,
> > Andres Freund, and Peter Geoghegan.
> 
> 
> This has apparently broken one case in the buildfarm, which now fails an
> upgrade from REL9_2_STABLE because we run amcheck after the upgrade:
> 
> 
> heap table "regression.public.xacttest", block 0, offset 1:
>     non-heap-only update produced a heap-only tuple at offset 18
> heap table "regression.public.xacttest", block 0, offset 2:
>     non-heap-only update produced a heap-only tuple at offset 19
> heap table "regression.public.xacttest", block 0, offset 4:
>     non-heap-only update produced a heap-only tuple at offset 20
> heap table "regression.public.xacttest", block 0, offset 5:
>     non-heap-only update produced a heap-only tuple at offset 21
> heap table "regression.public.wslot", block 0, offset 89:
>     non-heap-only update produced a heap-only tuple at offset 95
> heap table "regression.public.pslot", block 0, offset 95:
>     non-heap-only update produced a heap-only tuple at offset 100
> heap table "regression.public.pslot", block 0, offset 96:
>     non-heap-only update produced a heap-only tuple at offset 101
> 
> 
> I don't know if this is a bug in this commit or if the REL9_2_STABLE repo is
> really broken.

FWIW, I get errors on a bunch of catalog tables, on HEAD, if I run the
regression tests and then run amcheck.

heap table "ecpg1_regression.pg_catalog.pg_class", block 7, offset 16:
    non-heap-only update produced a heap-only tuple at offset 19
heap table "ecpg1_regression.pg_catalog.pg_class", block 13, offset 49:
    non-heap-only update produced a heap-only tuple at offset 52
heap table "ecpg1_regression.pg_catalog.pg_class", block 13, offset 53:
...
heap table "regression_pg_surgery.pg_catalog.pg_class", block 0, offset 4:
    non-heap-only update produced a heap-only tuple at offset 5
heap table "regression_pg_surgery.pg_catalog.pg_class", block 0, offset 5:
    non-heap-only update produced a heap-only tuple at offset 6


So either this very quickly bagged the first bugs, or there's something wrong
with the checks.

Greetings,

Andres Freund





^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: pgsql: Teach verify_heapam() to validate update chains within a page.
@ 2023-03-22 20:16  Andres Freund <andres@anarazel.de>
  parent: Andres Freund <andres@anarazel.de>
  0 siblings, 0 replies; 4+ messages in thread

From: Andres Freund @ 2023-03-22 20:16 UTC (permalink / raw)
  To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Robert Haas <rhaas@postgresql.org>; pgsql-committers@lists.postgresql.org

Hi,

On 2023-03-22 13:05:49 -0700, Andres Freund wrote:
> So either this very quickly bagged the first bugs, or there's something wrong
> with the checks.

I'll reply on the thread with a few more details / issues.

https://postgr.es/m/CA%2BTgmoZub-bCg%3DORmbcCCbteQ2UQUc14T2GK%2Bv%2B6-VLd-8yK4g%40mail.gmail.com

Greetings,

Andres Freund





^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2023-03-22 20:16 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-03-22 13:17 pgsql: Teach verify_heapam() to validate update chains within a page. Robert Haas <rhaas@postgresql.org>
2023-03-22 19:33 ` Andrew Dunstan <andrew@dunslane.net>
2023-03-22 20:05   ` Andres Freund <andres@anarazel.de>
2023-03-22 20:16     ` Andres Freund <andres@anarazel.de>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox