public inbox for [email protected]
help / color / mirror / Atom feedFrom: Antonin Houska <[email protected]>
To: SATYANARAYANA NARLAPURAM <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] Compressed TOAST data corruption with REPACK CONCURRENTLY
Date: Fri, 17 Apr 2026 17:45:52 +0200
Message-ID: <52301.1776440752@localhost> (raw)
In-Reply-To: <CAHg+QDeXb9HM2VGKXQedyCp52GzajJK5KOUdNi6oLjsS0nerQw@mail.gmail.com>
References: <CAHg+QDeXb9HM2VGKXQedyCp52GzajJK5KOUdNi6oLjsS0nerQw@mail.gmail.com>
SATYANARAYANA NARLAPURAM <[email protected]> wrote:
> restore_tuple() in repack.c uses SET_VARSIZE() to reconstruct the varlena header when
> reading back external attributes from the spill file. In this process, looks like the flag
> SET_VARSIZE_COMPRESSED is silently lost. Because of this, when REPACK CONCURRENTLY
> run any concurrently updated column whose value was TOAST-compressed ends up with raw
> compressed bytes behind an "uncompressed" header returning garbled data on subsequent reads.
> It appears that existing tests are using random chars which are uncompressable.
>
> Please find the attached 0001-Fix-restore_tuple-losing-varlena-compression-flag.patch to fix this.
> Additionally I updated the existing repack_toast test to include the scenario I was talking about.
Good catch, thanks!
I'd slightly prefer to fix it w/o checking the varlena type, as
attached. However, your test fails to reproduce the issue here, so I'm not
able to verify the fix. I'll take a closer look early next week.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] fix_restore_tuple.diff (576B, 2-fix_restore_tuple.diff)
download | inline diff:
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index e2600a8888d..3c07e44d649 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -2736,7 +2736,7 @@ restore_tuple(BufFile *file, Relation relation, TupleTableSlot *slot)
varlensz = VARSIZE_ANY(&chunk_header);
value = palloc(varlensz);
- SET_VARSIZE(value, VARSIZE_ANY(&chunk_header));
+ memcpy(value, &chunk_header, VARHDRSZ);
BufFileReadExact(file, (char *) value + VARHDRSZ, varlensz - VARHDRSZ);
slot->tts_values[i] = PointerGetDatum(value);
view thread (10+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: [PATCH] Compressed TOAST data corruption with REPACK CONCURRENTLY
In-Reply-To: <52301.1776440752@localhost>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox