MIME-Version: 1.0
References: 
 <CAHg+QDeXb9HM2VGKXQedyCp52GzajJK5KOUdNi6oLjsS0nerQw@mail.gmail.com>
 <52301.1776440752@localhost>
In-Reply-To: <52301.1776440752@localhost>
From: SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>
Date: Fri, 17 Apr 2026 10:40:39 -0700
Message-ID: 
 <CAHg+QDf+yGGizLHAOm_q8Y9SR-tuDa4vWYp+riJ1QWnWxeeLQw@mail.gmail.com>
Subject: Re: [PATCH] Compressed TOAST data corruption with REPACK CONCURRENTLY
To: Antonin Houska <ah@cybertec.at>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 alvherre@kurilemu.de
Content-Type: multipart/alternative; boundary="00000000000043c064064fab730b"
Archived-At: 
 <https://www.postgresql.org/message-id/CAHg%2BQDf%2ByGGizLHAOm_q8Y9SR-tuDa4vWYp%2BriJ1QWnWxeeLQw%40mail.gmail.com>
Precedence: bulk

--00000000000043c064064fab730b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi

On Fri, Apr 17, 2026 at 8:45=E2=80=AFAM Antonin Houska <ah@cybertec.at> wro=
te:

> SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:
>
> > restore_tuple() in repack.c uses SET_VARSIZE() to reconstruct the
> varlena header when
> > reading back external attributes from the spill file. In this process,
> looks like the flag
> > SET_VARSIZE_COMPRESSED is silently lost. Because of this, when REPACK
> CONCURRENTLY
> > run  any concurrently updated column whose value was TOAST-compressed
> ends up with raw
> > compressed bytes behind an "uncompressed" header returning garbled data
> on subsequent reads.
> > It appears that existing tests are using random chars which are
> uncompressable.
> >
> > Please find the attached
> 0001-Fix-restore_tuple-losing-varlena-compression-flag.patch to fix this.
> > Additionally I updated the existing repack_toast test to include the
> scenario I was talking about.
>
> Good catch, thanks!
>
> I'd slightly prefer to fix it w/o checking the varlena type, as
> attached. However, your test fails to reproduce the issue here, so I'm no=
t
> able to verify the fix. I'll take a closer look early next week.
>

I started with that but tried to follow the existing code pattern. This
LGTM.
Please add a comment as well.


>
> --
> Antonin Houska
> Web: https://www.cybertec-postgresql.com
>
>

--00000000000043c064064fab730b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">Hi</div><br><div class=3D"gmail_quote gma=
il_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Apr 17, 2=
026 at 8:45=E2=80=AFAM Antonin Houska &lt;<a href=3D"mailto:ah@cybertec.at"=
>ah@cybertec.at</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pad=
ding-left:1ex">SATYANARAYANA NARLAPURAM &lt;<a href=3D"mailto:satyanarlapur=
am@gmail.com" target=3D"_blank">satyanarlapuram@gmail.com</a>&gt; wrote:<br=
>
<br>
&gt; restore_tuple() in repack.c uses SET_VARSIZE() to reconstruct the varl=
ena header when <br>
&gt; reading back external attributes from the spill file. In this process,=
 looks like the flag <br>
&gt; SET_VARSIZE_COMPRESSED is silently lost. Because of this, when REPACK =
CONCURRENTLY<br>
&gt; run=C2=A0 any concurrently updated column whose value was TOAST-compre=
ssed ends up with raw <br>
&gt; compressed bytes behind an &quot;uncompressed&quot; header returning g=
arbled data on subsequent reads.<br>
&gt; It appears that existing tests are using random chars which are uncomp=
ressable.<br>
&gt; <br>
&gt; Please find the attached 0001-Fix-restore_tuple-losing-varlena-compres=
sion-flag.patch to fix this. <br>
&gt; Additionally I updated the existing repack_toast test to include the s=
cenario I was talking about.<br>
<br>
Good catch, thanks!<br>
<br>
I&#39;d slightly prefer to fix it w/o checking the varlena type, as<br>
attached. However, your test fails to reproduce the issue here, so I&#39;m =
not<br>
able to verify the fix. I&#39;ll take a closer look early next week.<br></b=
lockquote><div><br></div><div>I started with that but tried to follow the e=
xisting code pattern. This LGTM.</div><div>Please add a comment as well.</d=
iv><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
-- <br>
Antonin Houska<br>
Web: <a href=3D"https://www.cybertec-postgresql.com" rel=3D"noreferrer" tar=
get=3D"_blank">https://www.cybertec-postgresql.com</a><br>
<br>
</blockquote></div></div>

--00000000000043c064064fab730b--