MIME-Version: 1.0
References: 
 <CAA4eK1Jg21ODQ7fS2fvN5W_S5kDRhAP5inj3XMRQaa=s-GbYhw@mail.gmail.com>
 <202604071230.b5axxf3qna3m@alvherre.pgsql>
 <cdgw4sbbfcgk6du3iv54r2dgiy4tfywoklbotlmj4irxavdcr3@glxfw5jj277q>
 <227677.1775576304@localhost>
 <pveffyxhnuurhb44uzqlwo3rkyzorkfh2rot7uwzlf2axhfvbp@7nrs2omysxkc>
 <CAA4eK1JhuT5fyTosWDZ+Pgs+j7xEjObTyRMn80uNKgi_ivqHbw@mail.gmail.com>
 <CADzfLwWnbKcb3v8sStdgNE=WNc3uUqx5SiS4zftX2UaEfNzG5w@mail.gmail.com>
 <85813.1777901089@localhost> <27869.1777985266@localhost>
 <CAA4eK1KC6CGN-N2bUffSign8Sw4q6=8d3L-Xh4t+50GCdQb6zw@mail.gmail.com>
 <70574.1778512672@localhost> <82942.1778527801@localhost>
 <CAA4eK1L1SCa7LjOVzq3zmmqpSV9o7z7VBkPjbE6=2iRLTSBvXw@mail.gmail.com>
 <40976.1778584100@localhost>
In-Reply-To: <40976.1778584100@localhost>
From: Amit Kapila <amit.kapila16@gmail.com>
Date: Wed, 13 May 2026 18:04:17 +0530
Message-ID: 
 <CAA4eK1J=4ODBgxet2RaxmGwHx9h15-4SQBnm38M+05=KM92UvQ@mail.gmail.com>
Subject: Re: Adding REPACK [concurrently]
To: Antonin Houska <ah@cybertec.at>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>,
 Andres Freund <andres@anarazel.de>,
	Alvaro Herrera <alvherre@alvh.no-ip.org>,
 Srinath Reddy Sadipiralla <srinath2133@gmail.com>,
	Matthias van de Meent <boekewurm+postgres@gmail.com>,
	Pg Hackers <pgsql-hackers@lists.postgresql.org>,
 Robert Treat <rob@xzilla.net>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAA4eK1J%3D4ODBgxet2RaxmGwHx9h15-4SQBnm38M%2B05%3DKM92UvQ%40mail.gmail.com>
Precedence: bulk

On Tue, May 12, 2026 at 4:38=E2=80=AFPM Antonin Houska <ah@cybertec.at> wro=
te:
>
> Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> >
> > I see your point. Due to this, once the xmin regresses based on
> > cluster-wide running_xact, some transaction could appear to be running
> > when it should have appeared as committed.
>
> The problem is that xmin does not advance when it should. Attached is a t=
est
> that reproduces the problem (it includes [1], to handle injection points =
in
> background worker), I hope the comments in the specification file are hel=
pful.
>

Yes, the comments were helpful. IIUC, the test skipped insert into
repack_test because the transaction doing that insert happened before
the snapbuild reached FULL_SNAPSHOT/CONSISTENT state, so its commit is
not decoded. Then we also didn't update builder->xmin after reaching
CONSISTENT state in the last running_xact record for MyDatabase. So,
the insert is neither covered in initial copy, nor decoded, so after
repack (concurrently) is finished the table is empty.

I think the patch proposed will fix this specific issue but apart from
other points raised for this patch few more are: (a) Post CONSISTENT
state, for cleanup of db_specific snapshots, we need to separately
again LOG db-specific running xacts whenever we encounter another
running_xacts, (b) Other point is that because
repack-concurrently always use full-snapshot, the serialization of
snapshot is useless because it can't be used by others.

> It's actually not exactly the problem reported in the stress test, but IM=
O the
> core issue is the same: effects of some transactions are lost. In the str=
ess
> test, tuple deletion was lost, so the error was "could not create unique
> index". Here I only demonstrate lost INSERT.
>
> > Assuming, the problematic case is something
> > like what I described, even than the fix of skipping cluster-wide
> > running xacts and instead LOG db-specific running_xacts to help
> > updating builder's xmin sounds inelegant and probably inefficient. For
> > example, I think such a dependency means we can never enable
> > db-specific snapshots on standby.
>
> ok
>

So now the question is where do we go from here. I am not confident
that the current code to achieve db-specific snapshots in logical
decoding is the best possible solution both because of the drawbacks
(like we won't be able to enable this on standby) and inefficiencies
pointed out by me in this and previous emails in this work.

--=20
With Regards,
Amit Kapila.