From: Arseny Sher <a.sher@postgrespro.ru>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Hsu\, John <hsuchen@amazon.com>
Cc: pgsql-bugs\@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: Re: ERROR: subtransaction logged without previous top-level txn record
Date: Sun, 09 Feb 2020 19:07:50 +0300
Message-ID: <8736bjoiax.fsf@ars-thinkpad> (raw)
In-Reply-To: <CAA4eK1JnSKkNdgLAHWY+YCE_3Li454So_thyTyNJa_G3hSVscA@mail.gmail.com>
References: <AB5978B2-1772-4FEE-A245-74C91704ECB0@amazon.com>
<87ftjifoql.fsf@ars-thinkpad>
<20191024213157.7pm6niybfxgpvmgg@alap3.anarazel.de>
<87eez1fh48.fsf@ars-thinkpad>
<CAA4eK1L=MDbmGu5-+BmY7Svc07jr+ZabiH8C_qo3RSc8pgUpDQ@mail.gmail.com>
<8736bs81sx.fsf@ars-thinkpad>
<CAA4eK1LdNmrib1jub8b=KvYUrzXW0VT4P3MVPMyiMfMY3K64dA@mail.gmail.com>
<871rrb942q.fsf@ars-thinkpad>
<CAA4eK1LYzrZ_+8VhD_N_dsQwjxA9t+AyGKT-Wjnc8S7jCwAcBw@mail.gmail.com>
<87zhdx76d5.fsf@ars-thinkpad>
<CAA4eK1Jdh0zab=+D91MkFPbevzyjFCZsMwRsQxJh4F9+m_vCRA@mail.gmail.com>
<CAA4eK1JnSKkNdgLAHWY+YCE_3Li454So_thyTyNJa_G3hSVscA@mail.gmail.com>
Amit Kapila <amit.kapila16@gmail.com> writes:
>> 1) Decoding from existing slot (*not* initial snapshot construction)
>> starts up, immediately picks up snapshot at restart_lsn (getting into
>> SNAPBUILD_CONSISTENT) and in some xl_xact_assignment learns that it
>> hadn't seen in full (no toplevel records) transaction which it is not
>> even going to stream -- but still dies with "subtransation logged
>> without...". That's my example above, and that's what people are
>> complaining about. Here, usage of serialized snapshot and jump to
>> SNAPBUILD_CONSISTENT is not just legit, it is essential: or order to be
>> able to stream data since confirmed_flush_lsn, we must pick it up as we
>> might not be able to assemble it from scratch in time. I mean, what is
>> wrong here is not serialized snapshot usage but the check.
>>
>
> I was thinking if we have some way to skip processing of
> xl_xact_assignment for such cases, then it might be better. Say,
> along with restart_lsn, if have some way to find corresponding nextXid
> (below which we don't need to process records).
I don't believe you can that without persisting additional
data. Basically, what we need is list of transactions who are running at
the point of snapshot serialization *and* already wrote something before
it -- those we hadn't seen in full and can't decode. We have no such
data currently. The closest thing we have is xl_running_xacts->nextXid,
but
1) issued xid doesn't necessarily means xact actually wrote something,
so we can't just skip xl_xact_assignment for xid < nextXid, it might
still be decoded
2) snapshot might be serialized not at xl_running_xacts anyway
Surely this thing doesn't deserve changing persisted data format.
Somehow I hadn't realized this earlier, so my comments/commit messages
in patches above were not accurate here; I've edited them. Also in the
first patch serialized snapshots are not no longer used for new slot
creation at all, as Andres suggested above. This is not principal, as I
said, but arguably makes things simpler a bit.
I've also found a couple of issues with slot copying feature, will post
in separate thread on them.
Attachments:
[text/x-diff] 0001-Don-t-use-serialized-snapshots-during-logical-slo-v2.patch (0B, ../8736bjoiax.fsf@ars-thinkpad/2-0001-Don-t-use-serialized-snapshots-during-logical-slo-v2.patch)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: a.sher@postgrespro.ru, amit.kapila16@gmail.com, andres@anarazel.de, hsuchen@amazon.com, pgsql-bugs@lists.postgresql.org
Subject: Re: ERROR: subtransaction logged without previous top-level txn record
In-Reply-To: <8736bjoiax.fsf@ars-thinkpad>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox