From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Erikjan Rijkers <er@xs4all.nl>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Date: Sat, 23 Dec 2017 21:06:05 +0100
Message-ID: <1cc7daa2-027e-d490-2396-96e16d328c45@2ndquadrant.com> (raw)
In-Reply-To: <f29032f5b5f22c160d2916513a006e45@xs4all.nl>
References: <688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com>
<f29032f5b5f22c160d2916513a006e45@xs4all.nl>
On 12/23/2017 03:03 PM, Erikjan Rijkers wrote:
> On 2017-12-23 05:57, Tomas Vondra wrote:
>> Hi all,
>>
>> Attached is a patch series that implements two features to the logical
>> replication - ability to define a memory limit for the reorderbuffer
>> (responsible for building the decoded transactions), and ability to
>> stream large in-progress transactions (exceeding the memory limit).
>>
>
> logical replication of 2 instances is OK but 3 and up fail with:
>
> TRAP: FailedAssertion("!(last_lsn < change->lsn)", File:
> "reorderbuffer.c", Line: 1773)
>
> I can cobble up a script but I hope you have enough from the assertion
> to see what's going wrong...
The assertion says that the iterator produces changes in order that does
not correlate with LSN. But I have a hard time understanding how that
could happen, particularly because according to the line number this
happens in ReorderBufferCommit(), i.e. the current (non-streaming) case.
So instructions to reproduce the issue would be very helpful.
Attached is v2 of the patch series, fixing two bugs I discovered today.
I don't think any of these is related to your issue, though.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
0001-Introduce-logical_work_mem-to-limit-ReorderBuffer-v2.patch.gz (application/gzip, 6.8K) download
0002-Issue-XLOG_XACT_ASSIGNMENT-with-wal_level-logical-v2.patch.gz (application/gzip, 2.7K) download
0003-Issue-individual-invalidations-with-wal_level-log-v2.patch.gz (application/gzip, 4.8K) download
0004-Extend-the-output-plugin-API-with-stream-methods-v2.patch.gz (application/gzip, 5.3K) download
0005-Implement-streaming-mode-in-ReorderBuffer-v2.patch.gz (application/gzip, 10.6K) download
0006-Add-support-for-streaming-to-built-in-replication-v2.patch.gz (application/gzip, 13.9K) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, er@xs4all.nl
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
In-Reply-To: <1cc7daa2-027e-d490-2396-96e16d328c45@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox