pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Alexey Kondratov <a.kondratov@postgrespro.ru>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: Erik Rijkers <er@xs4all.nl>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Cc: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Date: Mon, 14 Jan 2019 19:23:31 +0100
Message-ID: <76fc440e-91c3-afe2-b78a-987205b3c758@2ndquadrant.com> (raw )
In-Reply-To: <f4cf3883-5ce2-6829-bb42-3038bfc7b4d9@postgrespro.ru >
References: <688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com >
<bbe1333f-17ba-1156-0713-7cc0bfe2d842@2ndquadrant.com >
<bc83b847-2fc7-6561-4ef1-a3c9d154cc31@2ndquadrant.com >
<47223121-222d-8e04-4928-a28ecdc5327d@2ndquadrant.com >
<CAD21AoBa2mhbhZW6hnDRKs1ggycJOBE69eHk3KgCJ5Cwj9-ZkA@mail.gmail.com >
<b9b306a3-c0a2-7eb9-92eb-a3ca111bd1bc@2ndquadrant.com >
<4762f6bd-b147-cd06-f286-22f54790b77d@2ndquadrant.com >
<cb2530b102afad854e051cca44577c6a@xs4all.nl >
<d807dc44-55f9-5c23-97e5-a5881f4135e2@2ndquadrant.com >
<5eddec05-404f-3fc4-bafd-15e9e4787705@2ndquadrant.com >
<e84013b6-b890-1c4c-2b82-eca185a2197c@2ndquadrant.com >
<8940f374-6651-26d0-cefe-6c719454b6fe@2ndquadrant.com >
<40c38758-04b5-74f4-c963-cf300f9e5dff@postgrespro.ru >
<b95ba070-cd21-72b9-2979-8cdf4d3dd1c6@2ndquadrant.com >
<126e3a30-a7f5-ede4-2bd0-b217749c8b33@postgrespro.ru >
<c511125d-33d7-5209-98ce-5c9977f3e437@2ndquadrant.com >
<f4cf3883-5ce2-6829-bb42-3038bfc7b4d9@postgrespro.ru >
Hi,
Attached is an updated patch series, merging fixes and changes to TAP
tests proposed by Alexey. I've merged the fixes into the appropriate
patches, and I've kept the TAP changes / new tests as separate patches
towards the end of the series.
I'm a bit unhappy with two aspects of the current patch series:
1) We now track schema changes in two ways - using the pre-existing
schema_sent flag in RelationSyncEntry, and the (newly added) flag in
ReorderBuffer. While those options are used for regular vs. streamed
transactions, fundamentally it's the same thing and so having two
competing ways seems like a bad idea. Not sure what's the best way to
resolve this, though.
2) We've removed quite a few asserts, particularly ensuring sanity of
cmin/cmax values. To some extent that's expected, because by allowing
decoding of in-progress transactions relaxes some of those rules. But
I'd be much happier if some of those asserts could be reinstated, even
if only in a weaker form.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
0001-Add-logical_work_mem-to-limit-ReorderBuffer-20190114.patch.gz (application/gzip, 9.9K) download
0002-Immediately-WAL-log-assignments-20190114.patch.gz (application/gzip, 6.4K) download
0003-Issue-individual-invalidations-with-wal_lev-20190114.patch.gz (application/gzip, 4.9K) download
0004-Extend-the-output-plugin-API-with-stream-me-20190114.patch.gz (application/gzip, 6.1K) download
0005-Implement-streaming-mode-in-ReorderBuffer-20190114.patch.gz (application/gzip, 12.8K) download
0006-Add-support-for-streaming-to-built-in-repli-20190114.patch.gz (application/gzip, 19.8K) download
0007-Track-statistics-for-streaming-spilling-20190114.patch.gz (application/gzip, 4.2K) download
0008-Enable-streaming-for-all-subscription-TAP-t-20190114.patch.gz (application/gzip, 1.8K) download
0009-BUGFIX-set-final_lsn-for-subxacts-before-cl-20190114.patch.gz (application/gzip, 639B) download
0010-Add-TAP-test-for-streaming-vs.-DDL-20190114.patch.gz (application/gzip, 1.6K) download
view thread (536+ messages) latest in thread
Message-ID: <76fc440e-91c3-afe2-b78a-987205b3c758@2ndquadrant.com>
Permalink: ../76fc440e-91c3-afe2-b78a-987205b3c758@2ndquadrant.com/
Also on: postgresql.org/message-id/76fc440e-91c3-afe2-b78a-987205b3c758@2ndquadrant.com
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, a.kondratov@postgrespro.ru, peter.eisentraut@2ndquadrant.com, er@xs4all.nl, sawada.mshk@gmail.com, k.knizhnik@postgrespro.ru
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
In-Reply-To: <76fc440e-91c3-afe2-b78a-987205b3c758@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox