agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v6 06/10] doc: Add logical replication support to replicate into partitioned tables 8+ messages / 5 participants [nested] [flat]
* [PATCH v6 06/10] doc: Add logical replication support to replicate into partitioned tables @ 2020-04-10 15:05 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Justin Pryzby @ 2020-04-10 15:05 UTC (permalink / raw) commit f1ac27bfda6ce8a399d8001843e9aefff5814f9b Author: Peter Eisentraut <[email protected]> --- doc/src/sgml/logical-replication.sgml | 2 +- src/backend/replication/logical/relation.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml index 3f69b71926..d2cdd13c58 100644 --- a/doc/src/sgml/logical-replication.sgml +++ b/doc/src/sgml/logical-replication.sgml @@ -404,7 +404,7 @@ <para> Replication is only supported by tables, including partitioned tables. Attempts to replicate other types of relations such as views, materialized - views, or foreign tables, will result in an error. + views, or foreign tables will result in an error. </para> </listitem> diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index a60c73d74d..e1a447e097 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -631,7 +631,7 @@ logicalrep_partition_open(LogicalRepRelMapEntry *root, /* * If the partition's attributes don't match the root relation's, we'll * need to make a new attrmap which maps partition attribute numbers to - * remoterel's, instead the original which maps root relation's attribute + * remoterel's, instead of the original which maps root relation's attribute * numbers to remoterel's. * * Note that 'map' which comes from the tuple routing data structure -- 2.17.0 --C+ts3FVlLX8+P6JN Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6-0007-is-vs-are-plural.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
* pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking @ 2022-08-05 16:04 Alvaro Herrera <[email protected]> 0 siblings, 2 replies; 8+ messages in thread From: Alvaro Herrera @ 2022-08-05 16:04 UTC (permalink / raw) To: [email protected] BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking That bit is unlogged and therefore it's wrong to consider it in WAL page comparison. Add a test that tickles the case, as branch testing technology allows. This has been a problem ever since wal consistency checking was introduced (commit a507b86900f6 for pg10), so backpatch to all supported branches. Author: 王海洋 (Haiyang Wang) <[email protected]> Reviewed-by: Kyotaro Horiguchi <[email protected]> Discussion: https://postgr.es/m/CACciXAD2UvLMOhc4jX9VvOKt7DtYLr3OYRBhvOZ-jRxtzc_7Jg@mail.gmail.com Discussion: https://postgr.es/m/CACciXADOfErX9Bx0nzE_SkdfXr6Bbpo5R=v_B6MUTEYW4ya+cg@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/e44dae07f931383151e2eb34ed9b4cbf4bf14482 Modified Files -------------- src/backend/access/brin/brin_pageops.c | 7 ++- src/backend/access/brin/brin_xlog.c | 6 +++ src/test/modules/brin/Makefile | 2 +- src/test/modules/brin/t/02_wal_consistency.pl | 75 +++++++++++++++++++++++++++ 4 files changed, 88 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking @ 2022-08-05 16:10 Alvaro Herrera <[email protected]> parent: Alvaro Herrera <[email protected]> 1 sibling, 0 replies; 8+ messages in thread From: Alvaro Herrera @ 2022-08-05 16:10 UTC (permalink / raw) To: [email protected] On 2022-Aug-05, Alvaro Herrera wrote: > Add a test that tickles the case, as branch testing technology allows. One point here is that this confirms that the backpatched renaming alias for PostgreSQL::Test::Cluster is working well. Another is that, as far as I know, this is the going to be the only case of any code being run under wal_consistency_checking=[not off] regularly. 027_stream_regress.pl is equipped to do so, but as far as I know we have no buildfarm animal with PG_EXTRA_TESTS set it so. I did consider to make this new test conditional on having that flag be on, but I disregarded it because of that. A third point is that in branches 15+ I made it use pg_walinspect to ensure that the desired WAL record is being emitted. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking @ 2022-08-06 19:01 Tom Lane <[email protected]> parent: Alvaro Herrera <[email protected]> 1 sibling, 1 reply; 8+ messages in thread From: Tom Lane @ 2022-08-06 19:01 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: [email protected] Alvaro Herrera <[email protected]> writes: > BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking snapper doesn't like this too much, because error running SQL: 'psql:<stdin>:17: ERROR: time zone "america/punta_arenas" not recognized CONTEXT: PL/pgSQL function inline_code_block line 3 during statement block local variable initialization' Is there a particular reason why you used that zone, rather than say UTC? regards, tom lane ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking @ 2022-08-06 20:03 Álvaro Herrera <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 8+ messages in thread From: Álvaro Herrera @ 2022-08-06 20:03 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: [email protected] On Sat, Aug 6, 2022, at 9:01 PM, Tom Lane wrote: > snapper doesn't like this too much, because > > error running SQL: 'psql:<stdin>:17: ERROR: time zone "america/punta_arenas" not recognized > CONTEXT: PL/pgSQL function inline_code_block line 3 during statement block local variable initialization' > > Is there a particular reason why you used that zone, rather than say UTC? None very good — I just wanted it to be not Moscow, which it was in the OP. I'll change it — to UTC, I suppose. ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking @ 2022-08-07 08:21 Álvaro Herrera <[email protected]> parent: Álvaro Herrera <[email protected]> 0 siblings, 1 reply; 8+ messages in thread From: Álvaro Herrera @ 2022-08-07 08:21 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: [email protected] On 2022-Aug-06, Álvaro Herrera wrote: > On Sat, Aug 6, 2022, at 9:01 PM, Tom Lane wrote: > > snapper doesn't like this too much, because > > > > error running SQL: 'psql:<stdin>:17: ERROR: time zone "america/punta_arenas" not recognized > > CONTEXT: PL/pgSQL function inline_code_block line 3 during statement block local variable initialization' > > > > Is there a particular reason why you used that zone, rather than say UTC? > > None very good — I just wanted it to be not Moscow, which it was in > the OP. I'll change it — to UTC, I suppose. Done. -- Álvaro Herrera ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking @ 2022-08-07 13:44 Tom Lane <[email protected]> parent: Álvaro Herrera <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Tom Lane @ 2022-08-07 13:44 UTC (permalink / raw) To: Álvaro Herrera <[email protected]>; +Cc: [email protected] =?utf-8?Q?=C3=81lvaro?= Herrera <[email protected]> writes: > On 2022-Aug-06, Álvaro Herrera wrote: >> None very good — I just wanted it to be not Moscow, which it was in >> the OP. I'll change it — to UTC, I suppose. > Done. Thanks. I wondered why this was a problem, when we have various other dependencies on specific zone names in the tests. The answer seems to be that America/Punta_Arenas is a fairly new zone name: it was introduced in tzdata 2017a [1]. So snapper's tzdata must be older than that. I see it is using the system tzdata not our own: '--with-system-tzdata=/usr/share/zoneinfo', You would've been fine with America/Santiago, likely :-( regards, tom lane [1] http://mm.icann.org/pipermail/tz-announce/2017-February/000045.html ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v54 1/5] Introduce an option to make logical replication database specific. @ 2026-04-05 07:51 Antonin Houska <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Antonin Houska @ 2026-04-05 07:51 UTC (permalink / raw) By default, the logical decoding assumes access to shared catalogs, so the snapshot builder needs to consider cluster-wide XIDs during startup. That in turn means that, if any transaction is already running (and has XID assigned), the snapshot builder needs to wait for its completion, as it does not know if that transaction performed catalog changes earlier. Possible problem with this concept is that if REPACK (CONCURRENTLY) is running in some database, backends running the same command in other databases get stuck until the first one has committed. Thus only as single backend in the cluster can run REPACK (CONCURRENTLY) at any time. Likewise, REPACK (CONCURRENTLY) can block walsenders starting on behalf of subscriptions throughout the cluster. This patch adds a new option to logical replication output plugin, to declare that it does not use shared catalogs (i.e. catalogs that can be changed by transactions running in other databases in the cluster). In that case, no snapshot the backend will use during the decoding needs to contain information about transactions running in other databases. Thus the snapshot builder only needs to wait for completion of transactions in the current database. Currently we only use this option in the REPACK background worker. It could possibly be used in the plugin for logical replication too, however that would need thorough analysis of that plugin. The patch bumps WAL version number, due to a new field in xl_running_xacts. --- contrib/pg_visibility/pg_visibility.c | 4 +- doc/src/sgml/logicaldecoding.sgml | 4 ++ src/backend/access/index/genam.c | 8 +++ src/backend/access/rmgrdesc/standbydesc.c | 2 + src/backend/access/transam/xlog.c | 2 +- src/backend/access/transam/xlogfuncs.c | 2 +- src/backend/postmaster/bgwriter.c | 2 +- src/backend/replication/logical/decode.c | 8 ++- src/backend/replication/logical/logical.c | 3 ++ src/backend/replication/logical/snapbuild.c | 54 +++++++++++++++++++-- src/backend/replication/slot.c | 2 +- src/backend/storage/ipc/procarray.c | 23 ++++++++- src/backend/storage/ipc/standby.c | 14 +++++- src/include/access/genam.h | 7 +++ src/include/access/xlog_internal.h | 2 +- src/include/replication/output_plugin.h | 1 + src/include/replication/snapbuild.h | 3 +- src/include/storage/procarray.h | 2 +- src/include/storage/standby.h | 3 +- src/include/storage/standbydefs.h | 1 + 20 files changed, 128 insertions(+), 19 deletions(-) diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c index dfab0b64cf5..d564bd2a00c 100644 --- a/contrib/pg_visibility/pg_visibility.c +++ b/contrib/pg_visibility/pg_visibility.c @@ -621,7 +621,7 @@ GetStrictOldestNonRemovableTransactionId(Relation rel) else if (rel == NULL || rel->rd_rel->relisshared) { /* Shared relation: take into account all running xids */ - runningTransactions = GetRunningTransactionData(); + runningTransactions = GetRunningTransactionData(InvalidOid); LWLockRelease(ProcArrayLock); LWLockRelease(XidGenLock); return runningTransactions->oldestRunningXid; @@ -632,7 +632,7 @@ GetStrictOldestNonRemovableTransactionId(Relation rel) * Normal relation: take into account xids running within the current * database */ - runningTransactions = GetRunningTransactionData(); + runningTransactions = GetRunningTransactionData(InvalidOid); LWLockRelease(ProcArrayLock); LWLockRelease(XidGenLock); return runningTransactions->oldestDatabaseRunningXid; diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 6dc49108997..28089d1053c 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -959,6 +959,7 @@ typedef struct OutputPluginOptions { OutputPluginOutputType output_type; bool receive_rewrites; + bool need_shared_catalogs; } OutputPluginOptions; </programlisting> <literal>output_type</literal> has to either be set to @@ -969,6 +970,9 @@ typedef struct OutputPluginOptions also be called for changes made by heap rewrites during certain DDL operations. These are of interest to plugins that handle DDL replication, but they require special handling. + <literal>need_shared_catalogs</literal> can be set to false if you are + sure the plugin functions do not access shared system catalogs. It can + speed up creation of replication slots that use this plugin. </para> <para> diff --git a/src/backend/access/index/genam.c b/src/backend/access/index/genam.c index 1408989c568..44df2605068 100644 --- a/src/backend/access/index/genam.c +++ b/src/backend/access/index/genam.c @@ -394,6 +394,14 @@ systable_beginscan(Relation heapRelation, SysScanDesc sysscan; Relation irel; + /* + * If this backend promised that it won't access shared catalogs during + * logical decoding, this seems to be the right place to check. + */ + Assert(!HistoricSnapshotActive() || + accessSharedCatalogsInDecoding || + !heapRelation->rd_rel->relisshared); + if (indexOK && !IgnoreSystemIndexes && !ReindexIsProcessingIndex(indexId)) diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 0a291354ae2..685d1bdb024 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -41,6 +41,8 @@ standby_desc_running_xacts(StringInfo buf, xl_running_xacts *xlrec) for (i = 0; i < xlrec->subxcnt; i++) appendStringInfo(buf, " %u", xlrec->xids[xlrec->xcnt + i]); } + + appendStringInfo(buf, "; dbid: %u", xlrec->dbid); } void diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 9e8999bbb61..1a6c1456d2c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7731,7 +7731,7 @@ CreateCheckPoint(int flags) * recovery we don't need to write running xact data. */ if (!shutdown && XLogStandbyInfoActive()) - LogStandbySnapshot(); + LogStandbySnapshot(InvalidOid); START_CRIT_SECTION(); diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index 65bbaeda59c..0f5979691e6 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -245,7 +245,7 @@ pg_log_standby_snapshot(PG_FUNCTION_ARGS) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("pg_log_standby_snapshot() can only be used if \"wal_level\" >= \"replica\""))); - recptr = LogStandbySnapshot(); + recptr = LogStandbySnapshot(InvalidOid); /* * As a convenience, return the WAL location of the last inserted record diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c index 1d8947774a9..a30de4262eb 100644 --- a/src/backend/postmaster/bgwriter.c +++ b/src/backend/postmaster/bgwriter.c @@ -289,7 +289,7 @@ BackgroundWriterMain(const void *startup_data, size_t startup_data_len) if (now >= timeout && last_snapshot_lsn <= GetLastImportantRecPtr()) { - last_snapshot_lsn = LogStandbySnapshot(); + last_snapshot_lsn = LogStandbySnapshot(InvalidOid); last_snapshot_ts = now; } } diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 57aaef57c61..4299d0bc867 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -381,7 +381,13 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { xl_running_xacts *running = (xl_running_xacts *) XLogRecGetData(r); - SnapBuildProcessRunningXacts(builder, buf->origptr, running); + + /* + * If the plugin does not access shared catalogs, only keep + * track of transactions in the current database. + */ + SnapBuildProcessRunningXacts(builder, buf->origptr, running, + !ctx->options.need_shared_catalogs); /* * Abort all transactions that we keep track of, that are diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index f20d0c542f3..8ceaf64d164 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -285,6 +285,9 @@ StartupDecodingContext(List *output_plugin_options, ctx->write = do_write; ctx->update_progress = update_progress; + /* Assume shared catalog access. The startup callback can change it. */ + ctx->options.need_shared_catalogs = true; + ctx->output_plugin_options = output_plugin_options; ctx->fast_forward = fast_forward; diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index b4269a3b102..8ef2326c66d 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -125,6 +125,7 @@ #include <sys/stat.h> #include <unistd.h> +#include "access/genam.h" #include "access/heapam_xlog.h" #include "access/transam.h" #include "access/xact.h" @@ -154,6 +155,14 @@ static ResourceOwner SavedResourceOwnerDuringExport = NULL; static bool ExportInProgress = false; +/* + * If a backend is going to do logical decoding and the output plugin does + * not need to access shared catalogs, setting this variable to false can make + * the decoding startup faster. In particular, the backend will not need to + * wait for completion of already running transactions in other databases. + */ +bool accessSharedCatalogsInDecoding = true; + /* ->committed and ->catchange manipulation */ static void SnapBuildPurgeOlderTxn(SnapBuild *builder); @@ -170,7 +179,8 @@ static inline bool SnapBuildXidHasCatalogChanges(SnapBuild *builder, Transaction uint32 xinfo); /* xlog reading helper functions for SnapBuildProcessRunningXacts */ -static bool SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *running); +static bool SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, + xl_running_xacts *running, bool db_specific); static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff); /* serialization functions */ @@ -1136,7 +1146,8 @@ SnapBuildXidHasCatalogChanges(SnapBuild *builder, TransactionId xid, * anymore. */ void -SnapBuildProcessRunningXacts(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *running) +SnapBuildProcessRunningXacts(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *running, + bool db_specific) { ReorderBufferTXN *txn; TransactionId xmin; @@ -1149,12 +1160,21 @@ SnapBuildProcessRunningXacts(SnapBuild *builder, XLogRecPtr lsn, xl_running_xact if (builder->state < SNAPBUILD_CONSISTENT) { /* returns false if there's no point in performing cleanup just yet */ - if (!SnapBuildFindSnapshot(builder, lsn, running)) + if (!SnapBuildFindSnapshot(builder, lsn, running, db_specific)) return; } else SnapBuildSerialize(builder, lsn); + /* + * If database specific transaction info was used during startup, the info + * for the whole cluster can make xmin go backwards. The code below + * probably does not expect that, let's wait with the cleanup for the next + * record. + */ + if (!NormalTransactionIdFollows(running->oldestRunningXid, builder->xmin)) + return; + /* * Update range of interesting xids based on the running xacts * information. We don't increase ->xmax using it, because once we are in @@ -1238,7 +1258,8 @@ SnapBuildProcessRunningXacts(SnapBuild *builder, XLogRecPtr lsn, xl_running_xact * using the xl_running_xacts record. */ static bool -SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *running) +SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *running, + bool db_specific) { /* --- * Build catalog decoding snapshot incrementally using information about @@ -1265,6 +1286,25 @@ SnapBuildFindSnapshot(SnapBuild *builder, XLogRecPtr lsn, xl_running_xacts *runn * --- */ + if (db_specific) + { + /* + * If we must only keep track of transactions running in the current + * database, we need transaction info from exactly that database. + */ + if (running->dbid != MyDatabaseId) + { + LogStandbySnapshot(MyDatabaseId); + return false; + } + + /* + * We'd better be able to check during scan if the plugin does not + * lie. + */ + accessSharedCatalogsInDecoding = false; + } + /* * xl_running_xacts record is older than what we can use, we might not * have all necessary catalog rows anymore. @@ -1465,7 +1505,11 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff) */ if (!RecoveryInProgress()) { - LogStandbySnapshot(); + /* + * If the last transaction info was about specific database, so needs + * to be the next one - at least until we're in the CONSISTENT state. + */ + LogStandbySnapshot(running->dbid); } } diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..9533515a63b 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -1762,7 +1762,7 @@ ReplicationSlotReserveWal(void) XLogRecPtr flushptr; /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); + flushptr = LogStandbySnapshot(InvalidOid); /* and make sure it's fsynced to disk */ XLogFlush(flushptr); diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index cc207cb56e3..92ed34128e5 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -2631,9 +2631,11 @@ ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc) * * Note that if any transaction has overflowed its cached subtransactions * then there is no real need include any subtransactions. + * + * If 'dbid' is valid, only gather transactions running in that database. */ RunningTransactions -GetRunningTransactionData(void) +GetRunningTransactionData(Oid dbid) { /* result workspace */ static RunningTransactionsData CurrentRunningXactsData; @@ -2708,6 +2710,18 @@ GetRunningTransactionData(void) if (!TransactionIdIsValid(xid)) continue; + /* + * Filter by database OID if requested. + */ + if (OidIsValid(dbid)) + { + int pgprocno = arrayP->pgprocnos[index]; + PGPROC *proc = &allProcs[pgprocno]; + + if (proc->databaseId != dbid) + continue; + } + /* * Be careful not to exclude any xids before calculating the values of * oldestRunningXid and suboverflowed, since these are used to clean @@ -2758,6 +2772,12 @@ GetRunningTransactionData(void) PGPROC *proc = &allProcs[pgprocno]; int nsubxids; + /* + * Filter by database OID if requested. + */ + if (OidIsValid(dbid) && proc->databaseId != dbid) + continue; + /* * Save subtransaction XIDs. Other backends can't add or remove * entries while we're holding XidGenLock. @@ -2791,6 +2811,7 @@ GetRunningTransactionData(void) * increases if slots do. */ + CurrentRunningXacts->dbid = dbid; CurrentRunningXacts->xcnt = count - subcount; CurrentRunningXacts->subxcnt = subcount; CurrentRunningXacts->subxid_status = suboverflowed ? SUBXIDS_IN_SUBTRANS : SUBXIDS_IN_ARRAY; diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index de9092fdf5b..c653ea742bc 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -1188,6 +1188,14 @@ standby_redo(XLogReaderState *record) xl_running_xacts *xlrec = (xl_running_xacts *) XLogRecGetData(record); RunningTransactionsData running; + /* + * Records issued for specific database are not suitable for physical + * replication because that affects the whole cluster. In particular, + * the list of XID is probably incomplete here. + */ + if (OidIsValid(xlrec->dbid)) + return; + running.xcnt = xlrec->xcnt; running.subxcnt = xlrec->subxcnt; running.subxid_status = xlrec->subxid_overflow ? SUBXIDS_MISSING : SUBXIDS_IN_ARRAY; @@ -1277,11 +1285,12 @@ standby_redo(XLogReaderState *record) * as there's no independent knob to just enable logical decoding. For * details of how this is used, check snapbuild.c's introductory comment. * + * If 'dbid' is valid, only gather transactions running in that database. * * Returns the RecPtr of the last inserted record. */ XLogRecPtr -LogStandbySnapshot(void) +LogStandbySnapshot(Oid dbid) { XLogRecPtr recptr; RunningTransactions running; @@ -1314,7 +1323,7 @@ LogStandbySnapshot(void) * Log details of all in-progress transactions. This should be the last * record we write, because standby will open up when it sees this. */ - running = GetRunningTransactionData(); + running = GetRunningTransactionData(dbid); /* * GetRunningTransactionData() acquired ProcArrayLock, we must release it. @@ -1358,6 +1367,7 @@ LogCurrentRunningXacts(RunningTransactions CurrRunningXacts) xl_running_xacts xlrec; XLogRecPtr recptr; + xlrec.dbid = CurrRunningXacts->dbid; xlrec.xcnt = CurrRunningXacts->xcnt; xlrec.subxcnt = CurrRunningXacts->subxcnt; xlrec.subxid_overflow = (CurrRunningXacts->subxid_status != SUBXIDS_IN_ARRAY); diff --git a/src/include/access/genam.h b/src/include/access/genam.h index b69320a7fc8..c9a52277be1 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -136,6 +136,13 @@ typedef struct IndexOrderByDistance bool isnull; } IndexOrderByDistance; + +/* + * Keep track of whether logical decoding in this backend promised not to + * access shared catalogs, as a safety check. Somewhat misplaced, but ... + */ +extern bool accessSharedCatalogsInDecoding; + /* * generalized index_ interface routines (in indexam.c) */ diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 10c18d39ff8..13ae3ad4fbb 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -32,7 +32,7 @@ /* * Each page of XLOG file has a header like this: */ -#define XLOG_PAGE_MAGIC 0xD11E /* can be used as WAL version indicator */ +#define XLOG_PAGE_MAGIC 0xD11F /* can be used as WAL version indicator */ typedef struct XLogPageHeaderData { diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h index 842fcde67f9..917f3cff232 100644 --- a/src/include/replication/output_plugin.h +++ b/src/include/replication/output_plugin.h @@ -27,6 +27,7 @@ typedef struct OutputPluginOptions { OutputPluginOutputType output_type; bool receive_rewrites; + bool need_shared_catalogs; } OutputPluginOptions; /* diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h index a22a83a2f23..d02530a912a 100644 --- a/src/include/replication/snapbuild.h +++ b/src/include/replication/snapbuild.h @@ -92,7 +92,8 @@ extern void SnapBuildProcessNewCid(SnapBuild *builder, TransactionId xid, XLogRecPtr lsn, xl_heap_new_cid *xlrec); extern void SnapBuildProcessRunningXacts(SnapBuild *builder, XLogRecPtr lsn, - xl_running_xacts *running); + xl_running_xacts *running, + bool db_specific); extern void SnapBuildSerializationPoint(SnapBuild *builder, XLogRecPtr lsn); extern bool SnapBuildSnapshotExists(XLogRecPtr lsn); diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h index abdf021e66e..377b3060b9f 100644 --- a/src/include/storage/procarray.h +++ b/src/include/storage/procarray.h @@ -49,7 +49,7 @@ extern bool ProcArrayInstallImportedXmin(TransactionId xmin, VirtualTransactionId *sourcevxid); extern bool ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc); -extern RunningTransactions GetRunningTransactionData(void); +extern RunningTransactions GetRunningTransactionData(Oid dbid); extern bool TransactionIdIsInProgress(TransactionId xid); extern TransactionId GetOldestNonRemovableTransactionId(Relation rel); diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 6a314c693cd..8715c08e94f 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -126,6 +126,7 @@ typedef enum typedef struct RunningTransactionsData { + Oid dbid; /* only track xacts in this database */ int xcnt; /* # of xact ids in xids[] */ int subxcnt; /* # of subxact ids in xids[] */ subxids_array_status subxid_status; @@ -143,7 +144,7 @@ typedef RunningTransactionsData *RunningTransactions; extern void LogAccessExclusiveLock(Oid dbOid, Oid relOid); extern void LogAccessExclusiveLockPrepare(void); -extern XLogRecPtr LogStandbySnapshot(void); +extern XLogRecPtr LogStandbySnapshot(Oid dbid); extern void LogStandbyInvalidations(int nmsgs, SharedInvalidationMessage *msgs, bool relcacheInitFileInval); diff --git a/src/include/storage/standbydefs.h b/src/include/storage/standbydefs.h index 231d251fd51..e75b7078766 100644 --- a/src/include/storage/standbydefs.h +++ b/src/include/storage/standbydefs.h @@ -46,6 +46,7 @@ typedef struct xl_standby_locks */ typedef struct xl_running_xacts { + Oid dbid; /* only track xacts in this database */ int xcnt; /* # of xact ids in xids[] */ int subxcnt; /* # of subxact ids in xids[] */ bool subxid_overflow; /* snapshot overflowed, subxids missing */ -- 2.47.3 --cdhh4t7ukb6tnjoq Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="v54-0002-Rename-cluster.c-h-repack.c-h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2026-04-05 07:51 UTC | newest] Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-04-10 15:05 [PATCH v6 06/10] doc: Add logical replication support to replicate into partitioned tables Justin Pryzby <[email protected]> 2022-08-05 16:04 pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking Alvaro Herrera <[email protected]> 2022-08-05 16:10 ` Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking Alvaro Herrera <[email protected]> 2022-08-06 19:01 ` Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking Tom Lane <[email protected]> 2022-08-06 20:03 ` Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking Álvaro Herrera <[email protected]> 2022-08-07 08:21 ` Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking Álvaro Herrera <[email protected]> 2022-08-07 13:44 ` Re: pgsql: BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking Tom Lane <[email protected]> 2026-04-05 07:51 [PATCH v54 1/5] Introduce an option to make logical replication database specific. Antonin Houska <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox