public inbox for [email protected]help / color / mirror / Atom feed
Re: Minimal logical decoding on standbys 41+ messages / 8 participants [nested] [flat]
* Re: Minimal logical decoding on standbys @ 2023-01-05 21:15 Robert Haas <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Robert Haas @ 2023-01-05 21:15 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; +Cc: Andres Freund <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Tue, Jan 3, 2023 at 2:42 AM Drouvot, Bertrand <[email protected]> wrote: > Please find attached v36, tiny rebase due to 1de58df4fe. 0001 looks committable to me now, though we probably shouldn't do that unless we're pretty confident about shipping enough of the rest of this to accomplish something useful. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-06 03:40 Andres Freund <[email protected]> parent: Robert Haas <[email protected]> 0 siblings, 3 replies; 41+ messages in thread From: Andres Freund @ 2023-01-06 03:40 UTC (permalink / raw) To: Robert Haas <[email protected]>; Drouvot, Bertrand <[email protected]>; Thomas Munro <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, Thomas, there's one point at the bottom wrt ConditionVariables that'd be interesting for you to comment on. On 2023-01-05 16:15:39 -0500, Robert Haas wrote: > On Tue, Jan 3, 2023 at 2:42 AM Drouvot, Bertrand > <[email protected]> wrote: > > Please find attached v36, tiny rebase due to 1de58df4fe. > > 0001 looks committable to me now, though we probably shouldn't do that > unless we're pretty confident about shipping enough of the rest of > this to accomplish something useful. Cool! ISTM that the ordering of patches isn't quite right later on. ISTM that it doesn't make sense to introduce working logic decoding without first fixing WalSndWaitForWal() (i.e. patch 0006). What made you order the patches that way? 0001: > 4. We can't rely on the standby's relcache entries for this purpose in > any way, because the WAL record that causes the problem might be > replayed before the standby even reaches consistency. The startup process can't access catalog contents in the first place, so the consistency issue is secondary. ISTM that the commit message omits a fairly significant portion of the change: The introduction of indisusercatalog / the reason for its introduction. Why is indisusercatalog stored as "full" column, whereas we store the fact of table being used as a catalog table in a reloption? I'm not adverse to moving to a full column, but then I think we should do the same for tables. Earlier version of the patches IIRC sourced the "catalogness" from the relation. What lead you to changing that? I'm not saying it's wrong, just not sure it's right either. It'd be good to introduce cross-checks that indisusercatalog is set correctly. RelationGetIndexList() seems like a good candidate. I'd probably split the introduction of indisusercatalog into a separate patch. Why was HEAP_DEFAULT_USER_CATALOG_TABLE introduced in this patch? I wonder if we instead should compute a relation's "catalogness" in the relcache. That'd would have the advantage of making RelationIsUsedAsCatalogTable() cheaper and working for all kinds of relations. VISIBILITYMAP_ON_CATALOG_ACCESSIBLE_IN_LOGICAL_DECODING is a very long identifier. Given that the field in the xlog records is just named isCatalogRel, any reason to not just name it correspondingly? 0002: > +/* > + * Helper for InvalidateConflictingLogicalReplicationSlot -- acquires the given slot > + * and mark it invalid, if necessary and possible. > + * > + * Returns whether ReplicationSlotControlLock was released in the interim (and > + * in that case we're not holding the lock at return, otherwise we are). > + * > + * This is inherently racy, because we release the LWLock > + * for syscalls, so caller must restart if we return true. > + */ > +static bool > +InvalidatePossiblyConflictingLogicalReplicationSlot(ReplicationSlot *s, TransactionId xid) This appears to be a near complete copy of InvalidatePossiblyObsoleteSlot(). I don't think we should have two versions of that non-trivial code. Seems we could just have an additional argument for InvalidatePossiblyObsoleteSlot()? > + ereport(LOG, > + (errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)))); > + I think this should report more details, similar to what InvalidateObsoleteReplicationSlots() does. > --- a/src/backend/replication/logical/logicalfuncs.c > +++ b/src/backend/replication/logical/logicalfuncs.c > @@ -216,11 +216,14 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin > > /* > * After the sanity checks in CreateDecodingContext, make sure the > - * restart_lsn is valid. Avoid "cannot get changes" wording in this > + * restart_lsn is valid or both xmin and catalog_xmin are valid. > + * Avoid "cannot get changes" wording in this > * errmsg because that'd be confusingly ambiguous about no changes > * being available. > */ > - if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) > + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn) > + || (!TransactionIdIsValid(MyReplicationSlot->data.xmin) > + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin))) > ereport(ERROR, > (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), > errmsg("can no longer get changes from replication slot \"%s\"", Hm. Feels like we should introduce a helper like SlotIsInvalidated() instead of having this condition in a bunch of places. > + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) > + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) > + ereport(ERROR, > + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), > + errmsg("cannot read from logical replication slot \"%s\"", > + cmd->slotname), > + errdetail("This slot has been invalidated because it was conflicting with recovery."))); > + This is a more precise error than the one in pg_logical_slot_get_changes_guts(). I think both places should output the same error. ISTM that the relevant code should be in CreateDecodingContext(). Imo the code to deal with the WAL version of this has been misplaced... > --- a/src/backend/storage/ipc/procarray.c > +++ b/src/backend/storage/ipc/procarray.c > @@ -3477,6 +3477,10 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode, > > GET_VXID_FROM_PGPROC(procvxid, *proc); > > + /* > + * Note: vxid.localTransactionId can be invalid, which means the > + * request is to signal the pid that is not running a transaction. > + */ > if (procvxid.backendId == vxid.backendId && > procvxid.localTransactionId == vxid.localTransactionId) > { I can't really parse the comment. > @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, > PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, > WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, > true); > + > + if (isCatalogRel) > + InvalidateConflictingLogicalReplicationSlots(locator.dbOid, snapshotConflictHorizon); > } Might be worth checking if wal_level >= logical before the somewhat expensive InvalidateConflictingLogicalReplicationSlots(). > @@ -3051,6 +3054,25 @@ RecoveryConflictInterrupt(ProcSignalReason reason) > case PROCSIG_RECOVERY_CONFLICT_LOCK: > case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: > case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: > + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: > + /* > + * For conflicts that require a logical slot to be invalidated, the > + * requirement is for the signal receiver to release the slot, > + * so that it could be invalidated by the signal sender. So for > + * normal backends, the transaction should be aborted, just > + * like for other recovery conflicts. But if it's walsender on > + * standby, then it has to be killed so as to release an > + * acquired logical slot. > + */ > + if (am_cascading_walsender && > + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && > + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) > + { > + RecoveryConflictPending = true; > + QueryCancelPending = true; > + InterruptPending = true; > + break; > + } > > /* > * If we aren't in a transaction any longer then ignore. Why does the walsender need to be killed? I think it might just be that IsTransactionOrTransactionBlock() might return false, even though we want to cancel. The code actually seems to only cancel (QueryCancelPending is set rather than ProcDiePending), but the comment talks about killing? 0003: > Allow a logical slot to be created on standby. Restrict its usage > or its creation if wal_level on primary is less than logical. > During slot creation, it's restart_lsn is set to the last replayed > LSN. Effectively, a logical slot creation on standby waits for an > xl_running_xact record to arrive from primary. Conflicting slots > would be handled in next commits. I think the commit message might be outdated, the next commit is a test. > + /* > + * Replay pointer may point one past the end of the record. If that > + * is a XLOG page boundary, it will not be a valid LSN for the > + * start of a record, so bump it up past the page header. > + */ > + if (!XRecOffIsValid(restart_lsn)) > + { > + if (restart_lsn % XLOG_BLCKSZ != 0) > + elog(ERROR, "invalid replay pointer"); > + > + /* For the first page of a segment file, it's a long header */ > + if (XLogSegmentOffset(restart_lsn, wal_segment_size) == 0) > + restart_lsn += SizeOfXLogLongPHD; > + else > + restart_lsn += SizeOfXLogShortPHD; > + } Is this actually needed? Supposedly xlogreader can work just fixe with an address at the start of a page? /* * Caller supplied a position to start at. * * In this case, NextRecPtr should already be pointing either to a * valid record starting position or alternatively to the beginning of * a page. See the header comments for XLogBeginRead. */ Assert(RecPtr % XLOG_BLCKSZ == 0 || XRecOffIsValid(RecPtr)); > /* > - * Since logical decoding is only permitted on a primary server, we know > - * that the current timeline ID can't be changing any more. If we did this > - * on a standby, we'd have to worry about the values we compute here > - * becoming invalid due to a promotion or timeline change. > + * Since logical decoding is also permitted on a standby server, we need > + * to check if the server is in recovery to decide how to get the current > + * timeline ID (so that it also cover the promotion or timeline change cases). > */ > + if (!RecoveryInProgress()) > + currTLI = GetWALInsertionTimeLine(); > + else > + GetXLogReplayRecPtr(&currTLI); > + This seems to remove some content from the !recovery case. It's a bit odd that here RecoveryInProgress() is used, whereas further down am_cascading_walsender is used. > @@ -3074,10 +3078,12 @@ XLogSendLogical(void) > * If first time through in this session, initialize flushPtr. Otherwise, > * we only need to update flushPtr if EndRecPtr is past it. > */ > - if (flushPtr == InvalidXLogRecPtr) > - flushPtr = GetFlushRecPtr(NULL); > - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) > - flushPtr = GetFlushRecPtr(NULL); > + if (flushPtr == InvalidXLogRecPtr || > + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) > + { > + flushPtr = (am_cascading_walsender ? > + GetStandbyFlushRecPtr(NULL) : GetFlushRecPtr(NULL)); > + } > > /* If EndRecPtr is still past our flushPtr, it means we caught up. */ > if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) A short if inside a normal if seems ugly to me. 0004: > @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { > > =pod > > +=item $node->create_logical_slot_on_standby(self, master, slot_name, dbname) > + > +Create logical replication slot on given standby > + > +=cut > + > +sub create_logical_slot_on_standby > +{ Any reason this has to be standby specific? > + # Now arrange for the xl_running_xacts record for which pg_recvlogical > + # is waiting. > + $master->safe_psql('postgres', 'CHECKPOINT'); > + Hm, that's quite expensive. Perhaps worth adding a C helper that can do that for us instead? This will likely also be needed in real applications after all. > + print "starting pg_recvlogical\n"; I don't think tests should just print somewhere. Either diag() or note() should be used. > + if ($wait) > + # make sure activeslot is in use > + { > + $node_standby->poll_query_until('testdb', > + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" > + ) or die "slot never became active"; > + } That comment placement imo is quite odd. > +# test if basic decoding works > +is(scalar(my @foobar = split /^/m, $result), > + 14, 'Decoding produced 14 rows'); Maybe mention that it's 2 transactions + 10 rows? > +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); There's enough copies of this that I wonder if we shouldn't introduce a Cluster.pm level helper for this. > +print "waiting to replay $endpos\n"; See above. > +my $stdout_recv = $node_standby->pg_recvlogical_upto( > + 'testdb', 'activeslot', $endpos, 180, > + 'include-xids' => '0', > + 'skip-empty-xacts' => '1'); I don't think this should use a hardcoded 180 but $PostgreSQL::Test::Utils::timeout_default. > +# One way to reproduce recovery conflict is to run VACUUM FULL with > +# hot_standby_feedback turned off on the standby. > +$node_standby->append_conf('postgresql.conf',q[ > +hot_standby_feedback = off > +]); > +$node_standby->restart; IIRC a reload should suffice. > +# This should trigger the conflict > +$node_primary->safe_psql('testdb', 'VACUUM FULL'); Can we do something cheaper than rewriting the entire database? Seems rewriting a single table ought to be sufficient? I think it'd also be good to test that rewriting a non-catalog table doesn't trigger an issue. > +################################################## > +# Recovery conflict: Invalidate conflicting slots, including in-use slots > +# Scenario 2: conflict due to row removal with hot_standby_feedback off. > +################################################## > + > +# get the position to search from in the standby logfile > +my $logstart = -s $node_standby->logfile; > + > +# drop the logical slots > +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); > +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); > + > +create_logical_slots(); > + > +# One way to produce recovery conflict is to create/drop a relation and launch a vacuum > +# with hot_standby_feedback turned off on the standby. > +$node_standby->append_conf('postgresql.conf',q[ > +hot_standby_feedback = off > +]); > +$node_standby->restart; > +# ensure walreceiver feedback off by waiting for expected xmin and > +# catalog_xmin on primary. Both should be NULL since hs_feedback is off > +wait_for_xmins($node_primary, $primary_slotname, > + "xmin IS NULL AND catalog_xmin IS NULL"); > + > +$handle = make_slot_active(1); This is a fair bit of repeated setup, maybe put it into a function? I think it'd be good to test the ongoing decoding via the SQL interface also gets correctly handled. But it might be too hard to do reliably. > +################################################## > +# Test standby promotion and logical decoding behavior > +# after the standby gets promoted. > +################################################## > + I think this also should test the streaming / walsender case. 0006: > diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c > index bc3c3eb3e7..98c96eb864 100644 > --- a/src/backend/access/transam/xlogrecovery.c > +++ b/src/backend/access/transam/xlogrecovery.c > @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData > RecoveryPauseState recoveryPauseState; > ConditionVariable recoveryNotPausedCV; > > + /* Replay state (see getReplayedCV() for more explanation) */ > + ConditionVariable replayedCV; > + > slock_t info_lck; /* locks shared variables shown above */ > } XLogRecoveryCtlData; > getReplayedCV() doesn't seem to fit into any of the naming scheems in use for xlogrecovery.h. > - * Sleep until something happens or we time out. Also wait for the > - * socket becoming writable, if there's still pending output. > + * When not in recovery, sleep until something happens or we time out. > + * Also wait for the socket becoming writable, if there's still pending output. Hm. Is there a problem with not handling the becoming-writable case in the in-recovery case? > + else > + /* > + * We are in the logical decoding on standby case. > + * We are waiting for the startup process to replay wal record(s) using > + * a timeout in case we are requested to stop. > + */ > + { I don't think pgindent will like that formatting.... > + ConditionVariablePrepareToSleep(replayedCV); > + ConditionVariableTimedSleep(replayedCV, 1000, > + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); > + } I think this is racy, see ConditionVariablePrepareToSleep()'s comment: * Caution: "before entering the loop" means you *must* test the exit * condition between calling ConditionVariablePrepareToSleep and calling * ConditionVariableSleep. If that is inconvenient, omit calling * ConditionVariablePrepareToSleep. Basically, the ConditionVariablePrepareToSleep() should be before the loop body. I don't think the fixed timeout here makes sense. For one, we need to wake up based on WalSndComputeSleeptime(), otherwise we're ignoring wal_sender_timeout (which can be quite small). It's also just way too frequent - we're trying to avoid constantly waking up unnecessarily. Perhaps we could deal with the pq_is_send_pending() issue by having a version of ConditionVariableTimedSleep() that accepts a WaitEventSet? Greetings, Andres Freund ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-10 08:33 Drouvot, Bertrand <[email protected]> parent: Andres Freund <[email protected]> 2 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-10 08:33 UTC (permalink / raw) To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/6/23 4:40 AM, Andres Freund wrote: > Hi, > 0002: > > >> +/* >> + * Helper for InvalidateConflictingLogicalReplicationSlot -- acquires the given slot >> + * and mark it invalid, if necessary and possible. >> + * >> + * Returns whether ReplicationSlotControlLock was released in the interim (and >> + * in that case we're not holding the lock at return, otherwise we are). >> + * >> + * This is inherently racy, because we release the LWLock >> + * for syscalls, so caller must restart if we return true. >> + */ >> +static bool >> +InvalidatePossiblyConflictingLogicalReplicationSlot(ReplicationSlot *s, TransactionId xid) > > This appears to be a near complete copy of InvalidatePossiblyObsoleteSlot(). I > don't think we should have two versions of that non-trivial code. Seems we > could just have an additional argument for InvalidatePossiblyObsoleteSlot()? Good point, done in V37 attached. The new logical slot invalidation handling has been "merged" with the obsolete LSN case into 2 new functions (InvalidateObsoleteOrConflictingLogicalReplicationSlots() and InvalidatePossiblyObsoleteOrConflictingLogicalSlot()), removing InvalidateObsoleteReplicationSlots() and InvalidatePossiblyObsoleteSlot(). > > >> + ereport(LOG, >> + (errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)))); >> + > > I think this should report more details, similar to what > InvalidateObsoleteReplicationSlots() does. > Agree, done in V37 (adding more details about the xid horizon and wal_level < logical on master cases). > >> --- a/src/backend/replication/logical/logicalfuncs.c >> +++ b/src/backend/replication/logical/logicalfuncs.c >> @@ -216,11 +216,14 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin >> >> /* >> * After the sanity checks in CreateDecodingContext, make sure the >> - * restart_lsn is valid. Avoid "cannot get changes" wording in this >> + * restart_lsn is valid or both xmin and catalog_xmin are valid. >> + * Avoid "cannot get changes" wording in this >> * errmsg because that'd be confusingly ambiguous about no changes >> * being available. >> */ >> - if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) >> + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn) >> + || (!TransactionIdIsValid(MyReplicationSlot->data.xmin) >> + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin))) >> ereport(ERROR, >> (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), >> errmsg("can no longer get changes from replication slot \"%s\"", > > Hm. Feels like we should introduce a helper like SlotIsInvalidated() instead > of having this condition in a bunch of places. > Agree, LogicalReplicationSlotIsInvalid() has been added in V37. >> + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) >> + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) >> + ereport(ERROR, >> + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), >> + errmsg("cannot read from logical replication slot \"%s\"", >> + cmd->slotname), >> + errdetail("This slot has been invalidated because it was conflicting with recovery."))); >> + > > This is a more precise error than the one in > pg_logical_slot_get_changes_guts(). > > I think both places should output the same error. Agree, done in V37 attached. > ISTM that the relevant code > should be in CreateDecodingContext(). Imo the code to deal with the WAL > version of this has been misplaced... > Looks like a good idea. I'll start a dedicated thread to move the already existing error reporting code part of pg_logical_slot_get_changes_guts() and StartLogicalReplication() into CreateDecodingContext(). >> --- a/src/backend/storage/ipc/procarray.c >> +++ b/src/backend/storage/ipc/procarray.c >> @@ -3477,6 +3477,10 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode, >> >> GET_VXID_FROM_PGPROC(procvxid, *proc); >> >> + /* >> + * Note: vxid.localTransactionId can be invalid, which means the >> + * request is to signal the pid that is not running a transaction. >> + */ >> if (procvxid.backendId == vxid.backendId && >> procvxid.localTransactionId == vxid.localTransactionId) >> { > > I can't really parse the comment. > Looks like it's there since a long time ago (before I started working on this thread). I did not pay that much attention to it, but now that you say it I'm not sure why it as been previously added. Given that 1) I don't get it too and 2) that this comment is the only one modification in this file then V37 just removes it. >> @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, >> PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, >> WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, >> true); >> + >> + if (isCatalogRel) >> + InvalidateConflictingLogicalReplicationSlots(locator.dbOid, snapshotConflictHorizon); >> } > > Might be worth checking if wal_level >= logical before the somewhat expensive > InvalidateConflictingLogicalReplicationSlots(). > Good point, done in V37. > >> @@ -3051,6 +3054,25 @@ RecoveryConflictInterrupt(ProcSignalReason reason) >> case PROCSIG_RECOVERY_CONFLICT_LOCK: >> case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: >> case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: >> + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: >> + /* >> + * For conflicts that require a logical slot to be invalidated, the >> + * requirement is for the signal receiver to release the slot, >> + * so that it could be invalidated by the signal sender. So for >> + * normal backends, the transaction should be aborted, just >> + * like for other recovery conflicts. But if it's walsender on >> + * standby, then it has to be killed so as to release an >> + * acquired logical slot. >> + */ >> + if (am_cascading_walsender && >> + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && >> + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) >> + { >> + RecoveryConflictPending = true; >> + QueryCancelPending = true; >> + InterruptPending = true; >> + break; >> + } >> >> /* >> * If we aren't in a transaction any longer then ignore. > > Why does the walsender need to be killed? I think it might just be that > IsTransactionOrTransactionBlock() might return false, even though we want to > cancel. The code actually seems to only cancel (QueryCancelPending is set > rather than ProcDiePending), but the comment talks about killing? > Oh right, this comment is also there since a long time ago. I think the code is OK (as we break in that case and so we don't go through IsTransactionOrTransactionBlock()). So, V37 just modifies the comment. Please find attached, V37 taking care of: 0001: commit message modifications and renaming VISIBILITYMAP_ON_CATALOG_ACCESSIBLE_IN_LOGICAL_DECODING to VISIBILITYMAP_IS_CATALOG_REL (It does not touch the other remarks as they are still discussed in [1]). 0002: All your remarks mentioned above. I'll look at the ones you've done in [2] on 0003, 0004 and 0006. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com [1]: https://www.postgresql.org/message-id/5c5151a6-a1a3-6c38-7d68-543c9faa22f4%40gmail.com [2]: https://www.postgresql.org/message-id/20230106034036.2m4qnn7ep7b5ipet%40awork3.anarazel.de From 6b389418bbe1172d72b1a17f2997b3f0c5b7bfa5 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:53:30 +0000 Subject: [PATCH v37 6/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index bc3c3eb3e7..98c96eb864 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4916,3 +4925,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index e89c210a8e..b0b6d6ffc7 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1548,6 +1548,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,7 +1567,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1650,20 +1650,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 0ece8f9b76bdaa4d08b0a9311791c2d0030565ea Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:52:27 +0000 Subject: [PATCH v37 5/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4cf863a76f..0387558d75 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From dbbd9fad6102a602878fd1c38ba05764185dd337 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:51:42 +0000 Subject: [PATCH v37 4/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 37 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 479 ++++++++++++++++++ 3 files changed, 517 insertions(+) 6.0% src/test/perl/PostgreSQL/Test/ 93.7% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..6f3c9a6910 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, master, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $master, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $master->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4258844c8f --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,479 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use Test::More tests => 42; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + print "starting pg_recvlogical\n"; + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-f', '-', '--no-loop', '--start'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + # make sure activeslot is in use + { + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); +print "waiting to replay $endpos\n"; + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +create_logical_slots(); + +# One way to reproduce recovery conflict is to run VACUUM FULL with +# hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', 'VACUUM FULL'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and launch a vacuum +# with hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to row removal'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to row removal'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 3: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on master +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on master"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +create_logical_slots(); +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); -- 2.34.1 From 014c2411ef116199bca76e335f72ec94b17bbce6 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:50:43 +0000 Subject: [PATCH v37 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Conflicting slots would be handled in next commits. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 ++++ src/backend/replication/logical/decode.c | 22 ++++++- src/backend/replication/logical/logical.c | 37 +++++++----- src/backend/replication/slot.c | 71 +++++++++++++++-------- src/backend/replication/walsender.c | 27 +++++---- src/include/access/xlog.h | 1 + 6 files changed, 117 insertions(+), 52 deletions(-) 4.5% src/backend/access/transam/ 36.6% src/backend/replication/logical/ 57.9% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 8625942516..edbead2970 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4462,6 +4462,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..c1e43dd2b3 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 52d1fe6269..b313aa93b6 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f22572be30..971cb2bd8c 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1175,37 +1176,46 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) { - XLogRecPtr flushptr; - - /* start at current insert position */ - restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); + restart_lsn = GetXLogReplayRecPtr(NULL); + /* + * Replay pointer may point one past the end of the record. If that + * is a XLOG page boundary, it will not be a valid LSN for the + * start of a record, so bump it up past the page header. + */ + if (!XRecOffIsValid(restart_lsn)) + { + if (restart_lsn % XLOG_BLCKSZ != 0) + elog(ERROR, "invalid replay pointer"); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); + /* For the first page of a segment file, it's a long header */ + if (XLogSegmentOffset(restart_lsn, wal_segment_size) == 0) + restart_lsn += SizeOfXLogLongPHD; + else + restart_lsn += SizeOfXLogShortPHD; + } } else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + restart_lsn = GetXLogInsertRecPtr(); + + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1221,6 +1231,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 87ab467446..e89c210a8e 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,14 +906,18 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + if (!RecoveryInProgress()) + currTLI = GetWALInsertionTimeLine(); + else + GetXLogReplayRecPtr(&currTLI); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; @@ -3074,10 +3078,12 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + flushPtr = (am_cascading_walsender ? + GetStandbyFlushRecPtr(NULL) : GetFlushRecPtr(NULL)); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3174,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 4d7c76f42f1a70fa21496e64966e879e860a24e1 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:49:53 +0000 Subject: [PATCH v37 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on master is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/monitoring.sgml | 11 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 3 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 191 +++++++++++++----- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 5 + src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 3 +- 22 files changed, 268 insertions(+), 55 deletions(-) 3.4% doc/src/sgml/ 8.5% src/backend/access/transam/ 5.3% src/backend/replication/logical/ 56.7% src/backend/replication/ 5.2% src/backend/storage/ipc/ 7.3% src/backend/tcop/ 5.5% src/backend/ 3.5% src/include/replication/ 3.4% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 358d2ff90f..aabf74478d 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4339,6 +4339,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 59e31fcc12..0cc9a7858a 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d0733923d4..b06fa69764 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8752,6 +8752,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8921,6 +8922,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9176,6 +9178,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 0070d56b0b..8625942516 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6442,6 +6442,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6802,7 +6803,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7081,6 +7083,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7246,7 +7249,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7958,6 +7962,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 447c9b970f..6080c17ac4 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1065,7 +1065,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..f22572be30 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -1224,20 +1224,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteOrConflictingLogicalReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1246,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1265,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,11 +1311,18 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - - /* Let caller know */ - *invalidated = true; + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + /* Let caller know */ + *invalidated = true; + } } SpinLockRelease(&s->mutex); @@ -1327,15 +1353,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1419,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1458,38 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * beeninvalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + *beeninvalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1500,35 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, beeninvalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any obsolete slots have been invalidated, recalculate the resource + * limits. */ - if (invalidated) + if (!xid && *beeninvalidated) { ReplicationSlotsComputeRequiredXmin(false); ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 015ae2995d..87ab467446 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..daba766947 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 224ab290af..9e06140f13 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2482,6 +2482,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3051,6 +3054,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6cddd74aa7..3ce69a4bbc 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1055,6 +1055,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1088,6 +1090,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3810de7b22..01f4ffef9a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5550,6 +5550,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index d3e965d744..64dc4e99ed 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..d392b5eec5 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index fb9f936d43..1cc62c447d 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1868,7 +1868,8 @@ pg_stat_database_conflicts| SELECT d.oid AS datid, pg_stat_get_db_conflict_lock(d.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(d.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(d.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(d.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(d.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(d.oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT s.pid, s.gss_auth AS gss_authenticated, -- 2.34.1 From e1a200852687e0a52ef3e5c957a36ad50463b5fc Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:44:00 +0000 Subject: [PATCH v37 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing replication conflicts much as hot standby does. 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: hen a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. To introduce the new isCatalogRel field for indexes, indisusercatalog has been added to pg_index. It allows us to check if there is a risk of conflict on indexes (without having to table_open() the linked table and so prevent any risk of deadlock on it.) Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- contrib/test_decoding/expected/ddl.out | 65 +++++++++++++++++++++++++ contrib/test_decoding/sql/ddl.sql | 23 +++++++++ doc/src/sgml/catalogs.sgml | 11 +++++ src/backend/access/common/reloptions.c | 2 +- src/backend/access/gist/gistxlog.c | 11 ++--- src/backend/access/hash/hash_xlog.c | 12 ++--- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtpage.c | 2 + src/backend/access/spgist/spgvacuum.c | 1 + src/backend/catalog/index.c | 10 ++-- src/backend/commands/tablecmds.c | 55 ++++++++++++++++++++- src/include/access/gistxlog.h | 11 +++-- src/include/access/hash_xlog.h | 8 +-- src/include/access/heapam_xlog.h | 8 +-- src/include/access/nbtxlog.h | 6 ++- src/include/access/spgxlog.h | 1 + src/include/access/visibilitymapdefs.h | 9 ++-- src/include/catalog/pg_index.h | 2 + src/include/utils/rel.h | 14 +++++- 22 files changed, 217 insertions(+), 44 deletions(-) 25.7% contrib/test_decoding/expected/ 11.0% contrib/test_decoding/sql/ 4.3% doc/src/sgml/ 3.7% src/backend/access/gist/ 3.7% src/backend/access/hash/ 5.1% src/backend/access/heap/ 14.9% src/backend/commands/ 5.2% src/backend/ 20.6% src/include/access/ 4.3% src/include/utils/ diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out index 9a28b5ddc5..48fb44c575 100644 --- a/contrib/test_decoding/expected/ddl.out +++ b/contrib/test_decoding/expected/ddl.out @@ -483,6 +483,7 @@ CREATE TABLE replication_metadata ( ) WITH (user_catalog_table = true) ; +CREATE INDEX replication_metadata_idx1 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -492,11 +493,19 @@ WITH (user_catalog_table = true) options | text[] | | | | extended | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) Options: user_catalog_table=true +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_and +---------- + t +(1 row) + INSERT INTO replication_metadata(relation, options) VALUES ('foo', ARRAY['a', 'b']); ALTER TABLE replication_metadata RESET (user_catalog_table); +CREATE INDEX replication_metadata_idx2 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -506,10 +515,19 @@ ALTER TABLE replication_metadata RESET (user_catalog_table); options | text[] | | | | extended | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) + "replication_metadata_idx2" btree (relation) + +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_or +--------- + f +(1 row) INSERT INTO replication_metadata(relation, options) VALUES ('bar', ARRAY['a', 'b']); ALTER TABLE replication_metadata SET (user_catalog_table = true); +CREATE INDEX replication_metadata_idx3 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -519,15 +537,52 @@ ALTER TABLE replication_metadata SET (user_catalog_table = true); options | text[] | | | | extended | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) + "replication_metadata_idx2" btree (relation) + "replication_metadata_idx3" btree (relation) Options: user_catalog_table=true +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_and +---------- + t +(1 row) + INSERT INTO replication_metadata(relation, options) VALUES ('blub', NULL); +-- Also checking that indisusercatalog is set correctly when a table is created with user_catalog_table = false +CREATE TABLE replication_metadata_false ( + id serial primary key, + relation name NOT NULL, + options text[] +) +WITH (user_catalog_table = false) +; +CREATE INDEX replication_metadata_false_idx1 on replication_metadata_false(relation); +\d+ replication_metadata_false + Table "public.replication_metadata_false" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +----------+---------+-----------+----------+--------------------------------------------------------+----------+--------------+------------- + id | integer | | not null | nextval('replication_metadata_false_id_seq'::regclass) | plain | | + relation | name | | not null | | plain | | + options | text[] | | | | extended | | +Indexes: + "replication_metadata_false_pkey" PRIMARY KEY, btree (id) + "replication_metadata_false_idx1" btree (relation) +Options: user_catalog_table=false + +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata_false'::regclass; + bool_or +--------- + f +(1 row) + -- make sure rewrites don't work ALTER TABLE replication_metadata ADD COLUMN rewritemeornot int; ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text; ERROR: cannot rewrite table "replication_metadata" used as a catalog table ALTER TABLE replication_metadata SET (user_catalog_table = false); +CREATE INDEX replication_metadata_idx4 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -538,8 +593,18 @@ ALTER TABLE replication_metadata SET (user_catalog_table = false); rewritemeornot | integer | | | | plain | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) + "replication_metadata_idx2" btree (relation) + "replication_metadata_idx3" btree (relation) + "replication_metadata_idx4" btree (relation) Options: user_catalog_table=false +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_or +--------- + f +(1 row) + INSERT INTO replication_metadata(relation, options) VALUES ('zaphod', NULL); SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1'); diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql index 4f76bed72c..51baac5c4e 100644 --- a/contrib/test_decoding/sql/ddl.sql +++ b/contrib/test_decoding/sql/ddl.sql @@ -276,29 +276,52 @@ CREATE TABLE replication_metadata ( ) WITH (user_catalog_table = true) ; + +CREATE INDEX replication_metadata_idx1 on replication_metadata(relation); + \d+ replication_metadata +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('foo', ARRAY['a', 'b']); ALTER TABLE replication_metadata RESET (user_catalog_table); +CREATE INDEX replication_metadata_idx2 on replication_metadata(relation); \d+ replication_metadata +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('bar', ARRAY['a', 'b']); ALTER TABLE replication_metadata SET (user_catalog_table = true); +CREATE INDEX replication_metadata_idx3 on replication_metadata(relation); \d+ replication_metadata +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('blub', NULL); +-- Also checking that indisusercatalog is set correctly when a table is created with user_catalog_table = false +CREATE TABLE replication_metadata_false ( + id serial primary key, + relation name NOT NULL, + options text[] +) +WITH (user_catalog_table = false) +; + +CREATE INDEX replication_metadata_false_idx1 on replication_metadata_false(relation); +\d+ replication_metadata_false +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata_false'::regclass; + -- make sure rewrites don't work ALTER TABLE replication_metadata ADD COLUMN rewritemeornot int; ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text; ALTER TABLE replication_metadata SET (user_catalog_table = false); +CREATE INDEX replication_metadata_idx4 on replication_metadata(relation); \d+ replication_metadata +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('zaphod', NULL); diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index c1e4048054..22616a0579 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -4447,6 +4447,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>indisusercatalog</structfield> <type>bool</type> + </para> + <para> + If true, the index is linked to a table that is declared as an additional + catalog table for purposes of logical replication (means has <link linkend="sql-createtable"><literal>user_catalog_table</literal></link>) + set to true. + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>indisreplident</structfield> <type>bool</type> diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index 14c23101ad..f5368e3a5b 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -120,7 +120,7 @@ static relopt_bool boolRelOpts[] = RELOPT_KIND_HEAP, AccessExclusiveLock }, - false + HEAP_DEFAULT_USER_CATALOG_TABLE }, { { diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..59e31fcc12 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -608,6 +602,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 63c4f01f0f..d0733923d4 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6871,6 +6871,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_xlog_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8303,7 +8304,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8315,6 +8316,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 1d1ca423a9..045c61edb8 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -283,8 +283,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..edc4fe866a 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..afd9275a10 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(index); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index e6579f2979..a038400fe1 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -123,7 +123,8 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid, bool isexclusion, bool immediate, bool isvalid, - bool isready); + bool isready, + bool is_user_catalog); static void index_update_stats(Relation rel, bool hasindex, double reltuples); @@ -545,7 +546,8 @@ UpdateIndexRelation(Oid indexoid, bool isexclusion, bool immediate, bool isvalid, - bool isready) + bool isready, + bool is_user_catalog) { int2vector *indkey; oidvector *indcollation; @@ -622,6 +624,7 @@ UpdateIndexRelation(Oid indexoid, values[Anum_pg_index_indcheckxmin - 1] = BoolGetDatum(false); values[Anum_pg_index_indisready - 1] = BoolGetDatum(isready); values[Anum_pg_index_indislive - 1] = BoolGetDatum(true); + values[Anum_pg_index_indisusercatalog - 1] = BoolGetDatum(is_user_catalog); values[Anum_pg_index_indisreplident - 1] = BoolGetDatum(false); values[Anum_pg_index_indkey - 1] = PointerGetDatum(indkey); values[Anum_pg_index_indcollation - 1] = PointerGetDatum(indcollation); @@ -1020,7 +1023,8 @@ index_create(Relation heapRelation, isprimary, is_exclusion, (constr_flags & INDEX_CONSTR_CREATE_DEFERRABLE) == 0, !concurrent && !invalid, - !concurrent); + !concurrent, + RelationIsUsedAsCatalogTable(heapRelation)); /* * Register relcache invalidation on the indexes' heap relation, to diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 1db3bd9e2e..092749d103 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -103,6 +103,7 @@ #include "utils/syscache.h" #include "utils/timestamp.h" #include "utils/typcache.h" +#include "utils/rel.h" /* * ON COMMIT action list @@ -14148,6 +14149,10 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, Datum repl_val[Natts_pg_class]; bool repl_null[Natts_pg_class]; bool repl_repl[Natts_pg_class]; + ListCell *cell; + List *rel_options; + bool catalog_table_val = HEAP_DEFAULT_USER_CATALOG_TABLE; + bool catalog_table = false; static char *validnsps[] = HEAP_RELOPT_NAMESPACES; if (defList == NIL && operation != AT_ReplaceRelOptions) @@ -14214,7 +14219,6 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, { Query *view_query = get_view_query(rel); List *view_options = untransformRelOptions(newOptions); - ListCell *cell; bool check_option = false; foreach(cell, view_options) @@ -14242,6 +14246,20 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, } } + /* If user_catalog_table is part of the new options, record its new value */ + rel_options = untransformRelOptions(newOptions); + + foreach(cell, rel_options) + { + DefElem *defel = (DefElem *) lfirst(cell); + + if (strcmp(defel->defname, "user_catalog_table") == 0) + { + catalog_table = true; + catalog_table_val = defGetBoolean(defel); + } + } + /* * All we need do here is update the pg_class row; the new options will be * propagated into relcaches during post-commit cache inval. @@ -14268,6 +14286,41 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, ReleaseSysCache(tuple); + /* Update the indexes if there is a need to */ + if (catalog_table || operation == AT_ResetRelOptions) + { + Relation pg_index; + HeapTuple pg_index_tuple; + Form_pg_index pg_index_form; + ListCell *index; + + pg_index = table_open(IndexRelationId, RowExclusiveLock); + + foreach(index, RelationGetIndexList(rel)) + { + Oid thisIndexOid = lfirst_oid(index); + + pg_index_tuple = SearchSysCacheCopy1(INDEXRELID, + ObjectIdGetDatum(thisIndexOid)); + if (!HeapTupleIsValid(pg_index_tuple)) + elog(ERROR, "cache lookup failed for index %u", thisIndexOid); + pg_index_form = (Form_pg_index) GETSTRUCT(pg_index_tuple); + + /* Modify the index only if user_catalog_table differ */ + if (catalog_table_val != pg_index_form->indisusercatalog) + { + pg_index_form->indisusercatalog = catalog_table_val; + CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple); + InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0, + InvalidOid, true); + } + + heap_freetuple(pg_index_tuple); + } + + table_close(pg_index, RowExclusiveLock); + } + /* repeat the whole exercise for the toast table, if there's one */ if (OidIsValid(rel->rd_rel->reltoastrelid)) { diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..191f0e5808 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,13 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +100,10 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..4a79e0c0a4 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,12 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; - - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + bool isCatalogRel; + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..1d43181a40 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,11 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +345,13 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +410,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..99d87d7189 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,10 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +236,14 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..29a6aa57a9 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,7 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..b27fdc0aef 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,10 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/catalog/pg_index.h b/src/include/catalog/pg_index.h index b0592571da..f5f5de1603 100644 --- a/src/include/catalog/pg_index.h +++ b/src/include/catalog/pg_index.h @@ -43,6 +43,8 @@ CATALOG(pg_index,2610,IndexRelationId) BKI_SCHEMA_MACRO bool indcheckxmin; /* must we wait for xmin to be old? */ bool indisready; /* is this index ready for inserts? */ bool indislive; /* is this index alive at all? */ + bool indisusercatalog; /* is this index linked to a user catalog + * relation? */ bool indisreplident; /* is this index the identity for replication? */ /* variable-length fields start here, but we allow direct access to indkey */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..2ef192c169 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* @@ -343,6 +344,7 @@ typedef struct StdRdOptions #define HEAP_MIN_FILLFACTOR 10 #define HEAP_DEFAULT_FILLFACTOR 100 +#define HEAP_DEFAULT_USER_CATALOG_TABLE false /* * RelationGetToastTupleTarget @@ -385,6 +387,15 @@ typedef struct StdRdOptions (relation)->rd_rel->relkind == RELKIND_MATVIEW) ? \ ((StdRdOptions *) (relation)->rd_options)->user_catalog_table : false) +/* + * IndexIsLinkedToUserCatalogTable + * Returns whether the relation should be treated as an index linked to + * a user catalog table from the pov of logical decoding. + */ +#define IndexIsLinkedToUserCatalogTable(relation) \ + ((relation)->rd_rel->relkind == RELKIND_INDEX && \ + (relation)->rd_index->indisusercatalog) + /* * RelationGetParallelWorkers * Returns the relation's parallel_workers reloption setting. @@ -682,7 +693,8 @@ RelationCloseSmgr(Relation relation) #define RelationIsAccessibleInLogicalDecoding(relation) \ (XLogLogicalInfoActive() && \ RelationNeedsWAL(relation) && \ - (IsCatalogRelation(relation) || RelationIsUsedAsCatalogTable(relation))) + (IsCatalogRelation(relation) || RelationIsUsedAsCatalogTable(relation) || \ + IndexIsLinkedToUserCatalogTable(relation))) /* * RelationIsLogicallyLogged -- 2.34.1 Attachments: [text/plain] v37-0006-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.5K, ../../[email protected]/2-v37-0006-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From 6b389418bbe1172d72b1a17f2997b3f0c5b7bfa5 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:53:30 +0000 Subject: [PATCH v37 6/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index bc3c3eb3e7..98c96eb864 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4916,3 +4925,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index e89c210a8e..b0b6d6ffc7 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1548,6 +1548,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,7 +1567,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1650,20 +1650,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v37-0005-Doc-changes-describing-details-about-logical-dec.patch (2.1K, ../../[email protected]/3-v37-0005-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From 0ece8f9b76bdaa4d08b0a9311791c2d0030565ea Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:52:27 +0000 Subject: [PATCH v37 5/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4cf863a76f..0387558d75 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v37-0004-New-TAP-test-for-logical-decoding-on-standby.patch (20.4K, ../../[email protected]/4-v37-0004-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From dbbd9fad6102a602878fd1c38ba05764185dd337 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:51:42 +0000 Subject: [PATCH v37 4/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 37 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 479 ++++++++++++++++++ 3 files changed, 517 insertions(+) 6.0% src/test/perl/PostgreSQL/Test/ 93.7% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..6f3c9a6910 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, master, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $master, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $master->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4258844c8f --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,479 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use Test::More tests => 42; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + print "starting pg_recvlogical\n"; + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-f', '-', '--no-loop', '--start'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + # make sure activeslot is in use + { + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); +print "waiting to replay $endpos\n"; + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +create_logical_slots(); + +# One way to reproduce recovery conflict is to run VACUUM FULL with +# hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', 'VACUUM FULL'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and launch a vacuum +# with hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to row removal'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to row removal'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 3: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on master +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on master"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +create_logical_slots(); +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); -- 2.34.1 [text/plain] v37-0003-Allow-logical-decoding-on-standby.patch (11.5K, ../../[email protected]/5-v37-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From 014c2411ef116199bca76e335f72ec94b17bbce6 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:50:43 +0000 Subject: [PATCH v37 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Conflicting slots would be handled in next commits. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 ++++ src/backend/replication/logical/decode.c | 22 ++++++- src/backend/replication/logical/logical.c | 37 +++++++----- src/backend/replication/slot.c | 71 +++++++++++++++-------- src/backend/replication/walsender.c | 27 +++++---- src/include/access/xlog.h | 1 + 6 files changed, 117 insertions(+), 52 deletions(-) 4.5% src/backend/access/transam/ 36.6% src/backend/replication/logical/ 57.9% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 8625942516..edbead2970 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4462,6 +4462,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..c1e43dd2b3 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 52d1fe6269..b313aa93b6 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f22572be30..971cb2bd8c 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1175,37 +1176,46 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) { - XLogRecPtr flushptr; - - /* start at current insert position */ - restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); + restart_lsn = GetXLogReplayRecPtr(NULL); + /* + * Replay pointer may point one past the end of the record. If that + * is a XLOG page boundary, it will not be a valid LSN for the + * start of a record, so bump it up past the page header. + */ + if (!XRecOffIsValid(restart_lsn)) + { + if (restart_lsn % XLOG_BLCKSZ != 0) + elog(ERROR, "invalid replay pointer"); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); + /* For the first page of a segment file, it's a long header */ + if (XLogSegmentOffset(restart_lsn, wal_segment_size) == 0) + restart_lsn += SizeOfXLogLongPHD; + else + restart_lsn += SizeOfXLogShortPHD; + } } else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + restart_lsn = GetXLogInsertRecPtr(); + + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1221,6 +1231,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 87ab467446..e89c210a8e 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,14 +906,18 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + if (!RecoveryInProgress()) + currTLI = GetWALInsertionTimeLine(); + else + GetXLogReplayRecPtr(&currTLI); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; @@ -3074,10 +3078,12 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + flushPtr = (am_cascading_walsender ? + GetStandbyFlushRecPtr(NULL) : GetFlushRecPtr(NULL)); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3174,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v37-0002-Handle-logical-slot-conflicts-on-standby.patch (32.4K, ../../[email protected]/6-v37-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 4d7c76f42f1a70fa21496e64966e879e860a24e1 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:49:53 +0000 Subject: [PATCH v37 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on master is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/monitoring.sgml | 11 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 3 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 191 +++++++++++++----- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 5 + src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 3 +- 22 files changed, 268 insertions(+), 55 deletions(-) 3.4% doc/src/sgml/ 8.5% src/backend/access/transam/ 5.3% src/backend/replication/logical/ 56.7% src/backend/replication/ 5.2% src/backend/storage/ipc/ 7.3% src/backend/tcop/ 5.5% src/backend/ 3.5% src/include/replication/ 3.4% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 358d2ff90f..aabf74478d 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4339,6 +4339,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 59e31fcc12..0cc9a7858a 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d0733923d4..b06fa69764 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8752,6 +8752,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8921,6 +8922,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9176,6 +9178,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 0070d56b0b..8625942516 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6442,6 +6442,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6802,7 +6803,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7081,6 +7083,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7246,7 +7249,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7958,6 +7962,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 447c9b970f..6080c17ac4 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1065,7 +1065,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..f22572be30 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -1224,20 +1224,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteOrConflictingLogicalReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1246,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1265,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,11 +1311,18 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - - /* Let caller know */ - *invalidated = true; + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + /* Let caller know */ + *invalidated = true; + } } SpinLockRelease(&s->mutex); @@ -1327,15 +1353,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1419,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1458,38 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * beeninvalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + *beeninvalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1500,35 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, beeninvalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any obsolete slots have been invalidated, recalculate the resource + * limits. */ - if (invalidated) + if (!xid && *beeninvalidated) { ReplicationSlotsComputeRequiredXmin(false); ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 015ae2995d..87ab467446 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..daba766947 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 224ab290af..9e06140f13 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2482,6 +2482,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3051,6 +3054,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6cddd74aa7..3ce69a4bbc 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1055,6 +1055,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1088,6 +1090,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3810de7b22..01f4ffef9a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5550,6 +5550,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index d3e965d744..64dc4e99ed 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..d392b5eec5 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index fb9f936d43..1cc62c447d 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1868,7 +1868,8 @@ pg_stat_database_conflicts| SELECT d.oid AS datid, pg_stat_get_db_conflict_lock(d.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(d.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(d.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(d.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(d.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(d.oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT s.pid, s.gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v37-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (33.9K, ../../[email protected]/7-v37-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From e1a200852687e0a52ef3e5c957a36ad50463b5fc Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 10 Jan 2023 07:44:00 +0000 Subject: [PATCH v37 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing replication conflicts much as hot standby does. 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: hen a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. To introduce the new isCatalogRel field for indexes, indisusercatalog has been added to pg_index. It allows us to check if there is a risk of conflict on indexes (without having to table_open() the linked table and so prevent any risk of deadlock on it.) Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- contrib/test_decoding/expected/ddl.out | 65 +++++++++++++++++++++++++ contrib/test_decoding/sql/ddl.sql | 23 +++++++++ doc/src/sgml/catalogs.sgml | 11 +++++ src/backend/access/common/reloptions.c | 2 +- src/backend/access/gist/gistxlog.c | 11 ++--- src/backend/access/hash/hash_xlog.c | 12 ++--- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtpage.c | 2 + src/backend/access/spgist/spgvacuum.c | 1 + src/backend/catalog/index.c | 10 ++-- src/backend/commands/tablecmds.c | 55 ++++++++++++++++++++- src/include/access/gistxlog.h | 11 +++-- src/include/access/hash_xlog.h | 8 +-- src/include/access/heapam_xlog.h | 8 +-- src/include/access/nbtxlog.h | 6 ++- src/include/access/spgxlog.h | 1 + src/include/access/visibilitymapdefs.h | 9 ++-- src/include/catalog/pg_index.h | 2 + src/include/utils/rel.h | 14 +++++- 22 files changed, 217 insertions(+), 44 deletions(-) 25.7% contrib/test_decoding/expected/ 11.0% contrib/test_decoding/sql/ 4.3% doc/src/sgml/ 3.7% src/backend/access/gist/ 3.7% src/backend/access/hash/ 5.1% src/backend/access/heap/ 14.9% src/backend/commands/ 5.2% src/backend/ 20.6% src/include/access/ 4.3% src/include/utils/ diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out index 9a28b5ddc5..48fb44c575 100644 --- a/contrib/test_decoding/expected/ddl.out +++ b/contrib/test_decoding/expected/ddl.out @@ -483,6 +483,7 @@ CREATE TABLE replication_metadata ( ) WITH (user_catalog_table = true) ; +CREATE INDEX replication_metadata_idx1 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -492,11 +493,19 @@ WITH (user_catalog_table = true) options | text[] | | | | extended | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) Options: user_catalog_table=true +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_and +---------- + t +(1 row) + INSERT INTO replication_metadata(relation, options) VALUES ('foo', ARRAY['a', 'b']); ALTER TABLE replication_metadata RESET (user_catalog_table); +CREATE INDEX replication_metadata_idx2 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -506,10 +515,19 @@ ALTER TABLE replication_metadata RESET (user_catalog_table); options | text[] | | | | extended | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) + "replication_metadata_idx2" btree (relation) + +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_or +--------- + f +(1 row) INSERT INTO replication_metadata(relation, options) VALUES ('bar', ARRAY['a', 'b']); ALTER TABLE replication_metadata SET (user_catalog_table = true); +CREATE INDEX replication_metadata_idx3 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -519,15 +537,52 @@ ALTER TABLE replication_metadata SET (user_catalog_table = true); options | text[] | | | | extended | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) + "replication_metadata_idx2" btree (relation) + "replication_metadata_idx3" btree (relation) Options: user_catalog_table=true +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_and +---------- + t +(1 row) + INSERT INTO replication_metadata(relation, options) VALUES ('blub', NULL); +-- Also checking that indisusercatalog is set correctly when a table is created with user_catalog_table = false +CREATE TABLE replication_metadata_false ( + id serial primary key, + relation name NOT NULL, + options text[] +) +WITH (user_catalog_table = false) +; +CREATE INDEX replication_metadata_false_idx1 on replication_metadata_false(relation); +\d+ replication_metadata_false + Table "public.replication_metadata_false" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +----------+---------+-----------+----------+--------------------------------------------------------+----------+--------------+------------- + id | integer | | not null | nextval('replication_metadata_false_id_seq'::regclass) | plain | | + relation | name | | not null | | plain | | + options | text[] | | | | extended | | +Indexes: + "replication_metadata_false_pkey" PRIMARY KEY, btree (id) + "replication_metadata_false_idx1" btree (relation) +Options: user_catalog_table=false + +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata_false'::regclass; + bool_or +--------- + f +(1 row) + -- make sure rewrites don't work ALTER TABLE replication_metadata ADD COLUMN rewritemeornot int; ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text; ERROR: cannot rewrite table "replication_metadata" used as a catalog table ALTER TABLE replication_metadata SET (user_catalog_table = false); +CREATE INDEX replication_metadata_idx4 on replication_metadata(relation); \d+ replication_metadata Table "public.replication_metadata" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description @@ -538,8 +593,18 @@ ALTER TABLE replication_metadata SET (user_catalog_table = false); rewritemeornot | integer | | | | plain | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) + "replication_metadata_idx1" btree (relation) + "replication_metadata_idx2" btree (relation) + "replication_metadata_idx3" btree (relation) + "replication_metadata_idx4" btree (relation) Options: user_catalog_table=false +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; + bool_or +--------- + f +(1 row) + INSERT INTO replication_metadata(relation, options) VALUES ('zaphod', NULL); SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1'); diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql index 4f76bed72c..51baac5c4e 100644 --- a/contrib/test_decoding/sql/ddl.sql +++ b/contrib/test_decoding/sql/ddl.sql @@ -276,29 +276,52 @@ CREATE TABLE replication_metadata ( ) WITH (user_catalog_table = true) ; + +CREATE INDEX replication_metadata_idx1 on replication_metadata(relation); + \d+ replication_metadata +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('foo', ARRAY['a', 'b']); ALTER TABLE replication_metadata RESET (user_catalog_table); +CREATE INDEX replication_metadata_idx2 on replication_metadata(relation); \d+ replication_metadata +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('bar', ARRAY['a', 'b']); ALTER TABLE replication_metadata SET (user_catalog_table = true); +CREATE INDEX replication_metadata_idx3 on replication_metadata(relation); \d+ replication_metadata +SELECT bool_and(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('blub', NULL); +-- Also checking that indisusercatalog is set correctly when a table is created with user_catalog_table = false +CREATE TABLE replication_metadata_false ( + id serial primary key, + relation name NOT NULL, + options text[] +) +WITH (user_catalog_table = false) +; + +CREATE INDEX replication_metadata_false_idx1 on replication_metadata_false(relation); +\d+ replication_metadata_false +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata_false'::regclass; + -- make sure rewrites don't work ALTER TABLE replication_metadata ADD COLUMN rewritemeornot int; ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text; ALTER TABLE replication_metadata SET (user_catalog_table = false); +CREATE INDEX replication_metadata_idx4 on replication_metadata(relation); \d+ replication_metadata +SELECT bool_or(indisusercatalog) from pg_index where indrelid = 'replication_metadata'::regclass; INSERT INTO replication_metadata(relation, options) VALUES ('zaphod', NULL); diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index c1e4048054..22616a0579 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -4447,6 +4447,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>indisusercatalog</structfield> <type>bool</type> + </para> + <para> + If true, the index is linked to a table that is declared as an additional + catalog table for purposes of logical replication (means has <link linkend="sql-createtable"><literal>user_catalog_table</literal></link>) + set to true. + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>indisreplident</structfield> <type>bool</type> diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index 14c23101ad..f5368e3a5b 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -120,7 +120,7 @@ static relopt_bool boolRelOpts[] = RELOPT_KIND_HEAP, AccessExclusiveLock }, - false + HEAP_DEFAULT_USER_CATALOG_TABLE }, { { diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..59e31fcc12 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -608,6 +602,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 63c4f01f0f..d0733923d4 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6871,6 +6871,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_xlog_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8303,7 +8304,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8315,6 +8316,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 1d1ca423a9..045c61edb8 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -283,8 +283,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..edc4fe866a 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..afd9275a10 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(index); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index e6579f2979..a038400fe1 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -123,7 +123,8 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid, bool isexclusion, bool immediate, bool isvalid, - bool isready); + bool isready, + bool is_user_catalog); static void index_update_stats(Relation rel, bool hasindex, double reltuples); @@ -545,7 +546,8 @@ UpdateIndexRelation(Oid indexoid, bool isexclusion, bool immediate, bool isvalid, - bool isready) + bool isready, + bool is_user_catalog) { int2vector *indkey; oidvector *indcollation; @@ -622,6 +624,7 @@ UpdateIndexRelation(Oid indexoid, values[Anum_pg_index_indcheckxmin - 1] = BoolGetDatum(false); values[Anum_pg_index_indisready - 1] = BoolGetDatum(isready); values[Anum_pg_index_indislive - 1] = BoolGetDatum(true); + values[Anum_pg_index_indisusercatalog - 1] = BoolGetDatum(is_user_catalog); values[Anum_pg_index_indisreplident - 1] = BoolGetDatum(false); values[Anum_pg_index_indkey - 1] = PointerGetDatum(indkey); values[Anum_pg_index_indcollation - 1] = PointerGetDatum(indcollation); @@ -1020,7 +1023,8 @@ index_create(Relation heapRelation, isprimary, is_exclusion, (constr_flags & INDEX_CONSTR_CREATE_DEFERRABLE) == 0, !concurrent && !invalid, - !concurrent); + !concurrent, + RelationIsUsedAsCatalogTable(heapRelation)); /* * Register relcache invalidation on the indexes' heap relation, to diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 1db3bd9e2e..092749d103 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -103,6 +103,7 @@ #include "utils/syscache.h" #include "utils/timestamp.h" #include "utils/typcache.h" +#include "utils/rel.h" /* * ON COMMIT action list @@ -14148,6 +14149,10 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, Datum repl_val[Natts_pg_class]; bool repl_null[Natts_pg_class]; bool repl_repl[Natts_pg_class]; + ListCell *cell; + List *rel_options; + bool catalog_table_val = HEAP_DEFAULT_USER_CATALOG_TABLE; + bool catalog_table = false; static char *validnsps[] = HEAP_RELOPT_NAMESPACES; if (defList == NIL && operation != AT_ReplaceRelOptions) @@ -14214,7 +14219,6 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, { Query *view_query = get_view_query(rel); List *view_options = untransformRelOptions(newOptions); - ListCell *cell; bool check_option = false; foreach(cell, view_options) @@ -14242,6 +14246,20 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, } } + /* If user_catalog_table is part of the new options, record its new value */ + rel_options = untransformRelOptions(newOptions); + + foreach(cell, rel_options) + { + DefElem *defel = (DefElem *) lfirst(cell); + + if (strcmp(defel->defname, "user_catalog_table") == 0) + { + catalog_table = true; + catalog_table_val = defGetBoolean(defel); + } + } + /* * All we need do here is update the pg_class row; the new options will be * propagated into relcaches during post-commit cache inval. @@ -14268,6 +14286,41 @@ ATExecSetRelOptions(Relation rel, List *defList, AlterTableType operation, ReleaseSysCache(tuple); + /* Update the indexes if there is a need to */ + if (catalog_table || operation == AT_ResetRelOptions) + { + Relation pg_index; + HeapTuple pg_index_tuple; + Form_pg_index pg_index_form; + ListCell *index; + + pg_index = table_open(IndexRelationId, RowExclusiveLock); + + foreach(index, RelationGetIndexList(rel)) + { + Oid thisIndexOid = lfirst_oid(index); + + pg_index_tuple = SearchSysCacheCopy1(INDEXRELID, + ObjectIdGetDatum(thisIndexOid)); + if (!HeapTupleIsValid(pg_index_tuple)) + elog(ERROR, "cache lookup failed for index %u", thisIndexOid); + pg_index_form = (Form_pg_index) GETSTRUCT(pg_index_tuple); + + /* Modify the index only if user_catalog_table differ */ + if (catalog_table_val != pg_index_form->indisusercatalog) + { + pg_index_form->indisusercatalog = catalog_table_val; + CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple); + InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0, + InvalidOid, true); + } + + heap_freetuple(pg_index_tuple); + } + + table_close(pg_index, RowExclusiveLock); + } + /* repeat the whole exercise for the toast table, if there's one */ if (OidIsValid(rel->rd_rel->reltoastrelid)) { diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..191f0e5808 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,13 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +100,10 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..4a79e0c0a4 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,12 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; - - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + bool isCatalogRel; + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..1d43181a40 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,11 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +345,13 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +410,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..99d87d7189 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,10 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +236,14 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..29a6aa57a9 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,7 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..b27fdc0aef 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,10 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/catalog/pg_index.h b/src/include/catalog/pg_index.h index b0592571da..f5f5de1603 100644 --- a/src/include/catalog/pg_index.h +++ b/src/include/catalog/pg_index.h @@ -43,6 +43,8 @@ CATALOG(pg_index,2610,IndexRelationId) BKI_SCHEMA_MACRO bool indcheckxmin; /* must we wait for xmin to be old? */ bool indisready; /* is this index ready for inserts? */ bool indislive; /* is this index alive at all? */ + bool indisusercatalog; /* is this index linked to a user catalog + * relation? */ bool indisreplident; /* is this index the identity for replication? */ /* variable-length fields start here, but we allow direct access to indkey */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..2ef192c169 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* @@ -343,6 +344,7 @@ typedef struct StdRdOptions #define HEAP_MIN_FILLFACTOR 10 #define HEAP_DEFAULT_FILLFACTOR 100 +#define HEAP_DEFAULT_USER_CATALOG_TABLE false /* * RelationGetToastTupleTarget @@ -385,6 +387,15 @@ typedef struct StdRdOptions (relation)->rd_rel->relkind == RELKIND_MATVIEW) ? \ ((StdRdOptions *) (relation)->rd_options)->user_catalog_table : false) +/* + * IndexIsLinkedToUserCatalogTable + * Returns whether the relation should be treated as an index linked to + * a user catalog table from the pov of logical decoding. + */ +#define IndexIsLinkedToUserCatalogTable(relation) \ + ((relation)->rd_rel->relkind == RELKIND_INDEX && \ + (relation)->rd_index->indisusercatalog) + /* * RelationGetParallelWorkers * Returns the relation's parallel_workers reloption setting. @@ -682,7 +693,8 @@ RelationCloseSmgr(Relation relation) #define RelationIsAccessibleInLogicalDecoding(relation) \ (XLogLogicalInfoActive() && \ RelationNeedsWAL(relation) && \ - (IsCatalogRelation(relation) || RelationIsUsedAsCatalogTable(relation))) + (IsCatalogRelation(relation) || RelationIsUsedAsCatalogTable(relation) || \ + IndexIsLinkedToUserCatalogTable(relation))) /* * RelationIsLogicallyLogged -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-11 07:32 Bharath Rupireddy <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 2 replies; 41+ messages in thread From: Bharath Rupireddy @ 2023-01-11 07:32 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; +Cc: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Tue, Jan 10, 2023 at 2:03 PM Drouvot, Bertrand <[email protected]> wrote: > > Please find attached, V37 taking care of: Thanks. I started to digest the design specified in the commit message and these patches. Here are some quick comments: 1. Does logical decoding on standby work without any issues if the standby is set for cascading replication? 2. Do logical decoding output plugins work without any issues on the standby with decoding enabled, say, when the slot is invalidated? 3. Is this feature still a 'minimal logical decoding on standby'? Firstly, why is it 'minimal'? 4. What happens in case of failover to the standby that's already decoding for its clients? Will the decoding work seamlessly? If not, what are recommended things that users need to take care of during/after failovers? 0002: 1. - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); Isn't the function name too long and verbose? How about just InvalidateLogicalReplicationSlots() let the function comment talk about what sorts of replication slots it invalides? 2. + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + * master wal_level is set back to replica, so existing logical slots need to invalidate such slots. Also do the same thing if wal_level on master Can we use 'primary server' instead of 'master' like elsewhere? This comment also applies for other patches too, if any. 3. Can we show a new status in pg_get_replication_slots's wal_status for invalidated due to the conflict so that the user can monitor for the new status and take necessary actions? 4. How will the user be notified when logical replication slots are invalidated due to conflict with the primary server? How can they (firstly, is there a way?) repair/restore such replication slots? Or is recreating/reestablishing logical replication only the way out for them? What users can do to avoid their logical replication slots getting invalidated and run into these situations? Because recreating/reestablishing logical replication comes with cost (sometimes huge) as it involves building another instance, syncing tables etc. Isn't it a good idea to touch up on all these aspects in the documentation at least as to why we're doing this and why we can't do this? 5. @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); Having the invalidation check in StartLogicalReplication() looks fine, however, what happens if the slot gets invalidated when the replication is in-progress? Do we need to error out in WalSndLoop() or XLogSendLogical() too? Or is it already taken care of somewhere? -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-11 15:23 Drouvot, Bertrand <[email protected]> parent: Bharath Rupireddy <[email protected]> 1 sibling, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-11 15:23 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; +Cc: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/11/23 8:32 AM, Bharath Rupireddy wrote: > On Tue, Jan 10, 2023 at 2:03 PM Drouvot, Bertrand > <[email protected]> wrote: >> >> Please find attached, V37 taking care of: > > Thanks. I started to digest the design specified in the commit message > and these patches. Thanks for looking at it! > Here are some quick comments: > > 1. Does logical decoding on standby work without any issues if the > standby is set for cascading replication? > Without "any issues" is hard to guarantee ;-) But according to my tests: Primary -> Standby1 with or without logical replication slot -> Standby2 with or without logical replication slot works as expected (and also with cascading promotion). We can add some TAP tests in 0004 though. > 2. Do logical decoding output plugins work without any issues on the > standby with decoding enabled, say, when the slot is invalidated? > Not sure, I got the question. If the slot is invalidated then it's expected to get errors like: pg_recvlogical: error: unexpected termination of replication stream: ERROR: canceling statement due to conflict with recovery DETAIL: User was using the logical slot that must be dropped. or pg_recvlogical: error: could not send replication command "START_REPLICATION SLOT "bdt_slot" LOGICAL 0/0": ERROR: cannot read from logical replication slot "bdt_slot" DETAIL: This slot has been invalidated because it was conflicting with recovery. > 3. Is this feature still a 'minimal logical decoding on standby'? > Firstly, why is it 'minimal'? > Good question and I don't have the answer. That's how it has been named when this thread started back in 2018. > 4. What happens in case of failover to the standby that's already > decoding for its clients? Will the decoding work seamlessly? If not, > what are recommended things that users need to take care of > during/after failovers? Yes, it's expected to work seamlessly. There is a TAP test in 0004 for this scenario. > > 0002: > 1. > - if (InvalidateObsoleteReplicationSlots(_logSegNo)) > + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, > &invalidated, InvalidOid, NULL); > > Isn't the function name too long and verbose? How about just > InvalidateLogicalReplicationSlots() The function also takes care of Invalidation of Physical replication slots that are Obsolete (aka LSN case). InvalidateObsoleteOrConflictingReplicationSlots() maybe? > let the function comment talk > about what sorts of replication slots it invalides? Agree to make the comment more clear. > > 2. > + errdetail("Logical decoding on > standby requires wal_level to be at least logical on master")); > + * master wal_level is set back to replica, so existing logical > slots need to > invalidate such slots. Also do the same thing if wal_level on master > > Can we use 'primary server' instead of 'master' like elsewhere? This > comment also applies for other patches too, if any. > Sure. > 3. Can we show a new status in pg_get_replication_slots's wal_status > for invalidated due to the conflict so that the user can monitor for > the new status and take necessary actions? > Not sure you've seen but the patch series is adding a new field (confl_active_logicalslot) in pg_stat_database_conflicts. That said, I like your idea about adding a new status in pg_replication_slots too. Do you think it's mandatory for this patch series? (I mean it could be added once this patch series is committed). I'm asking because this patch series looks already like a "big" one, is more than 4 years old and I'm afraid of adding more "reporting" stuff to it (unless we feel a strong need for it of course). > 4. How will the user be notified when logical replication slots are > invalidated due to conflict with the primary server? Emitting messages, like the ones mentioned above introduced in 0002. > How can they > (firstly, is there a way?) repair/restore such replication slots? Or > is recreating/reestablishing logical replication only the way out for > them? Drop/recreate is what is part of the current design and discussed up-thread IIRC. > What users can do to avoid their logical replication slots > getting invalidated and run into these situations? Because > recreating/reestablishing logical replication comes with cost > (sometimes huge) as it involves building another instance, syncing > tables etc. Isn't it a good idea to touch up on all these aspects in > the documentation at least as to why we're doing this and why we can't > do this? > 0005 adds a few words about it: + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> > 5. > @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) > > ReplicationSlotAcquire(cmd->slotname, true); > > + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) > + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) > + ereport(ERROR, > + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), > + errmsg("cannot read from logical replication slot \"%s\"", > + cmd->slotname), > + errdetail("This slot has been invalidated because it > was conflicting with recovery."))); > > Having the invalidation check in StartLogicalReplication() looks fine, > however, what happens if the slot gets invalidated when the > replication is in-progress? Do we need to error out in WalSndLoop() or > XLogSendLogical() too? Or is it already taken care of somewhere? > Yes, it's already taken care in pg_logical_slot_get_changes_guts(). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-11 16:52 Andres Freund <[email protected]> parent: Bharath Rupireddy <[email protected]> 1 sibling, 1 reply; 41+ messages in thread From: Andres Freund @ 2023-01-11 16:52 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; +Cc: Drouvot, Bertrand <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2023-01-11 13:02:13 +0530, Bharath Rupireddy wrote: > 3. Is this feature still a 'minimal logical decoding on standby'? > Firstly, why is it 'minimal'? It's minimal in comparison to other proposals at the time that did explicit / active coordination between primary and standby to allow logical decoding. > 0002: > 1. > - if (InvalidateObsoleteReplicationSlots(_logSegNo)) > + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, > &invalidated, InvalidOid, NULL); > > Isn't the function name too long and verbose? +1 > How about just InvalidateLogicalReplicationSlots() let the function comment > talk about what sorts of replication slots it invalides? I'd just leave the name unmodified at InvalidateObsoleteReplicationSlots(). > 2. > + errdetail("Logical decoding on > standby requires wal_level to be at least logical on master")); > + * master wal_level is set back to replica, so existing logical > slots need to > invalidate such slots. Also do the same thing if wal_level on master > > Can we use 'primary server' instead of 'master' like elsewhere? This > comment also applies for other patches too, if any. +1 > 3. Can we show a new status in pg_get_replication_slots's wal_status > for invalidated due to the conflict so that the user can monitor for > the new status and take necessary actions? Invalidated slots are not a new concept introduced in this patchset, so I'd say we can introduce such a field separately. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-18 10:24 Drouvot, Bertrand <[email protected]> parent: Andres Freund <[email protected]> 2 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-18 10:24 UTC (permalink / raw) To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/6/23 4:40 AM, Andres Freund wrote: > Hi, > 0004: > >> @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { >> >> =pod >> >> +=item $node->create_logical_slot_on_standby(self, master, slot_name, dbname) >> + >> +Create logical replication slot on given standby >> + >> +=cut >> + >> +sub create_logical_slot_on_standby >> +{ > > Any reason this has to be standby specific? > Due to the extra work to be done for this case (aka wait for restart_lsn and trigger a checkpoint on the primary). > >> + # Now arrange for the xl_running_xacts record for which pg_recvlogical >> + # is waiting. >> + $master->safe_psql('postgres', 'CHECKPOINT'); >> + > > Hm, that's quite expensive. Perhaps worth adding a C helper that can do that > for us instead? This will likely also be needed in real applications after all. > Not sure I got it. What the C helper would be supposed to do? > >> + print "starting pg_recvlogical\n"; > > I don't think tests should just print somewhere. Either diag() or note() > should be used. > > Will be done. >> + if ($wait) >> + # make sure activeslot is in use >> + { >> + $node_standby->poll_query_until('testdb', >> + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" >> + ) or die "slot never became active"; >> + } > > That comment placement imo is quite odd. > > Agree, will be done. >> +# test if basic decoding works >> +is(scalar(my @foobar = split /^/m, $result), >> + 14, 'Decoding produced 14 rows'); > > Maybe mention that it's 2 transactions + 10 rows? > > Agree, will be done. >> +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); > > There's enough copies of this that I wonder if we shouldn't introduce a > Cluster.pm level helper for this. > Done in [1]. > >> +print "waiting to replay $endpos\n"; > > See above. > > Will be done. >> +my $stdout_recv = $node_standby->pg_recvlogical_upto( >> + 'testdb', 'activeslot', $endpos, 180, >> + 'include-xids' => '0', >> + 'skip-empty-xacts' => '1'); > > I don't think this should use a hardcoded 180 but > $PostgreSQL::Test::Utils::timeout_default. > > Agree, will be done. >> +# One way to reproduce recovery conflict is to run VACUUM FULL with >> +# hot_standby_feedback turned off on the standby. >> +$node_standby->append_conf('postgresql.conf',q[ >> +hot_standby_feedback = off >> +]); >> +$node_standby->restart; > > IIRC a reload should suffice. > > Right. With a reload in place in my testing, now I notice that the catalog_xmin is updated on the primary physical slot after logical slots invalidation when reloading hot_standby_feedback from "off" to "on". This is not the case after a re-start (aka catalog_xmin is NULL). I think a re-start and reload should produce identical behavior on the primary physical slot. If so, I'm tempted to think that the catalog_xmin should be updated in case of a re-start too (even if all the logical slots are invalidated) because the slots are not dropped yet. What do you think? >> +# This should trigger the conflict >> +$node_primary->safe_psql('testdb', 'VACUUM FULL'); > > Can we do something cheaper than rewriting the entire database? Seems > rewriting a single table ought to be sufficient? > While implementing the test at the table level I discovered that It looks like there is no guarantee that say a "vacuum full pg_class;" would produce a conflict. Indeed, from what I can see in my testing it could generate a XLOG_HEAP2_PRUNE with snapshotConflictHorizon to 0: "rmgr: Heap2 len (rec/tot): 107/ 107, tx: 848, lsn: 0/03B98B30, prev 0/03B98AF0, desc: PRUNE snapshotConflictHorizon 0" Having a snapshotConflictHorizon to zero leads to ResolveRecoveryConflictWithSnapshot() simply returning without any conflict handling. It does look like that in the standby decoding case that's not the right behavior (and that the xid that generated the PRUNING should be used instead) , what do you think? > I think it'd also be good to test that rewriting a non-catalog table doesn't > trigger an issue. > > Good point, but need to understand the above first. >> +################################################## >> +# Recovery conflict: Invalidate conflicting slots, including in-use slots >> +# Scenario 2: conflict due to row removal with hot_standby_feedback off. >> +################################################## >> + >> +# get the position to search from in the standby logfile >> +my $logstart = -s $node_standby->logfile; >> + >> +# drop the logical slots >> +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); >> +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); >> + >> +create_logical_slots(); >> + >> +# One way to produce recovery conflict is to create/drop a relation and launch a vacuum >> +# with hot_standby_feedback turned off on the standby. >> +$node_standby->append_conf('postgresql.conf',q[ >> +hot_standby_feedback = off >> +]); >> +$node_standby->restart; >> +# ensure walreceiver feedback off by waiting for expected xmin and >> +# catalog_xmin on primary. Both should be NULL since hs_feedback is off >> +wait_for_xmins($node_primary, $primary_slotname, >> + "xmin IS NULL AND catalog_xmin IS NULL"); >> + >> +$handle = make_slot_active(1); > > This is a fair bit of repeated setup, maybe put it into a function? > > Yeah, good point: will be done. > I think it'd be good to test the ongoing decoding via the SQL interface also > gets correctly handled. But it might be too hard to do reliably. > > >> +################################################## >> +# Test standby promotion and logical decoding behavior >> +# after the standby gets promoted. >> +################################################## >> + > > I think this also should test the streaming / walsender case. > Do you mean cascading standby? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com [1]: https://www.postgresql.org/message-id/flat/846724b5-0723-f4c2-8b13-75301ec7509e%40gmail.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-19 02:46 Andres Freund <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Andres Freund @ 2023-01-19 02:46 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2023-01-18 11:24:19 +0100, Drouvot, Bertrand wrote: > On 1/6/23 4:40 AM, Andres Freund wrote: > > Hm, that's quite expensive. Perhaps worth adding a C helper that can do that > > for us instead? This will likely also be needed in real applications after all. > > > > Not sure I got it. What the C helper would be supposed to do? Call LogStandbySnapshot(). > With a reload in place in my testing, now I notice that the catalog_xmin > is updated on the primary physical slot after logical slots invalidation > when reloading hot_standby_feedback from "off" to "on". > > This is not the case after a re-start (aka catalog_xmin is NULL). > > I think a re-start and reload should produce identical behavior on > the primary physical slot. If so, I'm tempted to think that the catalog_xmin > should be updated in case of a re-start too (even if all the logical slots are invalidated) > because the slots are not dropped yet. What do you think? I can't quite follow the steps leading up to the difference. Could you list them in a bit more detail? > > Can we do something cheaper than rewriting the entire database? Seems > > rewriting a single table ought to be sufficient? > > > > While implementing the test at the table level I discovered that It looks like there is no guarantee that say a "vacuum full pg_class;" would > produce a conflict. I assume that's mostly when there weren't any removal > Indeed, from what I can see in my testing it could generate a XLOG_HEAP2_PRUNE with snapshotConflictHorizon to 0: > > "rmgr: Heap2 len (rec/tot): 107/ 107, tx: 848, lsn: 0/03B98B30, prev 0/03B98AF0, desc: PRUNE snapshotConflictHorizon 0" > > > Having a snapshotConflictHorizon to zero leads to ResolveRecoveryConflictWithSnapshot() simply returning > without any conflict handling. That doesn't have to mean anything bad. Some row versions can be removed without creating a conflict. See HeapTupleHeaderAdvanceConflictHorizon(), specifically * Ignore tuples inserted by an aborted transaction or if the tuple was * updated/deleted by the inserting transaction. > It does look like that in the standby decoding case that's not the right behavior (and that the xid that generated the PRUNING should be used instead) > , what do you think? That'd not work, because that'll be typically newer than the catalog_xmin. So we'd start invalidating things left and right, despite not needing to. Did you see anything else around this making you suspicious? > > > +################################################## > > > +# Test standby promotion and logical decoding behavior > > > +# after the standby gets promoted. > > > +################################################## > > > + > > > > I think this also should test the streaming / walsender case. > > > > Do you mean cascading standby? I mean a logical walsender that starts on a standby and continues across promotion of the standby. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-19 09:43 Drouvot, Bertrand <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 3 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-19 09:43 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/19/23 3:46 AM, Andres Freund wrote: > Hi, > > On 2023-01-18 11:24:19 +0100, Drouvot, Bertrand wrote: >> On 1/6/23 4:40 AM, Andres Freund wrote: >>> Hm, that's quite expensive. Perhaps worth adding a C helper that can do that >>> for us instead? This will likely also be needed in real applications after all. >>> >> >> Not sure I got it. What the C helper would be supposed to do? > > Call LogStandbySnapshot(). > Got it, I like the idea, will do. > >> With a reload in place in my testing, now I notice that the catalog_xmin >> is updated on the primary physical slot after logical slots invalidation >> when reloading hot_standby_feedback from "off" to "on". >> >> This is not the case after a re-start (aka catalog_xmin is NULL). >> >> I think a re-start and reload should produce identical behavior on >> the primary physical slot. If so, I'm tempted to think that the catalog_xmin >> should be updated in case of a re-start too (even if all the logical slots are invalidated) >> because the slots are not dropped yet. What do you think? > > I can't quite follow the steps leading up to the difference. Could you list > them in a bit more detail? > > Sure, so with: 1) hot_standby_feedback set to off on the standby 2) create 2 logical replication slots on the standby and activate one 3) Invalidate the logical slots on the standby with VACUUM FULL on the primary 4) change hot_standby_feedback to on on the standby If: 5) pg_reload_conf() on the standby, then on the primary we get a catalog_xmin for the physical slot that the standby is attached to: postgres=# select slot_type,xmin,catalog_xmin from pg_replication_slots ; slot_type | xmin | catalog_xmin -----------+------+-------------- physical | 822 | 748 (1 row) But if: 5) re-start the standby, then on the primary we get an empty catalog_xmin for the physical slot that the standby is attached to: postgres=# select slot_type,xmin,catalog_xmin from pg_replication_slots ; slot_type | xmin | catalog_xmin -----------+------+-------------- physical | 816 | (1 row) > >>> Can we do something cheaper than rewriting the entire database? Seems >>> rewriting a single table ought to be sufficient? >>> >> >> While implementing the test at the table level I discovered that It looks like there is no guarantee that say a "vacuum full pg_class;" would >> produce a conflict. > > I assume that's mostly when there weren't any removal > > >> Indeed, from what I can see in my testing it could generate a XLOG_HEAP2_PRUNE with snapshotConflictHorizon to 0: >> >> "rmgr: Heap2 len (rec/tot): 107/ 107, tx: 848, lsn: 0/03B98B30, prev 0/03B98AF0, desc: PRUNE snapshotConflictHorizon 0" >> >> >> Having a snapshotConflictHorizon to zero leads to ResolveRecoveryConflictWithSnapshot() simply returning >> without any conflict handling. > > That doesn't have to mean anything bad. Some row versions can be removed without > creating a conflict. See HeapTupleHeaderAdvanceConflictHorizon(), specifically > > * Ignore tuples inserted by an aborted transaction or if the tuple was > * updated/deleted by the inserting transaction. > > > >> It does look like that in the standby decoding case that's not the right behavior (and that the xid that generated the PRUNING should be used instead) >> , what do you think? > > That'd not work, because that'll be typically newer than the catalog_xmin. So > we'd start invalidating things left and right, despite not needing to. > > Okay, thanks for the explanations that makes sense. > Did you see anything else around this making you suspicious? > No, but a question still remains to me: Given the fact that the row removal case is already done in the next test (aka Scenario 2), If we want to replace the "vacuum full" test on the database (done in Scenario 1) with a cheaper one at the table level, what could it be to guarantee an invalidation? Same as scenario 2 but with "vacuum full pg_class" would not really add value to the tests, right? >>>> +################################################## >>>> +# Test standby promotion and logical decoding behavior >>>> +# after the standby gets promoted. >>>> +################################################## >>>> + >>> >>> I think this also should test the streaming / walsender case. >>> >> >> Do you mean cascading standby? > > I mean a logical walsender that starts on a standby and continues across > promotion of the standby. > Got it, thanks, will do. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-23 11:03 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 2 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-23 11:03 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/19/23 10:43 AM, Drouvot, Bertrand wrote: > Hi, > > On 1/19/23 3:46 AM, Andres Freund wrote: >> Hi, >> >> I mean a logical walsender that starts on a standby and continues across >> promotion of the standby. >> > > Got it, thanks, will do. > While working on it, I noticed that with V41 a: pg_recvlogical -S active_slot -P test_decoding -d postgres -f - --start on the standby is getting: pg_recvlogical: error: unexpected termination of replication stream: ERROR: could not find record while sending logically-decoded data: invalid record length at 0/311C438: wanted 24, got 0 pg_recvlogical: disconnected; waiting 5 seconds to try again when the standby gets promoted (the logical decoding is able to resume correctly after the error though). This is fixed in V42 attached (no error anymore and logical decoding through the walsender works correctly after the promotion). The fix is in 0003 where in logical_read_xlog_page() (as compare to V41): - We now check if RecoveryInProgress() (instead of relying on am_cascading_walsender) to check if the standby got promoted - Based on this, the currTLI is being retrieved with GetXLogReplayRecPtr() or GetWALInsertionTimeLine() (so, with GetWALInsertionTimeLine() after promotion) - This currTLI is being used as an argument in WALRead() (instead of state->seg.ws_tli, which anyhow sounds weird as being compared with itself that way "tli != state->seg.ws_tli" in WALRead()). That way WALRead() discovers that the timeline changed and then opens the right WAL file. Please find V42 attached. I'll resume working on the TAP tests comments. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From 54453b896174e9e28a5e27d4f749845e9260afb0 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:14:05 +0000 Subject: [PATCH v42 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From bcf71f46a01e4b84191627a6b94f0066c38f8301 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:13:23 +0000 Subject: [PATCH v42 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 37 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 479 ++++++++++++++++++ 3 files changed, 517 insertions(+) 6.0% src/test/perl/PostgreSQL/Test/ 93.7% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..6f3c9a6910 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, master, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $master, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $master->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4258844c8f --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,479 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use Test::More tests => 42; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + print "starting pg_recvlogical\n"; + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-f', '-', '--no-loop', '--start'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + # make sure activeslot is in use + { + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); +print "waiting to replay $endpos\n"; + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +create_logical_slots(); + +# One way to reproduce recovery conflict is to run VACUUM FULL with +# hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', 'VACUUM FULL'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and launch a vacuum +# with hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to row removal'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to row removal'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 3: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on master +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on master"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +create_logical_slots(); +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); -- 2.34.1 From 56f6baf2d4ed228caa0a147b7fd13ca6cdcadcf4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:12:15 +0000 Subject: [PATCH v42 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index c14d1f3ef6..45d170e008 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4921,3 +4930,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..b3fe5dbeb2 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1570,7 +1571,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1654,20 +1654,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From d9d4d4782a9827466e9724736c91977a34ba5b1f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:11:29 +0000 Subject: [PATCH v42 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.8% src/backend/access/transam/ 38.4% src/backend/replication/logical/ 55.8% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index a071fc6871..f14e1755b7 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..c1e43dd2b3 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..93a4fcf15a 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f22572be30..1f7a686cb1 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1175,37 +1176,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1221,6 +1213,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From e9d232145c57e03e7d01eaf8931014d6d1377ba2 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:10:24 +0000 Subject: [PATCH v42 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on master is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/monitoring.sgml | 11 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 3 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 191 +++++++++++++----- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 5 + src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 3 +- 22 files changed, 268 insertions(+), 55 deletions(-) 3.4% doc/src/sgml/ 8.5% src/backend/access/transam/ 5.3% src/backend/replication/logical/ 56.7% src/backend/replication/ 5.2% src/backend/storage/ipc/ 7.3% src/backend/tcop/ 5.5% src/backend/ 3.5% src/include/replication/ 3.4% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index d0cf1b2c81..5b1562fe0e 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 0e37bad213..b204dfe130 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8890,6 +8890,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9059,6 +9060,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9176,6 +9178,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..a071fc6871 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..4bd1aa401a 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1065,7 +1065,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..f22572be30 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -1224,20 +1224,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteOrConflictingLogicalReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1246,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1265,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,11 +1311,18 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - - /* Let caller know */ - *invalidated = true; + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + /* Let caller know */ + *invalidated = true; + } } SpinLockRelease(&s->mutex); @@ -1327,15 +1353,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1419,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1458,38 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * beeninvalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + *beeninvalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1500,35 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, beeninvalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any obsolete slots have been invalidated, recalculate the resource + * limits. */ - if (invalidated) + if (!xid && *beeninvalidated) { ReplicationSlotsComputeRequiredXmin(false); ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..daba766947 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..659e5bdc3a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..d392b5eec5 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..7d4831dffe 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1868,7 +1868,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 3c206bd77831d507f4f95e1942eb26855524571a Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:07:51 +0000 Subject: [PATCH v42 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing replication conflicts much as hot standby does. 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- contrib/amcheck/verify_nbtree.c | 17 ++-- src/backend/access/gist/gist.c | 2 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 14 ++- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 82 +++++++++--------- src/backend/access/nbtree/nbtpage.c | 99 ++++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 45 +++++----- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 7 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 4 +- src/include/access/gistxlog.h | 11 +-- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 8 +- src/include/access/nbtree.h | 31 ++++--- src/include/access/nbtxlog.h | 6 +- src/include/access/spgxlog.h | 1 + src/include/access/visibilitymapdefs.h | 9 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 3 +- 35 files changed, 234 insertions(+), 186 deletions(-) 4.8% contrib/amcheck/ 5.0% src/backend/access/gist/ 5.3% src/backend/access/heap/ 55.8% src/backend/access/nbtree/ 5.0% src/backend/access/ 3.3% src/backend/ 19.7% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..8d3abbdceb 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,7 +183,8 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, - IndexTuple itup); + IndexTuple itup, + Relation heaprel); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); static inline ItemPointer BTreeTupleGetHeapTIDCareful(BtreeCheckState *state, @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, &heapkeyspace, &allequalimage, heaprel); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, itup, state->heaprel); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, firstitup, state->heaprel); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, itup, state->heaprel); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL, state->heaprel); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, itup, heaprel); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..235f1a1843 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(heapRel, rel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..a87890b965 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(heap, index); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..119e34ce0f 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation heaprel, Relation r) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(heaprel, r, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..d0cf1b2c81 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation heaprel, Relation rel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 388df94a44..0e37bad213 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6871,6 +6871,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8441,7 +8442,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8453,6 +8454,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..30730c24bf 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,11 +720,16 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, maintenance_work_mem, - NULL, TUPLESORT_NONE); + NULL, TUPLESORT_NONE, NewHeap); else tuplesort = NULL; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..c48b272431 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate, + Relation heaprel); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,7 +42,7 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); +static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel); static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -50,14 +51,15 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page); + bool split_only_page, + Relation heaprel); static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, - IndexTuple nposting, uint16 postingoff); + IndexTuple nposting, uint16 postingoff, Relation heaprel); static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); + BTStack stack, bool isroot, bool isonly, Relation heaprel); +static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, itup, heapRel); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, &insertstate, heapRel); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -257,7 +259,7 @@ search: indexUnchanged, stack, heapRel); _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, itup, insertstate.itemsz, newitemoff, - insertstate.postingoff, false); + insertstate.postingoff, false, heapRel); } else { @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, BTInsertState insertstate, Relation heaprel) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -376,7 +378,7 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) /* Cannot use optimization -- descend tree, return proper descent stack */ return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + NULL, heaprel); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, rbuf, stack, heaprel); rbuf = InvalidBuffer; continue; } @@ -1107,7 +1109,8 @@ _bt_insertonpg(Relation rel, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page) + bool split_only_page, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1210,7 +1213,7 @@ _bt_insertonpg(Relation rel, /* split the buffer into left and right halves */ rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + origitup, nposting, postingoff, heaprel); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly, heaprel); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1461,7 +1464,8 @@ _bt_insertonpg(Relation rel, static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, - IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) + IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff, + Relation heaprel) { Buffer rbuf; Page origpage; @@ -1712,7 +1716,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1889,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE, heaprel); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2096,7 +2100,8 @@ _bt_insert_parent(Relation rel, Buffer rbuf, BTStack stack, bool isroot, - bool isonly) + bool isonly, + Relation heaprel) { /* * Here we have to do something Lehman and Yao don't talk about: deal with @@ -2118,7 +2123,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, buf, rbuf, heaprel); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2162,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL, + heaprel); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2189,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, stack, bknum, heaprel); /* * Unlock the right child. The left child will be unlocked in @@ -2209,7 +2215,7 @@ _bt_insert_parent(Relation rel, /* Recursively insert into the parent */ _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), - stack->bts_offset + 1, 0, isonly); + stack->bts_offset + 1, 0, isonly, heaprel); /* be tidy */ pfree(new_item); @@ -2227,7 +2233,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, Relation heaprel) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2246,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE, heaprel); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2258,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2275,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly, heaprel); } /* @@ -2304,7 +2310,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, Relation heaprel) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2324,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, blkno, BT_WRITE, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, buf, stack->bts_parent, heaprel); continue; } @@ -2428,7 +2434,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel) { Buffer rootbuf; Page lpage, @@ -2454,12 +2460,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..edca7aebb2 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -39,16 +39,19 @@ static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, - FullTransactionId safexid); + FullTransactionId safexid, + Relation heaprel); static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable); + BTVacuumPosting *updatable, int nupdatable, + Relation heaprel); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); + BTStack stack, + Relation heaprel); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, @@ -58,7 +61,8 @@ static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, - BlockNumber *topparentrightsib); + BlockNumber *topparentrightsib, + Relation heaprel); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, FullTransactionId safexid); @@ -178,7 +182,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +195,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +235,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, Relation heaprel) { Buffer metabuf; Page metapg; @@ -255,7 +259,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +344,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, int access, Relation heaprel) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +374,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, rootblkno, BT_READ, heaprel); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +400,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +433,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, access, heaprel); } /* @@ -437,7 +441,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +578,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +600,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +673,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +681,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +737,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage, Relation heaprel) { BTMetaPageData *metad; @@ -741,7 +745,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +829,7 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid, Relation heaprel) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +840,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +873,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, BlockNumber blkno, int access, Relation heaprel) { Buffer buf; @@ -944,7 +949,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) _bt_log_reuse_page(rel, blkno, - BTPageGetDeleteXid(page)); + BTPageGetDeleteXid(page), heaprel); /* Okay to use page. Re-initialize and return it. */ _bt_pageinit(page, BufferGetPageSize(buf)); @@ -1296,7 +1301,7 @@ static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable) + BTVacuumPosting *updatable, int nupdatable, Relation heaprel) { Page page = BufferGetPage(buf); BTPageOpaque opaque; @@ -1358,6 +1363,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1685,7 +1691,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, /* Physically delete tuples (or TIDs) using deletable (or updatable) */ _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + deletable, ndeletable, updatable, nupdatable, heapRel); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1712,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target, Relation heaprel) { Buffer buf; Page page; @@ -1717,7 +1723,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, leftsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1769,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib, Relation heaprel) { Buffer buf; Page page; @@ -1772,7 +1778,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, leafrightsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1967,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, leftsib, leafblkno, vstate->info->heaprel)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, targetkey, vstate->info->heaprel); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL, + vstate->info->heaprel); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2009,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, leafbuf, stack, vstate->info->heaprel)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2072,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); } } @@ -2084,7 +2091,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack, Relation heaprel) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2126,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, leafrightsib, heaprel)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2145,7 +2152,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) topparentrightsib = leafrightsib; if (!_bt_lock_subtree_parent(rel, leafblkno, stack, &subtreeparent, &poffset, - &topparent, &topparentrightsib)) + &topparent, &topparentrightsib, heaprel)) return false; /* @@ -2363,7 +2370,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, target, BT_READ, vstate->info->heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2397,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2447,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2511,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2540,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, + vstate->info->heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2775,7 +2783,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) + BlockNumber *topparent, BlockNumber *topparentrightsib, + Relation heaprel) { BlockNumber parent, leftsibparent; @@ -2789,7 +2798,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, stack, child, heaprel); if (pbuf == InvalidBuffer) { /* @@ -2889,13 +2898,13 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, leftsibparent, parent, heaprel)) return false; /* Recurse to examine child page's grandparent page */ return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, subtreeparent, poffset, - topparent, topparentrightsib); + topparent, topparentrightsib, heaprel); } /* diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..705716e333 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, num_delpages, info->heaprel); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..6466fe2f58 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,7 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -94,13 +94,13 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) */ BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, access, heaprel); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -130,7 +130,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * opportunity to finish splits of internal pages too. */ *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + page_access, snapshot, heaprel); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -191,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * move right to its new sibling. Do that. */ *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, - snapshot); + snapshot, heaprel); } return stack_in; @@ -239,7 +239,8 @@ _bt_moveright(Relation rel, bool forupdate, BTStack stack, int access, - Snapshot snapshot) + Snapshot snapshot, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -288,12 +289,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, buf, stack, heaprel); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, blkno, access, heaprel); continue; } @@ -860,6 +861,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1354,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage, heaprel); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1365,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot, heaprel); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2006,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, scan->heapRelation); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2080,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ, + scan->heapRelation); for (;;) { @@ -2093,7 +2096,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* Step to next physical page */ so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + scan->xs_snapshot, scan->heapRelation); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2143,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, + scan->heapRelation); } } } @@ -2185,7 +2189,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -2213,7 +2217,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, blkno, BT_READ, heaprel); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2305,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) */ Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { Buffer buf; Page page; @@ -2320,9 +2324,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, BT_READ, heaprel); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2407,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot, + scan->heapRelation); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..542029eec7 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, NULL, btspool->heap); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..9edd856371 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage, heaprel); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ, + scan->heapRelation); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..a8fc221153 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Buffer buffer, Relation heaprel) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..7d9c1c7eca 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -208,7 +208,8 @@ Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, - SortCoordinate coordinate, int sortopt) + SortCoordinate coordinate, int sortopt, + Relation heaprel) { Tuplesortstate *state = tuplesort_begin_common(workMem, coordinate, sortopt); @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heaprel); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heapRel); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..9bdac12baf 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -485,7 +485,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation heaprel, Relation r); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..191f0e5808 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,13 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +100,10 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..8f1dfedaaf 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,12 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; - - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + bool isCatalogRel; + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..1d43181a40 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,11 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +345,13 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +410,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..cdcfdd6030 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, + Relation heaprel); +extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, + Relation heaprel); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, + Relation heaprel); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); +extern Buffer _bt_getroot(Relation rel, int access, Relation heaprel); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); extern void _bt_metaversion(Relation rel, bool *heapkeyspace, - bool *allequalimage); + bool *allequalimage, Relation heaprel); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access, + Relation heaprel); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1230,20 +1234,21 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); * prototypes for functions in nbtsearch.c */ extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); + int access, Snapshot snapshot, Relation heaprel); extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); + bool forupdate, BTStack stack, int access, + Snapshot snapshot, Relation heaprel); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); + Snapshot snapshot, Relation heaprel); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..99d87d7189 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,10 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +236,14 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..29a6aa57a9 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,7 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..b27fdc0aef 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,10 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..06aebe6330 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -401,7 +401,8 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, SortCoordinate coordinate, - int sortopt); + int sortopt, + Relation heaprel); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, Relation indexRel, bool enforceUnique, -- 2.34.1 Attachments: [text/plain] v42-0006-Doc-changes-describing-details-about-logical-dec.patch (2.1K, ../../[email protected]/2-v42-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From 54453b896174e9e28a5e27d4f749845e9260afb0 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:14:05 +0000 Subject: [PATCH v42 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v42-0005-New-TAP-test-for-logical-decoding-on-standby.patch (20.4K, ../../[email protected]/3-v42-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From bcf71f46a01e4b84191627a6b94f0066c38f8301 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:13:23 +0000 Subject: [PATCH v42 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 37 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 479 ++++++++++++++++++ 3 files changed, 517 insertions(+) 6.0% src/test/perl/PostgreSQL/Test/ 93.7% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..6f3c9a6910 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, master, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $master, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $master->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4258844c8f --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,479 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use Test::More tests => 42; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + print "starting pg_recvlogical\n"; + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-f', '-', '--no-loop', '--start'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + # make sure activeslot is in use + { + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); +print "waiting to replay $endpos\n"; + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, 180, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +create_logical_slots(); + +# One way to reproduce recovery conflict is to run VACUUM FULL with +# hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', 'VACUUM FULL'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and launch a vacuum +# with hot_standby_feedback turned off on the standby. +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = off +]); +$node_standby->restart; +# ensure walreceiver feedback off by waiting for expected xmin and +# catalog_xmin on primary. Both should be NULL since hs_feedback is off +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to row removal'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to row removal'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +$node_standby->append_conf('postgresql.conf',q[ +hot_standby_feedback = on +]); +$node_standby->restart; + +# ensure walreceiver feedback sent by waiting for expected xmin and +# catalog_xmin on primary. With hot_standby_feedback on, xmin should advance, +# but catalog_xmin should still remain NULL since there is no logical slot. +wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 3: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); + +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across restarts +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on master +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on master"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +create_logical_slots(); +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); -- 2.34.1 [text/plain] v42-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.5K, ../../[email protected]/4-v42-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From 56f6baf2d4ed228caa0a147b7fd13ca6cdcadcf4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:12:15 +0000 Subject: [PATCH v42 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index c14d1f3ef6..45d170e008 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4921,3 +4930,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..b3fe5dbeb2 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1570,7 +1571,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1654,20 +1654,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v42-0003-Allow-logical-decoding-on-standby.patch (11.7K, ../../[email protected]/5-v42-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From d9d4d4782a9827466e9724736c91977a34ba5b1f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:11:29 +0000 Subject: [PATCH v42 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.8% src/backend/access/transam/ 38.4% src/backend/replication/logical/ 55.8% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index a071fc6871..f14e1755b7 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..c1e43dd2b3 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..93a4fcf15a 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires " + "wal_level to be at least logical on master"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f22572be30..1f7a686cb1 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1175,37 +1176,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1221,6 +1213,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v42-0002-Handle-logical-slot-conflicts-on-standby.patch (32.4K, ../../[email protected]/6-v42-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From e9d232145c57e03e7d01eaf8931014d6d1377ba2 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:10:24 +0000 Subject: [PATCH v42 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on master is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/monitoring.sgml | 11 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 3 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 191 +++++++++++++----- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 5 + src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 3 +- 22 files changed, 268 insertions(+), 55 deletions(-) 3.4% doc/src/sgml/ 8.5% src/backend/access/transam/ 5.3% src/backend/replication/logical/ 56.7% src/backend/replication/ 5.2% src/backend/storage/ipc/ 7.3% src/backend/tcop/ 5.5% src/backend/ 3.5% src/include/replication/ 3.4% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index d0cf1b2c81..5b1562fe0e 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 0e37bad213..b204dfe130 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8890,6 +8890,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9059,6 +9060,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9176,6 +9178,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..a071fc6871 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..4bd1aa401a 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1065,7 +1065,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..f22572be30 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -1224,20 +1224,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteOrConflictingLogicalReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1246,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1265,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,11 +1311,18 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - - /* Let caller know */ - *invalidated = true; + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + /* Let caller know */ + *invalidated = true; + } } SpinLockRelease(&s->mutex); @@ -1327,15 +1353,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1419,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on master")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1458,38 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * beeninvalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + *beeninvalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1500,35 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, beeninvalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any obsolete slots have been invalidated, recalculate the resource + * limits. */ - if (invalidated) + if (!xid && *beeninvalidated) { ReplicationSlotsComputeRequiredXmin(false); ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..daba766947 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteOrConflictingLogicalReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..659e5bdc3a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..d392b5eec5 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteOrConflictingLogicalReplicationSlots(XLogSegNo oldestSegno, bool *beeninvalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..7d4831dffe 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1868,7 +1868,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v42-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (72.1K, ../../[email protected]/7-v42-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 3c206bd77831d507f4f95e1942eb26855524571a Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 23 Jan 2023 10:07:51 +0000 Subject: [PATCH v42 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing replication conflicts much as hot standby does. 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- contrib/amcheck/verify_nbtree.c | 17 ++-- src/backend/access/gist/gist.c | 2 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 14 ++- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 82 +++++++++--------- src/backend/access/nbtree/nbtpage.c | 99 ++++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 45 +++++----- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 7 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 4 +- src/include/access/gistxlog.h | 11 +-- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 8 +- src/include/access/nbtree.h | 31 ++++--- src/include/access/nbtxlog.h | 6 +- src/include/access/spgxlog.h | 1 + src/include/access/visibilitymapdefs.h | 9 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 3 +- 35 files changed, 234 insertions(+), 186 deletions(-) 4.8% contrib/amcheck/ 5.0% src/backend/access/gist/ 5.3% src/backend/access/heap/ 55.8% src/backend/access/nbtree/ 5.0% src/backend/access/ 3.3% src/backend/ 19.7% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..8d3abbdceb 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,7 +183,8 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, - IndexTuple itup); + IndexTuple itup, + Relation heaprel); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); static inline ItemPointer BTreeTupleGetHeapTIDCareful(BtreeCheckState *state, @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, &heapkeyspace, &allequalimage, heaprel); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, itup, state->heaprel); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, firstitup, state->heaprel); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, itup, state->heaprel); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL, state->heaprel); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, itup, heaprel); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..235f1a1843 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(heapRel, rel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..a87890b965 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(heap, index); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..119e34ce0f 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation heaprel, Relation r) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(heaprel, r, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..d0cf1b2c81 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation heaprel, Relation rel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 388df94a44..0e37bad213 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6871,6 +6871,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8441,7 +8442,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8453,6 +8454,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..30730c24bf 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,11 +720,16 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, maintenance_work_mem, - NULL, TUPLESORT_NONE); + NULL, TUPLESORT_NONE, NewHeap); else tuplesort = NULL; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..c48b272431 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate, + Relation heaprel); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,7 +42,7 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); +static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel); static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -50,14 +51,15 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page); + bool split_only_page, + Relation heaprel); static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, - IndexTuple nposting, uint16 postingoff); + IndexTuple nposting, uint16 postingoff, Relation heaprel); static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); + BTStack stack, bool isroot, bool isonly, Relation heaprel); +static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, itup, heapRel); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, &insertstate, heapRel); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -257,7 +259,7 @@ search: indexUnchanged, stack, heapRel); _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, itup, insertstate.itemsz, newitemoff, - insertstate.postingoff, false); + insertstate.postingoff, false, heapRel); } else { @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, BTInsertState insertstate, Relation heaprel) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -376,7 +378,7 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) /* Cannot use optimization -- descend tree, return proper descent stack */ return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + NULL, heaprel); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, rbuf, stack, heaprel); rbuf = InvalidBuffer; continue; } @@ -1107,7 +1109,8 @@ _bt_insertonpg(Relation rel, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page) + bool split_only_page, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1210,7 +1213,7 @@ _bt_insertonpg(Relation rel, /* split the buffer into left and right halves */ rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + origitup, nposting, postingoff, heaprel); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly, heaprel); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1461,7 +1464,8 @@ _bt_insertonpg(Relation rel, static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, - IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) + IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff, + Relation heaprel) { Buffer rbuf; Page origpage; @@ -1712,7 +1716,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1889,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE, heaprel); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2096,7 +2100,8 @@ _bt_insert_parent(Relation rel, Buffer rbuf, BTStack stack, bool isroot, - bool isonly) + bool isonly, + Relation heaprel) { /* * Here we have to do something Lehman and Yao don't talk about: deal with @@ -2118,7 +2123,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, buf, rbuf, heaprel); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2162,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL, + heaprel); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2189,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, stack, bknum, heaprel); /* * Unlock the right child. The left child will be unlocked in @@ -2209,7 +2215,7 @@ _bt_insert_parent(Relation rel, /* Recursively insert into the parent */ _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), - stack->bts_offset + 1, 0, isonly); + stack->bts_offset + 1, 0, isonly, heaprel); /* be tidy */ pfree(new_item); @@ -2227,7 +2233,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, Relation heaprel) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2246,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE, heaprel); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2258,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2275,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly, heaprel); } /* @@ -2304,7 +2310,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, Relation heaprel) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2324,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, blkno, BT_WRITE, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, buf, stack->bts_parent, heaprel); continue; } @@ -2428,7 +2434,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel) { Buffer rootbuf; Page lpage, @@ -2454,12 +2460,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..edca7aebb2 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -39,16 +39,19 @@ static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, - FullTransactionId safexid); + FullTransactionId safexid, + Relation heaprel); static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable); + BTVacuumPosting *updatable, int nupdatable, + Relation heaprel); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); + BTStack stack, + Relation heaprel); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, @@ -58,7 +61,8 @@ static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, - BlockNumber *topparentrightsib); + BlockNumber *topparentrightsib, + Relation heaprel); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, FullTransactionId safexid); @@ -178,7 +182,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +195,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +235,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, Relation heaprel) { Buffer metabuf; Page metapg; @@ -255,7 +259,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +344,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, int access, Relation heaprel) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +374,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, rootblkno, BT_READ, heaprel); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +400,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +433,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, access, heaprel); } /* @@ -437,7 +441,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +578,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +600,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +673,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +681,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +737,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage, Relation heaprel) { BTMetaPageData *metad; @@ -741,7 +745,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +829,7 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid, Relation heaprel) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +840,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +873,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, BlockNumber blkno, int access, Relation heaprel) { Buffer buf; @@ -944,7 +949,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) _bt_log_reuse_page(rel, blkno, - BTPageGetDeleteXid(page)); + BTPageGetDeleteXid(page), heaprel); /* Okay to use page. Re-initialize and return it. */ _bt_pageinit(page, BufferGetPageSize(buf)); @@ -1296,7 +1301,7 @@ static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable) + BTVacuumPosting *updatable, int nupdatable, Relation heaprel) { Page page = BufferGetPage(buf); BTPageOpaque opaque; @@ -1358,6 +1363,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1685,7 +1691,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, /* Physically delete tuples (or TIDs) using deletable (or updatable) */ _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + deletable, ndeletable, updatable, nupdatable, heapRel); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1712,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target, Relation heaprel) { Buffer buf; Page page; @@ -1717,7 +1723,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, leftsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1769,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib, Relation heaprel) { Buffer buf; Page page; @@ -1772,7 +1778,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, leafrightsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1967,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, leftsib, leafblkno, vstate->info->heaprel)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, targetkey, vstate->info->heaprel); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL, + vstate->info->heaprel); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2009,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, leafbuf, stack, vstate->info->heaprel)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2072,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); } } @@ -2084,7 +2091,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack, Relation heaprel) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2126,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, leafrightsib, heaprel)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2145,7 +2152,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) topparentrightsib = leafrightsib; if (!_bt_lock_subtree_parent(rel, leafblkno, stack, &subtreeparent, &poffset, - &topparent, &topparentrightsib)) + &topparent, &topparentrightsib, heaprel)) return false; /* @@ -2363,7 +2370,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, target, BT_READ, vstate->info->heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2397,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2447,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2511,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2540,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, + vstate->info->heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2775,7 +2783,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) + BlockNumber *topparent, BlockNumber *topparentrightsib, + Relation heaprel) { BlockNumber parent, leftsibparent; @@ -2789,7 +2798,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, stack, child, heaprel); if (pbuf == InvalidBuffer) { /* @@ -2889,13 +2898,13 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, leftsibparent, parent, heaprel)) return false; /* Recurse to examine child page's grandparent page */ return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, subtreeparent, poffset, - topparent, topparentrightsib); + topparent, topparentrightsib, heaprel); } /* diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..705716e333 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, num_delpages, info->heaprel); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..6466fe2f58 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,7 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -94,13 +94,13 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) */ BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, access, heaprel); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -130,7 +130,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * opportunity to finish splits of internal pages too. */ *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + page_access, snapshot, heaprel); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -191,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * move right to its new sibling. Do that. */ *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, - snapshot); + snapshot, heaprel); } return stack_in; @@ -239,7 +239,8 @@ _bt_moveright(Relation rel, bool forupdate, BTStack stack, int access, - Snapshot snapshot) + Snapshot snapshot, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -288,12 +289,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, buf, stack, heaprel); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, blkno, access, heaprel); continue; } @@ -860,6 +861,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1354,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage, heaprel); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1365,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot, heaprel); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2006,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, scan->heapRelation); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2080,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ, + scan->heapRelation); for (;;) { @@ -2093,7 +2096,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* Step to next physical page */ so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + scan->xs_snapshot, scan->heapRelation); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2143,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, + scan->heapRelation); } } } @@ -2185,7 +2189,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -2213,7 +2217,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, blkno, BT_READ, heaprel); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2305,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) */ Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { Buffer buf; Page page; @@ -2320,9 +2324,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, BT_READ, heaprel); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2407,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot, + scan->heapRelation); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..542029eec7 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, NULL, btspool->heap); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..9edd856371 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage, heaprel); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ, + scan->heapRelation); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..a8fc221153 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Buffer buffer, Relation heaprel) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..7d9c1c7eca 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -208,7 +208,8 @@ Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, - SortCoordinate coordinate, int sortopt) + SortCoordinate coordinate, int sortopt, + Relation heaprel) { Tuplesortstate *state = tuplesort_begin_common(workMem, coordinate, sortopt); @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heaprel); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heapRel); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..9bdac12baf 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -485,7 +485,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation heaprel, Relation r); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..191f0e5808 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,13 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +100,10 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..8f1dfedaaf 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,12 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; - - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + bool isCatalogRel; + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..1d43181a40 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,11 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +345,13 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +410,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..cdcfdd6030 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, + Relation heaprel); +extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, + Relation heaprel); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, + Relation heaprel); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); +extern Buffer _bt_getroot(Relation rel, int access, Relation heaprel); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); extern void _bt_metaversion(Relation rel, bool *heapkeyspace, - bool *allequalimage); + bool *allequalimage, Relation heaprel); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access, + Relation heaprel); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1230,20 +1234,21 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); * prototypes for functions in nbtsearch.c */ extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); + int access, Snapshot snapshot, Relation heaprel); extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); + bool forupdate, BTStack stack, int access, + Snapshot snapshot, Relation heaprel); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); + Snapshot snapshot, Relation heaprel); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..99d87d7189 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,10 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +236,14 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..29a6aa57a9 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,7 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..b27fdc0aef 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,10 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..06aebe6330 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -401,7 +401,8 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, SortCoordinate coordinate, - int sortopt); + int sortopt, + Relation heaprel); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, Relation indexRel, bool enforceUnique, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-23 23:21 Melanie Plageman <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Melanie Plageman @ 2023-01-23 23:21 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; +Cc: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers I'm new to this thread and subject, but I had a few basic thoughts about the first patch in the set. On Mon, Jan 23, 2023 at 12:03:35PM +0100, Drouvot, Bertrand wrote: > > Please find V42 attached. > > From 3c206bd77831d507f4f95e1942eb26855524571a Mon Sep 17 00:00:00 2001 > From: bdrouvotAWS <[email protected]> > Date: Mon, 23 Jan 2023 10:07:51 +0000 > Subject: [PATCH v42 1/6] Add info in WAL records in preparation for logical > slot conflict handling. > MIME-Version: 1.0 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > Overall design: > > 1. We want to enable logical decoding on standbys, but replay of WAL > from the primary might remove data that is needed by logical decoding, > causing replication conflicts much as hot standby does. It is a little confusing to mention replication conflicts in point 1. It makes it sound like it already logs a recovery conflict. Without the recovery conflict handling in this patchset, logical decoding of statements using data that has been removed will fail with some error like : ERROR: could not map filenumber "xxx" to relation OID Part of what this patchset does is introduce the concept of a new kind of recovery conflict and a resolution process. > 2. Our chosen strategy for dealing with this type of replication slot > is to invalidate logical slots for which needed data has been removed. > > 3. To do this we need the latestRemovedXid for each change, just as we > do for physical replication conflicts, but we also need to know > whether any particular change was to data that logical replication > might access. It isn't clear from the above sentence why you would need both. I think it has something to do with what is below (hot_standby_feedback being off), but I'm not sure, so the order is confusing. > 4. We can't rely on the standby's relcache entries for this purpose in > any way, because the startup process can't access catalog contents. > > 5. Therefore every WAL record that potentially removes data from the > index or heap must carry a flag indicating whether or not it is one > that might be accessed during logical decoding. > > Why do we need this for logical decoding on standby? > > First, let's forget about logical decoding on standby and recall that > on a primary database, any catalog rows that may be needed by a logical > decoding replication slot are not removed. > > This is done thanks to the catalog_xmin associated with the logical > replication slot. > > But, with logical decoding on standby, in the following cases: > > - hot_standby_feedback is off > - hot_standby_feedback is on but there is no a physical slot between > the primary and the standby. Then, hot_standby_feedback will work, > but only while the connection is alive (for example a node restart > would break it) > > Then, the primary may delete system catalog rows that could be needed > by the logical decoding on the standby (as it does not know about the > catalog_xmin on the standby). > > So, it’s mandatory to identify those rows and invalidate the slots > that may need them if any. Identifying those rows is the purpose of > this commit. I would like a few more specifics about this commit (patch 1 in the set) itself in the commit message. I think it would be good to have the commit message mention what kinds of operations require WAL to contain information about whether or not it is operating on a catalog table and why this is. For example, I think the explanation of the feature makes it clear why vacuum and pruning operations would require isCatalogRel, however it isn't immediately obvious why page reuse would. Also, because the diff has so many function signatures changed, it is hard to tell which xlog record types are actually being changed to include isCatalogRel. It might be too detailed/repetitive for the final commit message to have a list of the xlog types requiring this info (gistxlogPageReuse, spgxlogVacuumRedirect, xl_hash_vacuum_one_page, xl_btree_reuse_page, xl_btree_delete, xl_heap_visible, xl_heap_prune, xl_heap_freeze_page) but perhaps you could enumerate all the general operations (freeze page, vacuum, prune, etc). > Implementation: > > When a WAL replay on standby indicates that a catalog table tuple is > to be deleted by an xid that is greater than a logical slot's > catalog_xmin, then that means the slot's catalog_xmin conflicts with > the xid, and we need to handle the conflict. While subsequent commits > will do the actual conflict handling, this commit adds a new field > isCatalogRel in such WAL records (and a new bit set in the > xl_heap_visible flags field), that is true for catalog tables, so as to > arrange for conflict handling. You do mention it a bit here, but I think it could be more clear and specific. > diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c > index ba394f08f6..235f1a1843 100644 > --- a/src/backend/access/gist/gist.c > +++ b/src/backend/access/gist/gist.c > @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, > for (; ptr; ptr = ptr->next) > { > /* Allocate new page */ > - ptr->buffer = gistNewBuffer(rel); > + ptr->buffer = gistNewBuffer(heapRel, rel); > GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); > ptr->page = BufferGetPage(ptr->buffer); > ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); > diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c > index d21a308d41..a87890b965 100644 > --- a/src/backend/access/gist/gistbuild.c > +++ b/src/backend/access/gist/gistbuild.c > @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) > Page page; > > /* initialize the root page */ > - buffer = gistNewBuffer(index); > + buffer = gistNewBuffer(heap, index); > Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); > page = BufferGetPage(buffer); > > diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c > index 56451fede1..119e34ce0f 100644 > --- a/src/backend/access/gist/gistutil.c > +++ b/src/backend/access/gist/gistutil.c > @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) > * Caller is responsible for initializing the page by calling GISTInitBuffer > */ > Buffer > -gistNewBuffer(Relation r) > +gistNewBuffer(Relation heaprel, Relation r) > { It is not very important, but I noticed you made "heaprel" the last parameter to all of the btree-related functions but the first parameter to the gist functions. I thought it might be nice to make the order consistent. I also was wondering why you made it the last argument to all the btree functions to begin with (i.e. instead of directly after the first rel argument). > diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h > index 8af33d7b40..9bdac12baf 100644 > --- a/src/include/access/gist_private.h > +++ b/src/include/access/gist_private.h > @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, > FullTransactionId xid, Buffer parentBuffer, > OffsetNumber downlinkOffset); > > -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, > +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, > FullTransactionId deleteXid); > > extern XLogRecPtr gistXLogUpdate(Buffer buffer, > @@ -485,7 +485,7 @@ extern bool gistproperty(Oid index_oid, int attno, > extern bool gistfitpage(IndexTuple *itvec, int len); > extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); > extern void gistcheckpage(Relation rel, Buffer buf); > -extern Buffer gistNewBuffer(Relation r); > +extern Buffer gistNewBuffer(Relation heaprel, Relation r); > extern bool gistPageRecyclable(Page page); > extern void gistfillbuffer(Page page, IndexTuple *itup, int len, > OffsetNumber off); > diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h > index 09f9b0f8c6..191f0e5808 100644 > --- a/src/include/access/gistxlog.h > +++ b/src/include/access/gistxlog.h > @@ -51,13 +51,13 @@ typedef struct gistxlogDelete > { > TransactionId snapshotConflictHorizon; > uint16 ntodelete; /* number of deleted offsets */ > + bool isCatalogRel; In some of these struct definitions, I think it would help comprehension to have a comment explaining the purpose of this member. > > - /* > - * In payload of blk 0 : todelete OffsetNumbers > - */ > + /* TODELETE OFFSET NUMBERS */ > + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; Thanks for all your hard work on this feature! Best, Melanie ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-24 00:46 Andres Freund <[email protected]> parent: Drouvot, Bertrand <[email protected]> 2 siblings, 1 reply; 41+ messages in thread From: Andres Freund @ 2023-01-24 00:46 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2023-01-19 10:43:27 +0100, Drouvot, Bertrand wrote: > > > With a reload in place in my testing, now I notice that the catalog_xmin > > > is updated on the primary physical slot after logical slots invalidation > > > when reloading hot_standby_feedback from "off" to "on". > > > > > > This is not the case after a re-start (aka catalog_xmin is NULL). > > > > > > I think a re-start and reload should produce identical behavior on > > > the primary physical slot. If so, I'm tempted to think that the catalog_xmin > > > should be updated in case of a re-start too (even if all the logical slots are invalidated) > > > because the slots are not dropped yet. What do you think? > > > > I can't quite follow the steps leading up to the difference. Could you list > > them in a bit more detail? > > > > > > Sure, so with: > > 1) hot_standby_feedback set to off on the standby > 2) create 2 logical replication slots on the standby and activate one > 3) Invalidate the logical slots on the standby with VACUUM FULL on the primary > 4) change hot_standby_feedback to on on the standby > > If: > > 5) pg_reload_conf() on the standby, then on the primary we get a catalog_xmin > for the physical slot that the standby is attached to: > > postgres=# select slot_type,xmin,catalog_xmin from pg_replication_slots ; > slot_type | xmin | catalog_xmin > -----------+------+-------------- > physical | 822 | 748 > (1 row) How long did you wait for this to change? I don't think there's anything right now that'd force a new hot-standby-feedback message to be sent to the primary, after slots got invalidated. I suspect that if you terminated the walsender connection on the primary, you'd not see it anymore either? If that isn't it, something is broken in InvalidateObsolete... > No, but a question still remains to me: > > Given the fact that the row removal case is already done > in the next test (aka Scenario 2), If we want to replace the "vacuum full" test > on the database (done in Scenario 1) with a cheaper one at the table level, > what could it be to guarantee an invalidation? > > Same as scenario 2 but with "vacuum full pg_class" would not really add value > to the tests, right? A database wide VACUUM FULL is also just a row removal test, no? I think it makes sense to test that both VACUUM and VACUUM FULL both trigger conflicts, because they internally use *very* different mechanisms. It'd probably be good to test at least conflicts triggered due to row removal via on-access pruning as well. And perhaps also for btree killtuples. I think those are the common cases for catalog tables. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-24 05:20 Drouvot, Bertrand <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-24 05:20 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/24/23 1:46 AM, Andres Freund wrote: > Hi, > > On 2023-01-19 10:43:27 +0100, Drouvot, Bertrand wrote: >>>> With a reload in place in my testing, now I notice that the catalog_xmin >>>> is updated on the primary physical slot after logical slots invalidation >>>> when reloading hot_standby_feedback from "off" to "on". >>>> >>>> This is not the case after a re-start (aka catalog_xmin is NULL). >>>> >>>> I think a re-start and reload should produce identical behavior on >>>> the primary physical slot. If so, I'm tempted to think that the catalog_xmin >>>> should be updated in case of a re-start too (even if all the logical slots are invalidated) >>>> because the slots are not dropped yet. What do you think? >>> >>> I can't quite follow the steps leading up to the difference. Could you list >>> them in a bit more detail? >>> >>> >> >> Sure, so with: >> >> 1) hot_standby_feedback set to off on the standby >> 2) create 2 logical replication slots on the standby and activate one >> 3) Invalidate the logical slots on the standby with VACUUM FULL on the primary >> 4) change hot_standby_feedback to on on the standby >> >> If: >> >> 5) pg_reload_conf() on the standby, then on the primary we get a catalog_xmin >> for the physical slot that the standby is attached to: >> >> postgres=# select slot_type,xmin,catalog_xmin from pg_replication_slots ; >> slot_type | xmin | catalog_xmin >> -----------+------+-------------- >> physical | 822 | 748 >> (1 row) > > How long did you wait for this to change? Almost instantaneous after pg_reload_conf() on the standby. > I don't think there's anything right > now that'd force a new hot-standby-feedback message to be sent to the primary, > after slots got invalidated. > > I suspect that if you terminated the walsender connection on the primary, > you'd not see it anymore either? > Still there after the standby is shutdown but disappears when the standby is re-started. > If that isn't it, something is broken in InvalidateObsolete... > Will look at what's going on and ensure catalog_xmin is not sent to the primary after pg_reload_conf() (if the slots are invalidated). > >> No, but a question still remains to me: >> >> Given the fact that the row removal case is already done >> in the next test (aka Scenario 2), If we want to replace the "vacuum full" test >> on the database (done in Scenario 1) with a cheaper one at the table level, >> what could it be to guarantee an invalidation? >> >> Same as scenario 2 but with "vacuum full pg_class" would not really add value >> to the tests, right? > > A database wide VACUUM FULL is also just a row removal test, no? Yeah, so I was wondering if Scenario 1 was simply not just useless. > I think it > makes sense to test that both VACUUM and VACUUM FULL both trigger conflicts, > because they internally use *very* different mechanisms. Got it, will do and replace Scenario 1 as you suggested initially. > It'd probably be > good to test at least conflicts triggered due to row removal via on-access > pruning as well. And perhaps also for btree killtuples. I think those are the > common cases for catalog tables. > Thanks for the proposal, will look at it. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-24 14:31 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-24 14:31 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/24/23 6:20 AM, Drouvot, Bertrand wrote: > Hi, > > On 1/24/23 1:46 AM, Andres Freund wrote: >> Hi, >> >> On 2023-01-19 10:43:27 +0100, Drouvot, Bertrand wrote: >>> Sure, so with: >>> >>> 1) hot_standby_feedback set to off on the standby >>> 2) create 2 logical replication slots on the standby and activate one >>> 3) Invalidate the logical slots on the standby with VACUUM FULL on the primary >>> 4) change hot_standby_feedback to on on the standby >>> >>> If: >>> >>> 5) pg_reload_conf() on the standby, then on the primary we get a catalog_xmin >>> for the physical slot that the standby is attached to: >>> >>> postgres=# select slot_type,xmin,catalog_xmin from pg_replication_slots ; >>> slot_type | xmin | catalog_xmin >>> -----------+------+-------------- >>> physical | 822 | 748 >>> (1 row) >> >> How long did you wait for this to change? > > Almost instantaneous after pg_reload_conf() on the standby. > >> I don't think there's anything right >> now that'd force a new hot-standby-feedback message to be sent to the primary, >> after slots got invalidated. >> >> I suspect that if you terminated the walsender connection on the primary, >> you'd not see it anymore either? >> > > Still there after the standby is shutdown but disappears when the standby is re-started. > >> If that isn't it, something is broken in InvalidateObsolete... >> Yeah, you are right: ReplicationSlotsComputeRequiredXmin() is missing for the logical slot invalidation case (and ReplicationSlotsComputeRequiredXmin() also needs to take care of it). I'll provide a fix in the next revision along with the TAP tests comments addressed. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-24 14:59 Drouvot, Bertrand <[email protected]> parent: Melanie Plageman <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-24 14:59 UTC (permalink / raw) To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/24/23 12:21 AM, Melanie Plageman wrote: > I'm new to this thread and subject, but I had a few basic thoughts about > the first patch in the set. > Thanks for looking at it! > On Mon, Jan 23, 2023 at 12:03:35PM +0100, Drouvot, Bertrand wrote: >> >> Please find V42 attached. >> >> From 3c206bd77831d507f4f95e1942eb26855524571a Mon Sep 17 00:00:00 2001 >> From: bdrouvotAWS <[email protected]> >> Date: Mon, 23 Jan 2023 10:07:51 +0000 >> Subject: [PATCH v42 1/6] Add info in WAL records in preparation for logical >> slot conflict handling. >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=UTF-8 >> Content-Transfer-Encoding: 8bit >> >> Overall design: >> >> 1. We want to enable logical decoding on standbys, but replay of WAL >> from the primary might remove data that is needed by logical decoding, >> causing replication conflicts much as hot standby does. > > It is a little confusing to mention replication conflicts in point 1. It > makes it sound like it already logs a recovery conflict. Without the > recovery conflict handling in this patchset, logical decoding of > statements using data that has been removed will fail with some error > like : > ERROR: could not map filenumber "xxx" to relation OID > Part of what this patchset does is introduce the concept of a new kind > of recovery conflict and a resolution process. > I think I understand what you mean, what about the following? 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors a new replication conflict scenario needs to be addressed (as much as hot standby does.) >> 2. Our chosen strategy for dealing with this type of replication slot >> is to invalidate logical slots for which needed data has been removed. >> >> 3. To do this we need the latestRemovedXid for each change, just as we >> do for physical replication conflicts, but we also need to know >> whether any particular change was to data that logical replication >> might access. > It isn't clear from the above sentence why you would need both. I think > it has something to do with what is below (hot_standby_feedback being > off), but I'm not sure, so the order is confusing. > Right, it has to deal with the xid horizons too. So the idea is to check if 1) there is a risk of conflict and 2) if there is a risk then check is there a conflict? (based on the xid). I'll reword this part. >> 4. We can't rely on the standby's relcache entries for this purpose in >> any way, because the startup process can't access catalog contents. >> >> 5. Therefore every WAL record that potentially removes data from the >> index or heap must carry a flag indicating whether or not it is one >> that might be accessed during logical decoding. >> >> Why do we need this for logical decoding on standby? >> >> First, let's forget about logical decoding on standby and recall that >> on a primary database, any catalog rows that may be needed by a logical >> decoding replication slot are not removed. >> >> This is done thanks to the catalog_xmin associated with the logical >> replication slot. >> >> But, with logical decoding on standby, in the following cases: >> >> - hot_standby_feedback is off >> - hot_standby_feedback is on but there is no a physical slot between >> the primary and the standby. Then, hot_standby_feedback will work, >> but only while the connection is alive (for example a node restart >> would break it) >> >> Then, the primary may delete system catalog rows that could be needed >> by the logical decoding on the standby (as it does not know about the >> catalog_xmin on the standby). >> >> So, it’s mandatory to identify those rows and invalidate the slots >> that may need them if any. Identifying those rows is the purpose of >> this commit. > > I would like a few more specifics about this commit (patch 1 in the set) > itself in the commit message. > > I think it would be good to have the commit message mention what kinds > of operations require WAL to contain information about whether or not it > is operating on a catalog table and why this is. > > For example, I think the explanation of the feature makes it clear why > vacuum and pruning operations would require isCatalogRel, however it > isn't immediately obvious why page reuse would. > What do you think about putting those extra explanations in the code instead? > Also, because the diff has so many function signatures changed, it is > hard to tell which xlog record types are actually being changed to > include isCatalogRel. It might be too detailed/repetitive for the final > commit message to have a list of the xlog types requiring this info > (gistxlogPageReuse, spgxlogVacuumRedirect, xl_hash_vacuum_one_page, > xl_btree_reuse_page, xl_btree_delete, xl_heap_visible, xl_heap_prune, > xl_heap_freeze_page) but perhaps you could enumerate all the general > operations (freeze page, vacuum, prune, etc). > Right, at the end there is only a few making "real" use of it: they can be enumerated in the commit message. Will do. >> Implementation: >> >> When a WAL replay on standby indicates that a catalog table tuple is >> to be deleted by an xid that is greater than a logical slot's >> catalog_xmin, then that means the slot's catalog_xmin conflicts with >> the xid, and we need to handle the conflict. While subsequent commits >> will do the actual conflict handling, this commit adds a new field >> isCatalogRel in such WAL records (and a new bit set in the >> xl_heap_visible flags field), that is true for catalog tables, so as to >> arrange for conflict handling. > > You do mention it a bit here, but I think it could be more clear and > specific. Ok, will try to be more clear. > >> diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c >> index ba394f08f6..235f1a1843 100644 >> --- a/src/backend/access/gist/gist.c >> +++ b/src/backend/access/gist/gist.c >> @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, >> for (; ptr; ptr = ptr->next) >> { >> /* Allocate new page */ >> - ptr->buffer = gistNewBuffer(rel); >> + ptr->buffer = gistNewBuffer(heapRel, rel); >> GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); >> ptr->page = BufferGetPage(ptr->buffer); >> ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); >> diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c >> index d21a308d41..a87890b965 100644 >> --- a/src/backend/access/gist/gistbuild.c >> +++ b/src/backend/access/gist/gistbuild.c >> @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) >> Page page; >> >> /* initialize the root page */ >> - buffer = gistNewBuffer(index); >> + buffer = gistNewBuffer(heap, index); >> Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); >> page = BufferGetPage(buffer); >> >> diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c >> index 56451fede1..119e34ce0f 100644 >> --- a/src/backend/access/gist/gistutil.c >> +++ b/src/backend/access/gist/gistutil.c >> @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) >> * Caller is responsible for initializing the page by calling GISTInitBuffer >> */ >> Buffer >> -gistNewBuffer(Relation r) >> +gistNewBuffer(Relation heaprel, Relation r) >> { > > It is not very important, but I noticed you made "heaprel" the last > parameter to all of the btree-related functions but the first parameter > to the gist functions. I thought it might be nice to make the order > consistent. Agree, will do. > I also was wondering why you made it the last argument to > all the btree functions to begin with (i.e. instead of directly after > the first rel argument). > No real reasons, will put all of them after the first rel argument (that seems a better place). >> diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h >> index 8af33d7b40..9bdac12baf 100644 >> --- a/src/include/access/gist_private.h >> +++ b/src/include/access/gist_private.h >> @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, >> FullTransactionId xid, Buffer parentBuffer, >> OffsetNumber downlinkOffset); >> >> -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, >> +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, >> FullTransactionId deleteXid); >> >> extern XLogRecPtr gistXLogUpdate(Buffer buffer, >> @@ -485,7 +485,7 @@ extern bool gistproperty(Oid index_oid, int attno, >> extern bool gistfitpage(IndexTuple *itvec, int len); >> extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); >> extern void gistcheckpage(Relation rel, Buffer buf); >> -extern Buffer gistNewBuffer(Relation r); >> +extern Buffer gistNewBuffer(Relation heaprel, Relation r); >> extern bool gistPageRecyclable(Page page); >> extern void gistfillbuffer(Page page, IndexTuple *itup, int len, >> OffsetNumber off); >> diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h >> index 09f9b0f8c6..191f0e5808 100644 >> --- a/src/include/access/gistxlog.h >> +++ b/src/include/access/gistxlog.h >> @@ -51,13 +51,13 @@ typedef struct gistxlogDelete >> { >> TransactionId snapshotConflictHorizon; >> uint16 ntodelete; /* number of deleted offsets */ >> + bool isCatalogRel; > > In some of these struct definitions, I think it would help comprehension > to have a comment explaining the purpose of this member. > Yeah, agree but it could be done in another patch (outside of this feature), agree? > Thanks for all your hard work on this feature! Thanks for the review and the feedback! Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-27 12:09 Drouvot, Bertrand <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-27 12:09 UTC (permalink / raw) To: Andres Freund <[email protected]>; Bharath Rupireddy <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/11/23 5:52 PM, Andres Freund wrote: > Hi, > > On 2023-01-11 13:02:13 +0530, Bharath Rupireddy wrote: >> 3. Is this feature still a 'minimal logical decoding on standby'? >> Firstly, why is it 'minimal'? > > It's minimal in comparison to other proposals at the time that did explicit / > active coordination between primary and standby to allow logical decoding. > > > >> 0002: >> 1. >> - if (InvalidateObsoleteReplicationSlots(_logSegNo)) >> + InvalidateObsoleteOrConflictingLogicalReplicationSlots(_logSegNo, >> &invalidated, InvalidOid, NULL); >> >> Isn't the function name too long and verbose? > > +1 > > >> How about just InvalidateLogicalReplicationSlots() let the function comment >> talk about what sorts of replication slots it invalides? > > I'd just leave the name unmodified at InvalidateObsoleteReplicationSlots(). > > Done in V44 attached. >> 2. >> + errdetail("Logical decoding on >> standby requires wal_level to be at least logical on master")); >> + * master wal_level is set back to replica, so existing logical >> slots need to >> invalidate such slots. Also do the same thing if wal_level on master >> >> Can we use 'primary server' instead of 'master' like elsewhere? This >> comment also applies for other patches too, if any. > > +1 > Done in V44. > >> 3. Can we show a new status in pg_get_replication_slots's wal_status >> for invalidated due to the conflict so that the user can monitor for >> the new status and take necessary actions? > > Invalidated slots are not a new concept introduced in this patchset, so I'd > say we can introduce such a field separately. > In V44, adding a new field "conflicting" in pg_replication_slots which is: - NULL for physical slots - True if the slot is a logical one and has been invalidated due to recovery conflict - False if the slot is a logical one and has not been invalidated due to recovery conflict I'm not checking if recovery is in progress while displaying the "conflicting". The reason is to still display the right status after a promotion. TAP tests are also updated to test that this new field behaves as expected (for both physical and logical slots). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From 84ee1e9e5a69590abfe33ba4fa3929f03f3b4074 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 11:32:44 +0000 Subject: [PATCH v44 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 712f9dfe724ae0f8be9ce151de4a547e00cb9f8f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 11:31:47 +0000 Subject: [PATCH v44 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 658 ++++++++++++++++++ 3 files changed, 698 insertions(+) 5.0% src/test/perl/PostgreSQL/Test/ 94.8% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4370d595d8 --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,658 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 62; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + { + # make sure activeslot is in use + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); +$handle = make_slot_active(1); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); -- 2.34.1 From a8b5832e9e5843caa22e0b283cf5ed4118aa55c4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:56:44 +0000 Subject: [PATCH v44 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2a5352f879..bb0de527ab 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4923,3 +4932,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..b3fe5dbeb2 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1570,7 +1571,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1654,20 +1654,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 28cba7f1e6799d3190c416dc324762da53091663 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:55:44 +0000 Subject: [PATCH v44 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 867675d5a1..1abe747cb5 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 84fcef80f9a0b1037ce4c8208caae583de33e098 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:50:51 +0000 Subject: [PATCH v44 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 743ee363c5..04a9b40271 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d478724b9d..d64fb4cc84 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8891,6 +8891,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9060,6 +9061,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9177,6 +9179,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..867675d5a1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 083ab448e6940d5f53c303958a4f8434281fd2a9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:48:07 +0000 Subject: [PATCH v44 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing replication conflicts much as hot standby does. 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- contrib/amcheck/verify_nbtree.c | 17 ++-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 82 +++++++++--------- src/backend/access/nbtree/nbtpage.c | 99 ++++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 45 +++++----- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 7 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 11 +-- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 8 +- src/include/access/nbtree.h | 31 ++++--- src/include/access/nbtxlog.h | 6 +- src/include/access/spgxlog.h | 1 + src/include/access/visibilitymapdefs.h | 9 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 3 +- 35 files changed, 240 insertions(+), 189 deletions(-) 4.7% contrib/amcheck/ 6.3% src/backend/access/gist/ 5.2% src/backend/access/heap/ 54.6% src/backend/access/nbtree/ 4.9% src/backend/access/ 3.2% src/backend/ 20.1% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..8d3abbdceb 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,7 +183,8 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, - IndexTuple itup); + IndexTuple itup, + Relation heaprel); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); static inline ItemPointer BTreeTupleGetHeapTIDCareful(BtreeCheckState *state, @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, &heapkeyspace, &allequalimage, heaprel); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, itup, state->heaprel); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, firstitup, state->heaprel); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, itup, state->heaprel); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL, state->heaprel); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, itup, heaprel); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..f8bc488d4f 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(heapRel, rel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..a87890b965 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(heap, index); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..119e34ce0f 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation heaprel, Relation r) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(heaprel, r, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..743ee363c5 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation heaprel, Relation rel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e6024a980b..d478724b9d 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6872,6 +6872,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8442,7 +8443,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8454,6 +8455,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..30730c24bf 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,11 +720,16 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, maintenance_work_mem, - NULL, TUPLESORT_NONE); + NULL, TUPLESORT_NONE, NewHeap); else tuplesort = NULL; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..c48b272431 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate, + Relation heaprel); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,7 +42,7 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); +static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel); static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -50,14 +51,15 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page); + bool split_only_page, + Relation heaprel); static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, - IndexTuple nposting, uint16 postingoff); + IndexTuple nposting, uint16 postingoff, Relation heaprel); static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); + BTStack stack, bool isroot, bool isonly, Relation heaprel); +static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, itup, heapRel); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, &insertstate, heapRel); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -257,7 +259,7 @@ search: indexUnchanged, stack, heapRel); _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, itup, insertstate.itemsz, newitemoff, - insertstate.postingoff, false); + insertstate.postingoff, false, heapRel); } else { @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, BTInsertState insertstate, Relation heaprel) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -376,7 +378,7 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) /* Cannot use optimization -- descend tree, return proper descent stack */ return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + NULL, heaprel); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, rbuf, stack, heaprel); rbuf = InvalidBuffer; continue; } @@ -1107,7 +1109,8 @@ _bt_insertonpg(Relation rel, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page) + bool split_only_page, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1210,7 +1213,7 @@ _bt_insertonpg(Relation rel, /* split the buffer into left and right halves */ rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + origitup, nposting, postingoff, heaprel); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly, heaprel); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1461,7 +1464,8 @@ _bt_insertonpg(Relation rel, static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, - IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) + IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff, + Relation heaprel) { Buffer rbuf; Page origpage; @@ -1712,7 +1716,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1889,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE, heaprel); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2096,7 +2100,8 @@ _bt_insert_parent(Relation rel, Buffer rbuf, BTStack stack, bool isroot, - bool isonly) + bool isonly, + Relation heaprel) { /* * Here we have to do something Lehman and Yao don't talk about: deal with @@ -2118,7 +2123,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, buf, rbuf, heaprel); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2162,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL, + heaprel); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2189,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, stack, bknum, heaprel); /* * Unlock the right child. The left child will be unlocked in @@ -2209,7 +2215,7 @@ _bt_insert_parent(Relation rel, /* Recursively insert into the parent */ _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), - stack->bts_offset + 1, 0, isonly); + stack->bts_offset + 1, 0, isonly, heaprel); /* be tidy */ pfree(new_item); @@ -2227,7 +2233,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, Relation heaprel) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2246,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE, heaprel); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2258,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2275,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly, heaprel); } /* @@ -2304,7 +2310,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, Relation heaprel) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2324,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, blkno, BT_WRITE, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, buf, stack->bts_parent, heaprel); continue; } @@ -2428,7 +2434,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel) { Buffer rootbuf; Page lpage, @@ -2454,12 +2460,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..edca7aebb2 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -39,16 +39,19 @@ static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, - FullTransactionId safexid); + FullTransactionId safexid, + Relation heaprel); static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable); + BTVacuumPosting *updatable, int nupdatable, + Relation heaprel); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); + BTStack stack, + Relation heaprel); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, @@ -58,7 +61,8 @@ static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, - BlockNumber *topparentrightsib); + BlockNumber *topparentrightsib, + Relation heaprel); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, FullTransactionId safexid); @@ -178,7 +182,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +195,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +235,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, Relation heaprel) { Buffer metabuf; Page metapg; @@ -255,7 +259,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +344,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, int access, Relation heaprel) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +374,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, rootblkno, BT_READ, heaprel); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +400,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +433,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, access, heaprel); } /* @@ -437,7 +441,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +578,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +600,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +673,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +681,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +737,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage, Relation heaprel) { BTMetaPageData *metad; @@ -741,7 +745,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +829,7 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid, Relation heaprel) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +840,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +873,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, BlockNumber blkno, int access, Relation heaprel) { Buffer buf; @@ -944,7 +949,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) _bt_log_reuse_page(rel, blkno, - BTPageGetDeleteXid(page)); + BTPageGetDeleteXid(page), heaprel); /* Okay to use page. Re-initialize and return it. */ _bt_pageinit(page, BufferGetPageSize(buf)); @@ -1296,7 +1301,7 @@ static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable) + BTVacuumPosting *updatable, int nupdatable, Relation heaprel) { Page page = BufferGetPage(buf); BTPageOpaque opaque; @@ -1358,6 +1363,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1685,7 +1691,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, /* Physically delete tuples (or TIDs) using deletable (or updatable) */ _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + deletable, ndeletable, updatable, nupdatable, heapRel); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1712,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target, Relation heaprel) { Buffer buf; Page page; @@ -1717,7 +1723,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, leftsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1769,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib, Relation heaprel) { Buffer buf; Page page; @@ -1772,7 +1778,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, leafrightsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1967,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, leftsib, leafblkno, vstate->info->heaprel)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, targetkey, vstate->info->heaprel); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL, + vstate->info->heaprel); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2009,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, leafbuf, stack, vstate->info->heaprel)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2072,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); } } @@ -2084,7 +2091,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack, Relation heaprel) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2126,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, leafrightsib, heaprel)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2145,7 +2152,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) topparentrightsib = leafrightsib; if (!_bt_lock_subtree_parent(rel, leafblkno, stack, &subtreeparent, &poffset, - &topparent, &topparentrightsib)) + &topparent, &topparentrightsib, heaprel)) return false; /* @@ -2363,7 +2370,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, target, BT_READ, vstate->info->heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2397,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2447,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2511,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2540,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, + vstate->info->heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2775,7 +2783,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) + BlockNumber *topparent, BlockNumber *topparentrightsib, + Relation heaprel) { BlockNumber parent, leftsibparent; @@ -2789,7 +2798,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, stack, child, heaprel); if (pbuf == InvalidBuffer) { /* @@ -2889,13 +2898,13 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, leftsibparent, parent, heaprel)) return false; /* Recurse to examine child page's grandparent page */ return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, subtreeparent, poffset, - topparent, topparentrightsib); + topparent, topparentrightsib, heaprel); } /* diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..705716e333 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, num_delpages, info->heaprel); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..6466fe2f58 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,7 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -94,13 +94,13 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) */ BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, access, heaprel); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -130,7 +130,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * opportunity to finish splits of internal pages too. */ *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + page_access, snapshot, heaprel); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -191,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * move right to its new sibling. Do that. */ *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, - snapshot); + snapshot, heaprel); } return stack_in; @@ -239,7 +239,8 @@ _bt_moveright(Relation rel, bool forupdate, BTStack stack, int access, - Snapshot snapshot) + Snapshot snapshot, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -288,12 +289,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, buf, stack, heaprel); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, blkno, access, heaprel); continue; } @@ -860,6 +861,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1354,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage, heaprel); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1365,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot, heaprel); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2006,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, scan->heapRelation); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2080,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ, + scan->heapRelation); for (;;) { @@ -2093,7 +2096,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* Step to next physical page */ so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + scan->xs_snapshot, scan->heapRelation); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2143,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, + scan->heapRelation); } } } @@ -2185,7 +2189,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -2213,7 +2217,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, blkno, BT_READ, heaprel); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2305,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) */ Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { Buffer buf; Page page; @@ -2320,9 +2324,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, BT_READ, heaprel); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2407,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot, + scan->heapRelation); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..542029eec7 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, NULL, btspool->heap); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..9edd856371 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage, heaprel); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ, + scan->heapRelation); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..a8fc221153 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Buffer buffer, Relation heaprel) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..7d9c1c7eca 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -208,7 +208,8 @@ Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, - SortCoordinate coordinate, int sortopt) + SortCoordinate coordinate, int sortopt, + Relation heaprel) { Tuplesortstate *state = tuplesort_begin_common(workMem, coordinate, sortopt); @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heaprel); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heapRel); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..b76ed4c6f8 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation heaprel, Relation r); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..191f0e5808 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,13 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +100,10 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..8f1dfedaaf 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,12 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; - - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + bool isCatalogRel; + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..1d43181a40 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,11 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +345,13 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +410,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..cdcfdd6030 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, + Relation heaprel); +extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, + Relation heaprel); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, + Relation heaprel); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); +extern Buffer _bt_getroot(Relation rel, int access, Relation heaprel); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); extern void _bt_metaversion(Relation rel, bool *heapkeyspace, - bool *allequalimage); + bool *allequalimage, Relation heaprel); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access, + Relation heaprel); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1230,20 +1234,21 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); * prototypes for functions in nbtsearch.c */ extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); + int access, Snapshot snapshot, Relation heaprel); extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); + bool forupdate, BTStack stack, int access, + Snapshot snapshot, Relation heaprel); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); + Snapshot snapshot, Relation heaprel); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..99d87d7189 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,10 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +236,14 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..29a6aa57a9 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,7 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..b27fdc0aef 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,10 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..06aebe6330 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -401,7 +401,8 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, SortCoordinate coordinate, - int sortopt); + int sortopt, + Relation heaprel); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, Relation indexRel, bool enforceUnique, -- 2.34.1 Attachments: [text/plain] v44-0006-Doc-changes-describing-details-about-logical-dec.patch (2.1K, ../../[email protected]/2-v44-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From 84ee1e9e5a69590abfe33ba4fa3929f03f3b4074 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 11:32:44 +0000 Subject: [PATCH v44 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v44-0005-New-TAP-test-for-logical-decoding-on-standby.patch (26.5K, ../../[email protected]/3-v44-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 712f9dfe724ae0f8be9ce151de4a547e00cb9f8f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 11:31:47 +0000 Subject: [PATCH v44 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 658 ++++++++++++++++++ 3 files changed, 698 insertions(+) 5.0% src/test/perl/PostgreSQL/Test/ 94.8% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4370d595d8 --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,658 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 62; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + { + # make sure activeslot is in use + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); +$handle = make_slot_active(1); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); -- 2.34.1 [text/plain] v44-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.5K, ../../[email protected]/4-v44-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From a8b5832e9e5843caa22e0b283cf5ed4118aa55c4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:56:44 +0000 Subject: [PATCH v44 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2a5352f879..bb0de527ab 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4923,3 +4932,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..b3fe5dbeb2 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1570,7 +1571,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1654,20 +1654,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v44-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v44-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From 28cba7f1e6799d3190c416dc324762da53091663 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:55:44 +0000 Subject: [PATCH v44 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 867675d5a1..1abe747cb5 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v44-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v44-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 84fcef80f9a0b1037ce4c8208caae583de33e098 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:50:51 +0000 Subject: [PATCH v44 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 743ee363c5..04a9b40271 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d478724b9d..d64fb4cc84 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8891,6 +8891,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9060,6 +9061,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9177,6 +9179,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..867675d5a1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v44-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (73.3K, ../../[email protected]/7-v44-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 083ab448e6940d5f53c303958a4f8434281fd2a9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Fri, 27 Jan 2023 09:48:07 +0000 Subject: [PATCH v44 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing replication conflicts much as hot standby does. 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- contrib/amcheck/verify_nbtree.c | 17 ++-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 82 +++++++++--------- src/backend/access/nbtree/nbtpage.c | 99 ++++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 45 +++++----- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 7 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 11 +-- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 8 +- src/include/access/nbtree.h | 31 ++++--- src/include/access/nbtxlog.h | 6 +- src/include/access/spgxlog.h | 1 + src/include/access/visibilitymapdefs.h | 9 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 3 +- 35 files changed, 240 insertions(+), 189 deletions(-) 4.7% contrib/amcheck/ 6.3% src/backend/access/gist/ 5.2% src/backend/access/heap/ 54.6% src/backend/access/nbtree/ 4.9% src/backend/access/ 3.2% src/backend/ 20.1% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..8d3abbdceb 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,7 +183,8 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, - IndexTuple itup); + IndexTuple itup, + Relation heaprel); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); static inline ItemPointer BTreeTupleGetHeapTIDCareful(BtreeCheckState *state, @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, &heapkeyspace, &allequalimage, heaprel); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, itup, state->heaprel); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, firstitup, state->heaprel); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, itup, state->heaprel); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL, state->heaprel); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, itup, heaprel); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..f8bc488d4f 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(heapRel, rel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..a87890b965 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(heap, index); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..119e34ce0f 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation heaprel, Relation r) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(heaprel, r, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..743ee363c5 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation heaprel, Relation rel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e6024a980b..d478724b9d 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6872,6 +6872,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8442,7 +8443,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8454,6 +8455,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..30730c24bf 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,11 +720,16 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, maintenance_work_mem, - NULL, TUPLESORT_NONE); + NULL, TUPLESORT_NONE, NewHeap); else tuplesort = NULL; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..c48b272431 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate, + Relation heaprel); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,7 +42,7 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); +static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel); static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -50,14 +51,15 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page); + bool split_only_page, + Relation heaprel); static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, - IndexTuple nposting, uint16 postingoff); + IndexTuple nposting, uint16 postingoff, Relation heaprel); static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); + BTStack stack, bool isroot, bool isonly, Relation heaprel); +static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, itup, heapRel); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, &insertstate, heapRel); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -257,7 +259,7 @@ search: indexUnchanged, stack, heapRel); _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, itup, insertstate.itemsz, newitemoff, - insertstate.postingoff, false); + insertstate.postingoff, false, heapRel); } else { @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, BTInsertState insertstate, Relation heaprel) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -376,7 +378,7 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) /* Cannot use optimization -- descend tree, return proper descent stack */ return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + NULL, heaprel); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, insertstate, stack, heapRel); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, rbuf, stack, heaprel); rbuf = InvalidBuffer; continue; } @@ -1107,7 +1109,8 @@ _bt_insertonpg(Relation rel, Size itemsz, OffsetNumber newitemoff, int postingoff, - bool split_only_page) + bool split_only_page, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -1210,7 +1213,7 @@ _bt_insertonpg(Relation rel, /* split the buffer into left and right halves */ rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + origitup, nposting, postingoff, heaprel); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly, heaprel); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1461,7 +1464,8 @@ _bt_insertonpg(Relation rel, static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, - IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) + IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff, + Relation heaprel) { Buffer rbuf; Page origpage; @@ -1712,7 +1716,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1889,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE, heaprel); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2096,7 +2100,8 @@ _bt_insert_parent(Relation rel, Buffer rbuf, BTStack stack, bool isroot, - bool isonly) + bool isonly, + Relation heaprel) { /* * Here we have to do something Lehman and Yao don't talk about: deal with @@ -2118,7 +2123,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, buf, rbuf, heaprel); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2162,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL, + heaprel); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2189,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, stack, bknum, heaprel); /* * Unlock the right child. The left child will be unlocked in @@ -2209,7 +2215,7 @@ _bt_insert_parent(Relation rel, /* Recursively insert into the parent */ _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), - stack->bts_offset + 1, 0, isonly); + stack->bts_offset + 1, 0, isonly, heaprel); /* be tidy */ pfree(new_item); @@ -2227,7 +2233,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, Relation heaprel) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2246,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE, heaprel); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2258,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2275,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly, heaprel); } /* @@ -2304,7 +2310,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, Relation heaprel) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2324,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, blkno, BT_WRITE, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, buf, stack->bts_parent, heaprel); continue; } @@ -2428,7 +2434,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf, Relation heaprel) { Buffer rootbuf; Page lpage, @@ -2454,12 +2460,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..edca7aebb2 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -39,16 +39,19 @@ static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, - FullTransactionId safexid); + FullTransactionId safexid, + Relation heaprel); static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable); + BTVacuumPosting *updatable, int nupdatable, + Relation heaprel); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); + BTStack stack, + Relation heaprel); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, @@ -58,7 +61,8 @@ static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, - BlockNumber *topparentrightsib); + BlockNumber *topparentrightsib, + Relation heaprel); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, FullTransactionId safexid); @@ -178,7 +182,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +195,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +235,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, Relation heaprel) { Buffer metabuf; Page metapg; @@ -255,7 +259,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +344,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, int access, Relation heaprel) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +374,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, rootblkno, BT_READ, heaprel); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +400,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +433,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, access, heaprel); } /* @@ -437,7 +441,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE, heaprel); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +578,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +600,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +673,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +681,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +737,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage, Relation heaprel) { BTMetaPageData *metad; @@ -741,7 +745,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ, heaprel); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +829,7 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid, Relation heaprel) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +840,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +873,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, BlockNumber blkno, int access, Relation heaprel) { Buffer buf; @@ -944,7 +949,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) _bt_log_reuse_page(rel, blkno, - BTPageGetDeleteXid(page)); + BTPageGetDeleteXid(page), heaprel); /* Okay to use page. Re-initialize and return it. */ _bt_pageinit(page, BufferGetPageSize(buf)); @@ -1296,7 +1301,7 @@ static void _bt_delitems_delete(Relation rel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, - BTVacuumPosting *updatable, int nupdatable) + BTVacuumPosting *updatable, int nupdatable, Relation heaprel) { Page page = BufferGetPage(buf); BTPageOpaque opaque; @@ -1358,6 +1363,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1685,7 +1691,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, /* Physically delete tuples (or TIDs) using deletable (or updatable) */ _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + deletable, ndeletable, updatable, nupdatable, heapRel); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1712,7 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target, Relation heaprel) { Buffer buf; Page page; @@ -1717,7 +1723,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, leftsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1769,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib, Relation heaprel) { Buffer buf; Page page; @@ -1772,7 +1778,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, leafrightsib, BT_READ, heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1967,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, leftsib, leafblkno, vstate->info->heaprel)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, targetkey, vstate->info->heaprel); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL, + vstate->info->heaprel); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2009,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, leafbuf, stack, vstate->info->heaprel)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2072,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); } } @@ -2084,7 +2091,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack, Relation heaprel) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2126,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, leafrightsib, heaprel)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2145,7 +2152,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) topparentrightsib = leafrightsib; if (!_bt_lock_subtree_parent(rel, leafblkno, stack, &subtreeparent, &poffset, - &topparent, &topparentrightsib)) + &topparent, &topparentrightsib, heaprel)) return false; /* @@ -2363,7 +2370,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, target, BT_READ, vstate->info->heaprel); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2397,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2447,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, leftsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2511,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, rightsib, BT_WRITE, vstate->info->heaprel); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2540,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE, + vstate->info->heaprel); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2775,7 +2783,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) + BlockNumber *topparent, BlockNumber *topparentrightsib, + Relation heaprel) { BlockNumber parent, leftsibparent; @@ -2789,7 +2798,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, stack, child, heaprel); if (pbuf == InvalidBuffer) { /* @@ -2889,13 +2898,13 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, leftsibparent, parent, heaprel)) return false; /* Recurse to examine child page's grandparent page */ return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, subtreeparent, poffset, - topparent, topparentrightsib); + topparent, topparentrightsib, heaprel); } /* diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..705716e333 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, num_delpages, info->heaprel); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..6466fe2f58 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,7 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -94,13 +94,13 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) */ BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, access, heaprel); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -130,7 +130,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * opportunity to finish splits of internal pages too. */ *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + page_access, snapshot, heaprel); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -191,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * move right to its new sibling. Do that. */ *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, - snapshot); + snapshot, heaprel); } return stack_in; @@ -239,7 +239,8 @@ _bt_moveright(Relation rel, bool forupdate, BTStack stack, int access, - Snapshot snapshot) + Snapshot snapshot, + Relation heaprel) { Page page; BTPageOpaque opaque; @@ -288,12 +289,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, buf, stack, heaprel); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, blkno, access, heaprel); continue; } @@ -860,6 +861,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1354,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage, heaprel); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1365,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot, heaprel); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2006,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, scan->heapRelation); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2080,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ, + scan->heapRelation); for (;;) { @@ -2093,7 +2096,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* Step to next physical page */ so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + scan->xs_snapshot, scan->heapRelation); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2143,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ, + scan->heapRelation); } } } @@ -2185,7 +2189,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot, Relation heaprel) { Page page; BTPageOpaque opaque; @@ -2213,7 +2217,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, blkno, BT_READ, heaprel); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2305,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) */ Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot) + Snapshot snapshot, Relation heaprel) { Buffer buf; Page page; @@ -2320,9 +2324,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, BT_READ, heaprel); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2407,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot, + scan->heapRelation); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..542029eec7 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, NULL, btspool->heap); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..9edd856371 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage, heaprel); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ, + scan->heapRelation); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..a8fc221153 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Buffer buffer, Relation heaprel) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, buffer, bds->info->heaprel); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..7d9c1c7eca 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -208,7 +208,8 @@ Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, - SortCoordinate coordinate, int sortopt) + SortCoordinate coordinate, int sortopt, + Relation heaprel) { Tuplesortstate *state = tuplesort_begin_common(workMem, coordinate, sortopt); @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heaprel); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, NULL, heapRel); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..b76ed4c6f8 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation heaprel, Relation r); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..191f0e5808 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,13 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +100,10 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..8f1dfedaaf 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,12 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; - - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + bool isCatalogRel; + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..1d43181a40 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,11 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +345,13 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +410,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..cdcfdd6030 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack, + Relation heaprel); +extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child, + Relation heaprel); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages, + Relation heaprel); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); +extern Buffer _bt_getroot(Relation rel, int access, Relation heaprel); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); extern void _bt_metaversion(Relation rel, bool *heapkeyspace, - bool *allequalimage); + bool *allequalimage, Relation heaprel); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access, + Relation heaprel); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1230,20 +1234,21 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); * prototypes for functions in nbtsearch.c */ extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); + int access, Snapshot snapshot, Relation heaprel); extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); + bool forupdate, BTStack stack, int access, + Snapshot snapshot, Relation heaprel); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); + Snapshot snapshot, Relation heaprel); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup, Relation heaprel); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..99d87d7189 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,10 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +236,14 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..29a6aa57a9 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,7 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..b27fdc0aef 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,10 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..06aebe6330 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -401,7 +401,8 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, int workMem, SortCoordinate coordinate, - int sortopt); + int sortopt, + Relation heaprel); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, Relation indexRel, bool enforceUnique, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-30 16:18 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-30 16:18 UTC (permalink / raw) To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers; Melanie Plageman <[email protected]> Hi, Just realized that Melanie was missing in the up-thread reply to her feedback (not sure what happened, sorry about that).... So, adding her here. Please find attached V45 addressing Melanie's feedback. On 1/24/23 3:59 PM, Drouvot, Bertrand wrote: > Hi, > > On 1/24/23 12:21 AM, Melanie Plageman wrote: >> I'm new to this thread and subject, but I had a few basic thoughts about >> the first patch in the set. >> > > Thanks for looking at it! > >> On Mon, Jan 23, 2023 at 12:03:35PM +0100, Drouvot, Bertrand wrote: >>> 1. We want to enable logical decoding on standbys, but replay of WAL >>> from the primary might remove data that is needed by logical decoding, >>> causing replication conflicts much as hot standby does. >> >> It is a little confusing to mention replication conflicts in point 1. It >> makes it sound like it already logs a recovery conflict. Without the >> recovery conflict handling in this patchset, logical decoding of >> statements using data that has been removed will fail with some error >> like : >> ERROR: could not map filenumber "xxx" to relation OID >> Part of what this patchset does is introduce the concept of a new kind >> of recovery conflict and a resolution process. Changed the wording in V45's commit message. >>> 3. To do this we need the latestRemovedXid for each change, just as we >>> do for physical replication conflicts, but we also need to know >>> whether any particular change was to data that logical replication >>> might access. > >> It isn't clear from the above sentence why you would need both. I think >> it has something to do with what is below (hot_standby_feedback being >> off), but I'm not sure, so the order is confusing. > Trying to be more clear in the new commit message. > >>> Implementation: >>> >>> When a WAL replay on standby indicates that a catalog table tuple is >>> to be deleted by an xid that is greater than a logical slot's >>> catalog_xmin, then that means the slot's catalog_xmin conflicts with >>> the xid, and we need to handle the conflict. While subsequent commits >>> will do the actual conflict handling, this commit adds a new field >>> isCatalogRel in such WAL records (and a new bit set in the >>> xl_heap_visible flags field), that is true for catalog tables, so as to >>> arrange for conflict handling. >> >> You do mention it a bit here, but I think it could be more clear and >> specific. > Added the WAL record types impacted by the change in the new commit message. >> >> It is not very important, but I noticed you made "heaprel" the last >> parameter to all of the btree-related functions but the first parameter >> to the gist functions. I thought it might be nice to make the order >> consistent. > > Agree, will do. Done. > >> I also was wondering why you made it the last argument to >> all the btree functions to begin with (i.e. instead of directly after >> the first rel argument). >> > > No real reasons, will put all of them after the first rel argument (that seems a better place). Done. > >>> diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h >>> index 8af33d7b40..9bdac12baf 100644 >>> --- a/src/include/access/gist_private.h >>> +++ b/src/include/access/gist_private.h >>> @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, >>> FullTransactionId xid, Buffer parentBuffer, >>> OffsetNumber downlinkOffset); >>> -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, >>> +extern void gistXLogPageReuse(Relation heaprel, Relation rel, BlockNumber blkno, >>> FullTransactionId deleteXid); >>> extern XLogRecPtr gistXLogUpdate(Buffer buffer, >>> @@ -485,7 +485,7 @@ extern bool gistproperty(Oid index_oid, int attno, >>> extern bool gistfitpage(IndexTuple *itvec, int len); >>> extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); >>> extern void gistcheckpage(Relation rel, Buffer buf); >>> -extern Buffer gistNewBuffer(Relation r); >>> +extern Buffer gistNewBuffer(Relation heaprel, Relation r); >>> extern bool gistPageRecyclable(Page page); >>> extern void gistfillbuffer(Page page, IndexTuple *itup, int len, >>> OffsetNumber off); >>> diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h >>> index 09f9b0f8c6..191f0e5808 100644 >>> --- a/src/include/access/gistxlog.h >>> +++ b/src/include/access/gistxlog.h >>> @@ -51,13 +51,13 @@ typedef struct gistxlogDelete >>> { >>> TransactionId snapshotConflictHorizon; >>> uint16 ntodelete; /* number of deleted offsets */ >>> + bool isCatalogRel; >> >> In some of these struct definitions, I think it would help comprehension >> to have a comment explaining the purpose of this member. >> > > Yeah, agree but it could be done in another patch (outside of this feature), agree? Please forget about my previous reply (I misunderstood and thought you were mentioning the offset's Array). Added comments about isCatalogRel in V45 attached. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From f45d02edeeed24e604c1e48cf39c09224eac1644 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:44:00 +0000 Subject: [PATCH v45 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 61471ca687aa95eb334fd23448836bc8100ed92f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:43:21 +0000 Subject: [PATCH v45 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 658 ++++++++++++++++++ 3 files changed, 698 insertions(+) 5.0% src/test/perl/PostgreSQL/Test/ 94.8% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4370d595d8 --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,658 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 62; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + { + # make sure activeslot is in use + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); +$handle = make_slot_active(1); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); -- 2.34.1 From 31f671b55fc9eb9eba2799182ae50c20ba6ed6c9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:42:26 +0000 Subject: [PATCH v45 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2a5352f879..bb0de527ab 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4923,3 +4932,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..b3fe5dbeb2 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1570,7 +1571,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1654,20 +1654,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From d184a98f9c9aeefdcfbeeebab3aba4bf0cee9815 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:41:38 +0000 Subject: [PATCH v45 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 867675d5a1..1abe747cb5 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 700c26ef8ee1c5caa40c6e0ce14796f7662c6f97 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:40:56 +0000 Subject: [PATCH v45 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d478724b9d..d64fb4cc84 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8891,6 +8891,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9060,6 +9061,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9177,6 +9179,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..867675d5a1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 7ab0a6ec046f0c181c8f3b96cb18ae938f573ba9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:30:15 +0000 Subject: [PATCH v45 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..4462022904 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..aad14a401d 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e6024a980b..d478724b9d 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6872,6 +6872,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8442,7 +8443,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8454,6 +8455,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..70a0c2418a 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v45-0006-Doc-changes-describing-details-about-logical-dec.patch (2.1K, ../../[email protected]/2-v45-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From f45d02edeeed24e604c1e48cf39c09224eac1644 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:44:00 +0000 Subject: [PATCH v45 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v45-0005-New-TAP-test-for-logical-decoding-on-standby.patch (26.5K, ../../[email protected]/3-v45-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 61471ca687aa95eb334fd23448836bc8100ed92f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:43:21 +0000 Subject: [PATCH v45 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 ++ src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 658 ++++++++++++++++++ 3 files changed, 698 insertions(+) 5.0% src/test/perl/PostgreSQL/Test/ 94.8% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index edaaa1a3ce..52b2816c7a 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..4370d595d8 --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,658 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 62; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + { + # make sure activeslot is in use + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); +$handle = make_slot_active(1); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); -- 2.34.1 [text/plain] v45-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.5K, ../../[email protected]/4-v45-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From 31f671b55fc9eb9eba2799182ae50c20ba6ed6c9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:42:26 +0000 Subject: [PATCH v45 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 ++++++++++++++++++++ src/backend/replication/walsender.c | 31 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 59 insertions(+), 8 deletions(-) 41.2% src/backend/access/transam/ 48.5% src/backend/replication/ 3.6% src/backend/utils/activity/ 3.4% src/include/access/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2a5352f879..bb0de527ab 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see getReplayedCV() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4923,3 +4932,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +getReplayedCV(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..b3fe5dbeb2 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = getReplayedCV(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1570,7 +1571,6 @@ WalSndWaitForWal(XLogRecPtr loc) for (;;) { - long sleeptime; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1654,20 +1654,35 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndKeepaliveIfNecessary(); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); + if (!RecoveryInProgress()) + { + long sleeptime; + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - wakeEvents = WL_SOCKET_READABLE; + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime * 10, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + { + ConditionVariablePrepareToSleep(replayedCV); + ConditionVariableTimedSleep(replayedCV, 1000, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..b65c2cf1f0 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *getReplayedCV(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v45-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v45-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From d184a98f9c9aeefdcfbeeebab3aba4bf0cee9815 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:41:38 +0000 Subject: [PATCH v45 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 867675d5a1..1abe747cb5 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v45-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v45-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 700c26ef8ee1c5caa40c6e0ce14796f7662c6f97 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:40:56 +0000 Subject: [PATCH v45 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d478724b9d..d64fb4cc84 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8891,6 +8891,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9060,6 +9061,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9177,6 +9179,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..867675d5a1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v45-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v45-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 7ab0a6ec046f0c181c8f3b96cb18ae938f573ba9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Mon, 30 Jan 2023 15:30:15 +0000 Subject: [PATCH v45 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..4462022904 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..aad14a401d 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e6024a980b..d478724b9d 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6872,6 +6872,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8442,7 +8443,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8454,6 +8455,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..70a0c2418a 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-01-31 11:50 Drouvot, Bertrand <[email protected]> parent: Andres Freund <[email protected]> 2 siblings, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-01-31 11:50 UTC (permalink / raw) To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/6/23 4:40 AM, Andres Freund wrote: > Hi, > On 2023-01-05 16:15:39 -0500, Robert Haas wrote: >> On Tue, Jan 3, 2023 at 2:42 AM Drouvot, Bertrand >> <[email protected]> wrote: > 0006: > >> diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c >> index bc3c3eb3e7..98c96eb864 100644 >> --- a/src/backend/access/transam/xlogrecovery.c >> +++ b/src/backend/access/transam/xlogrecovery.c >> @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData >> RecoveryPauseState recoveryPauseState; >> ConditionVariable recoveryNotPausedCV; >> >> + /* Replay state (see getReplayedCV() for more explanation) */ >> + ConditionVariable replayedCV; >> + >> slock_t info_lck; /* locks shared variables shown above */ >> } XLogRecoveryCtlData; >> > > getReplayedCV() doesn't seem to fit into any of the naming scheems in use for > xlogrecovery.h. Changed to check_for_replay() in V46 attached. >> - * Sleep until something happens or we time out. Also wait for the >> - * socket becoming writable, if there's still pending output. >> + * When not in recovery, sleep until something happens or we time out. >> + * Also wait for the socket becoming writable, if there's still pending output. > > Hm. Is there a problem with not handling the becoming-writable case in the > in-recovery case? > > Yes, when not in recovery we'd wait for the timeout to occur in ConditionVariableTimedSleep() (as the CV is broadcasted only in ApplyWalRecord()). >> + else >> + /* >> + * We are in the logical decoding on standby case. >> + * We are waiting for the startup process to replay wal record(s) using >> + * a timeout in case we are requested to stop. >> + */ >> + { > > I don't think pgindent will like that formatting.... Oops, fixed. > > >> + ConditionVariablePrepareToSleep(replayedCV); >> + ConditionVariableTimedSleep(replayedCV, 1000, >> + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); >> + } > > I think this is racy, see ConditionVariablePrepareToSleep()'s comment: > > * Caution: "before entering the loop" means you *must* test the exit > * condition between calling ConditionVariablePrepareToSleep and calling > * ConditionVariableSleep. If that is inconvenient, omit calling > * ConditionVariablePrepareToSleep. > > Basically, the ConditionVariablePrepareToSleep() should be before the loop > body. > I missed it, thanks! Moved it before the loop body. > > I don't think the fixed timeout here makes sense. For one, we need to wake up > based on WalSndComputeSleeptime(), otherwise we're ignoring wal_sender_timeout > (which can be quite small). Good point. Making use of WalSndComputeSleeptime() instead in V46. > It's also just way too frequent - we're trying to > avoid constantly waking up unnecessarily. > > > Perhaps we could deal with the pq_is_send_pending() issue by having a version > of ConditionVariableTimedSleep() that accepts a WaitEventSet? > What issue do you see? The one that I see with V46 (keeping the in/not recovery branches) is that one may need to wait for wal_sender_timeout to see changes that occurred right after the promotion. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From 796efdfb0c4d367a7ec4d433a9111c09cad769f5 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:11:40 +0000 Subject: [PATCH v46 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From f6dbf79a137c618c3dfa0bdd568f66866a88fcfa Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:03:50 +0000 Subject: [PATCH v46 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 665 ++++++++++++++++++ 3 files changed, 705 insertions(+) 4.9% src/test/perl/PostgreSQL/Test/ 94.8% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..690170daaa --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,665 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 62; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + { + # make sure activeslot is in use + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); +$handle = make_slot_active(1); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); -- 2.34.1 From ddb877d6a780065b25824008b941ba6daa5193ee Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:02:35 +0000 Subject: [PATCH v46 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2a5352f879..30dddda5f8 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4923,3 +4932,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..3fc7b42d15 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 4fa4e5ef5099e2771ca2e7e1813a1914ce751c74 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:01:31 +0000 Subject: [PATCH v46 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 867675d5a1..1abe747cb5 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 5cfbe1368d1baf13c2884defeeadb61df215d979 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:00:39 +0000 Subject: [PATCH v46 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d478724b9d..d64fb4cc84 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8891,6 +8891,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9060,6 +9061,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9177,6 +9179,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..867675d5a1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From a94d82b17eb7a39ef45c6d617bfe9adf104bd60a Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 09:59:08 +0000 Subject: [PATCH v46 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..4462022904 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..aad14a401d 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e6024a980b..d478724b9d 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6872,6 +6872,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8442,7 +8443,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8454,6 +8455,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..70a0c2418a 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v46-0006-Doc-changes-describing-details-about-logical-dec.patch (2.1K, ../../[email protected]/2-v46-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From 796efdfb0c4d367a7ec4d433a9111c09cad769f5 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:11:40 +0000 Subject: [PATCH v46 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v46-0005-New-TAP-test-for-logical-decoding-on-standby.patch (26.6K, ../../[email protected]/3-v46-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From f6dbf79a137c618c3dfa0bdd568f66866a88fcfa Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:03:50 +0000 Subject: [PATCH v46 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 665 ++++++++++++++++++ 3 files changed, 705 insertions(+) 4.9% src/test/perl/PostgreSQL/Test/ 94.8% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..690170daaa --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,665 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 62; + +my ($stdin, $stdout, $stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + $node_standby->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node_standby->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my $wait = shift; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node_standby->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', \$stdout, '2>', \$stderr); + + if ($wait) + { + # make sure activeslot is in use + $node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots(); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active(1); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active(1); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active(0); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active(0); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots(); + +$handle = make_slot_active(1); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots(); +$handle = make_slot_active(1); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); -- 2.34.1 [text/plain] v46-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.7K, ../../[email protected]/4-v46-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From ddb877d6a780065b25824008b941ba6daa5193ee Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:02:35 +0000 Subject: [PATCH v46 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2a5352f879..30dddda5f8 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -467,6 +470,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1916,6 +1920,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4923,3 +4932,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..3fc7b42d15 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v46-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v46-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From 4fa4e5ef5099e2771ca2e7e1813a1914ce751c74 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:01:31 +0000 Subject: [PATCH v46 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 867675d5a1..1abe747cb5 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4465,6 +4465,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v46-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v46-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 5cfbe1368d1baf13c2884defeeadb61df215d979 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 10:00:39 +0000 Subject: [PATCH v46 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d478724b9d..d64fb4cc84 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8891,6 +8891,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -9060,6 +9061,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -9177,6 +9179,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fb4c860bde..867675d5a1 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7966,6 +7970,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..0041896620 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v46-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v46-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From a94d82b17eb7a39ef45c6d617bfe9adf104bd60a Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 31 Jan 2023 09:59:08 +0000 Subject: [PATCH v46 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index d21a308d41..4462022904 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 56451fede1..aad14a401d 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e6024a980b..d478724b9d 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6872,6 +6872,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8442,7 +8443,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8454,6 +8455,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 8003583c0a..70a0c2418a 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index c86e690980..321fc0d31b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-02-07 10:36 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-02-07 10:36 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; +Cc: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/11/23 4:23 PM, Drouvot, Bertrand wrote: > Hi, > > On 1/11/23 8:32 AM, Bharath Rupireddy wrote: >> On Tue, Jan 10, 2023 at 2:03 PM Drouvot, Bertrand >> <[email protected]> wrote: >>> >>> Please find attached, V37 taking care of: >> >> Thanks. I started to digest the design specified in the commit message >> and these patches. > > Thanks for looking at it! > >> Here are some quick comments: >> >> 1. Does logical decoding on standby work without any issues if the >> standby is set for cascading replication? >> > > Without "any issues" is hard to guarantee ;-) But according to my tests: > > Primary -> Standby1 with or without logical replication slot -> Standby2 with or without logical replication slot > > works as expected (and also with cascading promotion). > We can add some TAP tests in 0004 though. Cascading standby tests have been added in V48 attached. It does test that: - a sql logical decoding session on the cascading standby get expected output before/after the standby promotion - a pg_recvlogical logical decoding session on the cascading standby (started before the standby promotion) get expected output before/after the standby promotion Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From d01ad6fbc8a23c3ddcc10fc3e11feb65e0fcdc5a Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:48:52 +0000 Subject: [PATCH v48 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 305eb583edf89057b6cd39539e1920da10a0269e Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v48 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 710 ++++++++++++++++++ 3 files changed, 750 insertions(+) 4.4% src/test/perl/PostgreSQL/Test/ 95.3% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..cf1277bd1b --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_standby->lsn('flush')); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 From fa17810dee089ccfb8c058e25b7804a47f01f67f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v48 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..3fc7b42d15 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From c4e2303b2b56c9328a6296a33911005537cfcd77 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v48 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 8dfd8911256b1841190205f6fff2052de7f1fe3b Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v48 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 5d439f2710..b2a75b6d72 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 2c753f732ef92d6d1780cd4778dba8d09e16a5b0 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v48 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v48-0006-Doc-changes-describing-details-about-logical-dec.patch (2.1K, ../../[email protected]/2-v48-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From d01ad6fbc8a23c3ddcc10fc3e11feb65e0fcdc5a Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:48:52 +0000 Subject: [PATCH v48 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..2e8bee033f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,27 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v48-0005-New-TAP-test-for-logical-decoding-on-standby.patch (29.3K, ../../[email protected]/3-v48-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 305eb583edf89057b6cd39539e1920da10a0269e Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v48 5/6] New TAP test for logical decoding on standby. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/test/perl/PostgreSQL/Test/Cluster.pm | 39 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 710 ++++++++++++++++++ 3 files changed, 750 insertions(+) 4.4% src/test/perl/PostgreSQL/Test/ 95.3% src/test/recovery/t/ diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..fd81ddcf39 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,45 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + # Note: Write a C helper function to call LogStandbySnapshot() instead + # of asking for a checkpoint. + $primary->safe_psql('postgres', 'CHECKPOINT'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..cf1277bd1b --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_standby->lsn('flush')); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 [text/plain] v48-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.7K, ../../[email protected]/4-v48-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From fa17810dee089ccfb8c058e25b7804a47f01f67f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v48 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..3fc7b42d15 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v48-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v48-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From c4e2303b2b56c9328a6296a33911005537cfcd77 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v48 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v48-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v48-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 8dfd8911256b1841190205f6fff2052de7f1fe3b Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v48 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 5d439f2710..b2a75b6d72 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v48-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v48-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 2c753f732ef92d6d1780cd4778dba8d09e16a5b0 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v48 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-02-07 15:29 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 2 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-02-07 15:29 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 1/19/23 10:43 AM, Drouvot, Bertrand wrote: > Hi, > > On 1/19/23 3:46 AM, Andres Freund wrote: >> Hi, >> >> On 2023-01-18 11:24:19 +0100, Drouvot, Bertrand wrote: >>> On 1/6/23 4:40 AM, Andres Freund wrote: >>>> Hm, that's quite expensive. Perhaps worth adding a C helper that can do that >>>> for us instead? This will likely also be needed in real applications after all. >>>> >>> >>> Not sure I got it. What the C helper would be supposed to do? >> >> Call LogStandbySnapshot(). >> > > Got it, I like the idea, will do. > 0005 in V49 attached is introducing a new pg_log_standby_snapshot() function and the TAP test is making use of it. Documentation about this new function is also added in the "Snapshot Synchronization Functions" section. I'm not sure that's the best place for it but did not find a better place yet. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From 78a09b5097d44f457f5d19a18226ac32f03f5797 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v49 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 85494784ca3e37cedcdc714c1f216b229a9e112f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v49 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.0% src/backend/ 3.9% src/test/perl/PostgreSQL/Test/ 89.9% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index e09e289a43..59334dd422 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26534,6 +26534,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c8e11ab710..48d7be075b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6384,6 +6384,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..247265d328 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..cf1277bd1b --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_standby->lsn('flush')); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 From f86b422471169795995fc9dba694eef719ede673 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v49 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..3fc7b42d15 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From b4f03a2fde8bc091427999ea0ea63335d905243f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v49 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From b1b8b19ff0aefb60227ee2a38ff407c317ed7a3f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v49 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 5d439f2710..b2a75b6d72 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 05d208a94131552851917e021062af1334cf15e4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v49 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v49-0006-Doc-changes-describing-details-about-logical-dec.patch (2.2K, ../../[email protected]/2-v49-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From 78a09b5097d44f457f5d19a18226ac32f03f5797 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v49 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v49-0005-New-TAP-test-for-logical-decoding-on-standby.patch (33.4K, ../../[email protected]/3-v49-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 85494784ca3e37cedcdc714c1f216b229a9e112f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v49 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.0% src/backend/ 3.9% src/test/perl/PostgreSQL/Test/ 89.9% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index e09e289a43..59334dd422 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26534,6 +26534,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c8e11ab710..48d7be075b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6384,6 +6384,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 04921ca3a3..247265d328 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3037,6 +3037,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..cf1277bd1b --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_catchup($node_standby, 'replay', $node_primary->lsn('flush')); +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_primary->lsn('flush')); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_catchup($node_cascading_standby, 'replay', $node_standby->lsn('flush')); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 [text/plain] v49-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.7K, ../../[email protected]/4-v49-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From f86b422471169795995fc9dba694eef719ede673 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v49 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1e91cbc564..3fc7b42d15 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v49-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v49-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From b4f03a2fde8bc091427999ea0ea63335d905243f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v49 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 1a58dd7649..91acc0c155 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -119,23 +119,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -331,6 +330,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 8885cdeebc..1e91cbc564 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v49-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v49-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From b1b8b19ff0aefb60227ee2a38ff407c317ed7a3f Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v49 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 1756f1a4b6..e25f71a776 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4365,6 +4365,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8608e3fa5b..a272bd4a88 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 4ed3747e3f..8885cdeebc 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 5d439f2710..b2a75b6d72 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 6737493402..afd62d3cc0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c0f2a8a77c..c8e11ab710 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10946,9 +10951,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 5e3326a3b9..872eb35757 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -291,6 +291,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e7a2f5856a..11ea206337 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v49-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v49-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 05d208a94131552851917e021062af1334cf15e4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v49 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index af9785038d..0cfe02aa4a 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-02-13 15:27 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-02-13 15:27 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2/7/23 4:29 PM, Drouvot, Bertrand wrote: > Hi, > > On 1/19/23 10:43 AM, Drouvot, Bertrand wrote: >> Hi, >> >> On 1/19/23 3:46 AM, Andres Freund wrote: >>> Hi, >>> >>> On 2023-01-18 11:24:19 +0100, Drouvot, Bertrand wrote: >>>> On 1/6/23 4:40 AM, Andres Freund wrote: >>>>> Hm, that's quite expensive. Perhaps worth adding a C helper that can do that >>>>> for us instead? This will likely also be needed in real applications after all. >>>>> >>>> >>>> Not sure I got it. What the C helper would be supposed to do? >>> >>> Call LogStandbySnapshot(). >>> >> >> Got it, I like the idea, will do. >> > > 0005 in V49 attached is introducing a new pg_log_standby_snapshot() function > and the TAP test is making use of it. > > Documentation about this new function is also added in the "Snapshot Synchronization Functions" > section. I'm not sure that's the best place for it but did not find a better place yet. > Attaching V50, tiny update in the TAP test (aka 0005) to make use of the wait_for_replay_catchup() wrapper just added in a1acdacada. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From 81d85eb3b4ae84cf7516410359b79a65042ae0a9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v50 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 0fbca11968af00ee099d62c993f46104574e2db9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v50 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.1% src/backend/ 4.0% src/test/perl/PostgreSQL/Test/ 89.7% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index e09e289a43..59334dd422 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26534,6 +26534,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index ca88f48079..d1a9082625 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6393,6 +6393,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 3e2a27fb71..da58257f4f 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3060,6 +3060,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..8c45180c35 --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_replay_catchup($node_standby); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_replay_catchup($node_standby); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_replay_catchup($node_standby); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_replay_catchup($node_standby); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_replay_catchup($node_cascading_standby); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 From 56520fa522b9c470a341091fee5db7ec34899ed8 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v50 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..5034194e1b 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 1a1cfab5e2e350b37b34ddde52522869721044db Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v50 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index c3ec97a0a6..743d12ba14 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -124,23 +124,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -342,6 +341,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index c2523c5caf..3042e5bd64 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 0d07e9cbd16b2ddcaf18fd654f5ea5ea4fd7da93 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v50 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index dca50707ad..d5ec656c10 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4658,6 +4658,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 34ca0e739f..20c70be5a2 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 75e8363e24..c2523c5caf 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 5d439f2710..b2a75b6d72 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 9d707c3521..048af5bf40 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 66b73c3900..ca88f48079 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10955,9 +10960,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index db9675884f..c1095e374c 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -341,6 +341,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 174b725fff..56e48b50f0 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 66a5bb3b9bf95f250eceaed7f98a30794ea4fb5d Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v50 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 67f994cb3e..52845497cc 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v50-0006-Doc-changes-describing-details-about-logical-dec.patch (2.2K, ../../[email protected]/2-v50-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From 81d85eb3b4ae84cf7516410359b79a65042ae0a9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v50 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v50-0005-New-TAP-test-for-logical-decoding-on-standby.patch (32.9K, ../../[email protected]/3-v50-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 0fbca11968af00ee099d62c993f46104574e2db9 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v50 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/034_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.1% src/backend/ 4.0% src/test/perl/PostgreSQL/Test/ 89.7% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index e09e289a43..59334dd422 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26534,6 +26534,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index ca88f48079..d1a9082625 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6393,6 +6393,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 3e2a27fb71..da58257f4f 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3060,6 +3060,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 209118a639..eca90c5c8c 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -39,6 +39,7 @@ tests += { 't/031_recovery_conflict.pl', 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', + 't/034_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/034_standby_logical_decoding.pl b/src/test/recovery/t/034_standby_logical_decoding.pl new file mode 100644 index 0000000000..8c45180c35 --- /dev/null +++ b/src/test/recovery/t/034_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_replay_catchup($node_standby); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_replay_catchup($node_standby); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_replay_catchup($node_standby); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_replay_catchup($node_standby); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_replay_catchup($node_cascading_standby); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 [text/plain] v50-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.7K, ../../[email protected]/4-v50-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From 56520fa522b9c470a341091fee5db7ec34899ed8 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v50 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..5034194e1b 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index 6e4599278c..38c747b786 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -463,6 +463,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 6cacd6edaf..04a37feee4 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -130,6 +130,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v50-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v50-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From 1a1cfab5e2e350b37b34ddde52522869721044db Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v50 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index a53e23c679..6b66a971ba 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index c3ec97a0a6..743d12ba14 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -124,23 +124,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -342,6 +341,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index c2523c5caf..3042e5bd64 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v50-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v50-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 0d07e9cbd16b2ddcaf18fd654f5ea5ea4fd7da93 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v50 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index dca50707ad..d5ec656c10 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4658,6 +4658,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 34ca0e739f..20c70be5a2 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 75e8363e24..c2523c5caf 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 94cc860f5f..ec817381a1 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -475,6 +476,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -500,6 +502,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -508,6 +513,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -526,7 +532,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1487,6 +1495,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 5d439f2710..b2a75b6d72 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2481,6 +2481,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3050,6 +3053,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 9d707c3521..048af5bf40 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 66b73c3900..ca88f48079 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10955,9 +10960,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index db9675884f..c1095e374c 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -341,6 +341,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 174b725fff..56e48b50f0 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v50-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v50-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 66a5bb3b9bf95f250eceaed7f98a30794ea4fb5d Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v50 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 1cc88da032..4e8a85fb5d 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 67b7b1710c..8c58fdb8d1 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 41b16cb89b..48d1d6b506 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3352,6 +3352,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index a2f0f39213..7e9e47ce67 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; int ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(int)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 67f994cb3e..52845497cc 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-02-27 08:40 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-02-27 08:40 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2/13/23 4:27 PM, Drouvot, Bertrand wrote: > Hi, > > On 2/7/23 4:29 PM, Drouvot, Bertrand wrote: >> Hi, >> >> On 1/19/23 10:43 AM, Drouvot, Bertrand wrote: >>> Hi, >>> >>> On 1/19/23 3:46 AM, Andres Freund wrote: >>>> Hi, >>>> >>>> On 2023-01-18 11:24:19 +0100, Drouvot, Bertrand wrote: >>>>> On 1/6/23 4:40 AM, Andres Freund wrote: >>>>>> Hm, that's quite expensive. Perhaps worth adding a C helper that can do that >>>>>> for us instead? This will likely also be needed in real applications after all. >>>>>> >>>>> >>>>> Not sure I got it. What the C helper would be supposed to do? >>>> >>>> Call LogStandbySnapshot(). >>>> >>> >>> Got it, I like the idea, will do. >>> >> >> 0005 in V49 attached is introducing a new pg_log_standby_snapshot() function >> and the TAP test is making use of it. >> >> Documentation about this new function is also added in the "Snapshot Synchronization Functions" >> section. I'm not sure that's the best place for it but did not find a better place yet. >> > > Attaching V50, tiny update in the TAP test (aka 0005) to make use of the wait_for_replay_catchup() > wrapper just added in a1acdacada. > Please find attached V51 tiny rebase due to a6cd1fc692 (for 0001) and 8a8661828a (for 0005). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From cb1d824602f67526029eaecce0a5e962f0188ce6 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v51 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 330fd5faef129399dfc5f6a2cbac2ef9a6799baa Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v51 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/035_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.1% src/backend/ 4.0% src/test/perl/PostgreSQL/Test/ 89.7% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 0cbdf63632..4006100f42 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26548,6 +26548,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 8329d05d68..c49162f662 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6393,6 +6393,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 3e2a27fb71..da58257f4f 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3060,6 +3060,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 59465b97f3..e834ad5e0d 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', 't/034_create_database.pl', + 't/035_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/035_standby_logical_decoding.pl b/src/test/recovery/t/035_standby_logical_decoding.pl new file mode 100644 index 0000000000..8c45180c35 --- /dev/null +++ b/src/test/recovery/t/035_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_replay_catchup($node_standby); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_replay_catchup($node_standby); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_replay_catchup($node_standby); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_replay_catchup($node_standby); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_replay_catchup($node_cascading_standby); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 From 4b06d11910884fbbd26a53feb5f727e4c5fc167d Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v51 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..5034194e1b 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..e1b80e6202 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 9981fbb75d8a90a00af843c56a7ba07c99e7e573 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v51 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 8fe7bb65f1..8457eec4c4 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index c3ec97a0a6..743d12ba14 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -124,23 +124,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -342,6 +341,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index c2523c5caf..3042e5bd64 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From e1c324b86ada1ce62b17d9b6e95c76e0e7ad0945 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v51 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b0b997f092..33f525a689 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4663,6 +4663,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 34ca0e739f..20c70be5a2 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 75e8363e24..c2523c5caf 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 9a73ae67d0..db5c3333cc 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -466,6 +467,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -491,6 +493,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -499,6 +504,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -517,7 +523,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1478,6 +1486,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..b5f9aa285c 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2488,6 +2488,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3057,6 +3060,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 9d707c3521..048af5bf40 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index e2a7642a2b..8329d05d68 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10955,9 +10960,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index db9675884f..c1095e374c 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -341,6 +341,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e953d1f515..1b6600884e 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 9c22d89c0ec2033b35465f0e9586af89c9ca5969 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v51 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.1% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 3f7b541e9d..a213407fee 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 02b9601bec..1207a49689 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 7777e7ec77..98a712f4ec 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3365,6 +3365,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index 9894ab9afe..6c5535fe73 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; uint16 ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(uint16)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 67f994cb3e..52845497cc 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v51-0006-Doc-changes-describing-details-about-logical-dec.patch (2.2K, ../../[email protected]/2-v51-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From cb1d824602f67526029eaecce0a5e962f0188ce6 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v51 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v51-0005-New-TAP-test-for-logical-decoding-on-standby.patch (32.9K, ../../[email protected]/3-v51-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 330fd5faef129399dfc5f6a2cbac2ef9a6799baa Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v51 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/035_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.1% src/backend/ 4.0% src/test/perl/PostgreSQL/Test/ 89.7% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 0cbdf63632..4006100f42 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26548,6 +26548,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 8329d05d68..c49162f662 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6393,6 +6393,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 3e2a27fb71..da58257f4f 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3060,6 +3060,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 59465b97f3..e834ad5e0d 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', 't/034_create_database.pl', + 't/035_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/035_standby_logical_decoding.pl b/src/test/recovery/t/035_standby_logical_decoding.pl new file mode 100644 index 0000000000..8c45180c35 --- /dev/null +++ b/src/test/recovery/t/035_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_replay_catchup($node_standby); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_replay_catchup($node_standby); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_replay_catchup($node_standby); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_replay_catchup($node_standby); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_replay_catchup($node_cascading_standby); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 [text/plain] v51-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.7K, ../../[email protected]/4-v51-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From 4b06d11910884fbbd26a53feb5f727e4c5fc167d Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v51 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..5034194e1b 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..e1b80e6202 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v51-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v51-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From 9981fbb75d8a90a00af843c56a7ba07c99e7e573 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v51 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 54d344a59c..5864c5e304 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4464,6 +4464,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 8fe7bb65f1..8457eec4c4 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index c3ec97a0a6..743d12ba14 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -124,23 +124,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -342,6 +341,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index c2523c5caf..3042e5bd64 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v51-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v51-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From e1c324b86ada1ce62b17d9b6e95c76e0e7ad0945 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v51 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b0b997f092..33f525a689 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4663,6 +4663,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 04e9bc5eb2..6524784583 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8686,6 +8686,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8855,6 +8856,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8972,6 +8974,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9f0f6db8d..54d344a59c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6444,6 +6444,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6804,7 +6805,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7083,6 +7085,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7248,7 +7251,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7961,6 +7965,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 34ca0e739f..20c70be5a2 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 75e8363e24..c2523c5caf 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 9a73ae67d0..db5c3333cc 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -466,6 +467,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -491,6 +493,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -499,6 +504,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -517,7 +523,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1478,6 +1486,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..b5f9aa285c 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2488,6 +2488,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3057,6 +3060,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 9d707c3521..048af5bf40 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index e2a7642a2b..8329d05d68 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10955,9 +10960,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index db9675884f..c1095e374c 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -341,6 +341,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e953d1f515..1b6600884e 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v51-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v51-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 9c22d89c0ec2033b35465f0e9586af89c9ca5969 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v51 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 13 ++- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 203 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.1% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7eb79cee58..04e9bc5eb2 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6667,6 +6667,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8237,7 +8238,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8249,6 +8250,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 3f7b541e9d..a213407fee 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 02b9601bec..1207a49689 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 7777e7ec77..98a712f4ec 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3365,6 +3365,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 09f9b0f8c6..2eea866f06 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,13 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* - * In payload of blk 0 : todelete OffsetNumbers - */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -100,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index 9894ab9afe..6c5535fe73 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; uint16 ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(uint16)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 8cb0d8da19..223db4b199 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,12 +346,14 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* FREEZE PLANS FOLLOW */ /* OFFSET NUMBER ARRAY FOLLOWS */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -408,7 +412,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index edd1333d9b..1e45d58845 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,13 +237,15 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* DELETED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TARGET OFFSET NUMBERS FOLLOW */ /* UPDATED TUPLES METADATA (xl_btree_update) ARRAY FOLLOWS */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 67f994cb3e..52845497cc 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-01 00:48 Jeff Davis <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Jeff Davis @ 2023-03-01 00:48 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Mon, 2023-02-27 at 09:40 +0100, Drouvot, Bertrand wrote: > Please find attached V51 tiny rebase due to a6cd1fc692 (for 0001) and > 8a8661828a (for 0005). [ Jumping into this thread late, so I apologize if these comments have already been covered. ] Regarding v51-0004: * Why is the CV sleep not being canceled? * Comments on WalSndWaitForWal need to be updated to explain the difference between the flush (primary) and the replay (standby) cases. Overall, it seems like what you really want for the sleep/wakeup logic in WalSndWaitForLSN is something like this: condVar = RecoveryInProgress() ? replayCV : flushCV; waitEvent = RecoveryInProgress() ? WAIT_EVENT_WAL_SENDER_WAIT_REPLAY : WAIT_EVENT_WAL_SENDER_WAIT_FLUSH; ConditionVariablePrepareToSleep(condVar); for(;;) { ... sleeptime = WalSndComputeSleepTime(GetCurrentTimestamp()); socketEvents = WL_SOCKET_READABLE; if (pq_is_send_pending()) socketEvents = WL_SOCKET_WRITABLE; ConditionVariableTimedSleepOrEvents( condVar, sleeptime, socketEvents, waitEvent); } ConditionVariableCancelSleep(); But the problem is that ConditionVariableTimedSleepOrEvents() doesn't exist, and I think that's what Andres was suggesting here[1]. WalSndWait() only waits for a timeout or a socket event, but not a CV; ConditionVariableTimedSleep() only waits for a timeout or a CV, but not a socket event. I'm also missing how WalSndWait() works currently. It calls ModifyWaitEvent() with NULL for the latch, so how does WalSndWakeup() wake it up? Assuming I'm wrong, and WalSndWait() does use the latch, then I guess it could be extended by having two different latches in the WalSnd structure, and waking them up separately and waiting on the right one. Not sure if that's a good idea though. [1] https://www.postgresql.org/message-id/[email protected] -- Jeff Davis PostgreSQL Contributor Team - AWS ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-01 10:51 Drouvot, Bertrand <[email protected]> parent: Jeff Davis <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-01 10:51 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/1/23 1:48 AM, Jeff Davis wrote: > On Mon, 2023-02-27 at 09:40 +0100, Drouvot, Bertrand wrote: >> Please find attached V51 tiny rebase due to a6cd1fc692 (for 0001) and >> 8a8661828a (for 0005). > > [ Jumping into this thread late, so I apologize if these comments have > already been covered. ] > Thanks for looking at it! > Regarding v51-0004: > > * Why is the CV sleep not being canceled? I think that's an oversight, I'll look at it. > * Comments on WalSndWaitForWal need to be updated to explain the > difference between the flush (primary) and the replay (standby) cases. > Yeah, will do. > Overall, it seems like what you really want for the sleep/wakeup logic > in WalSndWaitForLSN Typo for WalSndWaitForWal()? > is something like this: > > condVar = RecoveryInProgress() ? replayCV : flushCV; > waitEvent = RecoveryInProgress() ? > WAIT_EVENT_WAL_SENDER_WAIT_REPLAY : > WAIT_EVENT_WAL_SENDER_WAIT_FLUSH; > > ConditionVariablePrepareToSleep(condVar); > for(;;) > { > ... > sleeptime = WalSndComputeSleepTime(GetCurrentTimestamp()); > socketEvents = WL_SOCKET_READABLE; > if (pq_is_send_pending()) > socketEvents = WL_SOCKET_WRITABLE; > ConditionVariableTimedSleepOrEvents( > condVar, sleeptime, socketEvents, waitEvent); > } > ConditionVariableCancelSleep(); > > > But the problem is that ConditionVariableTimedSleepOrEvents() doesn't > exist, and I think that's what Andres was suggesting here[1]. > WalSndWait() only waits for a timeout or a socket event, but not a CV; > ConditionVariableTimedSleep() only waits for a timeout or a CV, but not > a socket event. > > I'm also missing how WalSndWait() works currently. It calls > ModifyWaitEvent() with NULL for the latch, so how does WalSndWakeup() > wake it up? I think it works because the latch is already assigned to the FeBeWaitSet in pq_init()->AddWaitEventToSet() (for latch_pos). > > Assuming I'm wrong, and WalSndWait() does use the latch, then I guess > it could be extended by having two different latches in the WalSnd > structure, and waking them up separately and waiting on the right one. I'm not sure this is needed in this particular case, because: Why not "simply" call ConditionVariablePrepareToSleep() without any call to ConditionVariableTimedSleep() later? In that case the walsender will be put in the wait queue (thanks to ConditionVariablePrepareToSleep()) and will be waked up by the event on the socket, the timeout or the CV broadcast (since IIUC they all rely on the same latch). So, something like: condVar = RecoveryInProgress() ? replayCV : flushCV; ConditionVariablePrepareToSleep(condVar); for(;;) { ... sleeptime = WalSndComputeSleepTime(GetCurrentTimestamp()); socketEvents = WL_SOCKET_READABLE; if (pq_is_send_pending()) socketEvents = WL_SOCKET_WRITABLE; WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); <-- Note: the code within the loop does not change at all } ConditionVariableCancelSleep(); If the walsender is waked up by the CV broadcast, then it means the flush/replay occurred and then we should exit the loop right after due to: " /* check whether we're done */ if (loc <= RecentFlushPtr) break; " meaning that in this particular case there is only one wake up due to the CV broadcast before exiting the loop. That looks weird to use ConditionVariablePrepareToSleep() without actually using ConditionVariableTimedSleep() but it looks to me that it would achieve the same goal: having the walsender being waked up by the event on the socket, the timeout or the CV broadcast. In that case we would be missing the WAIT_EVENT_WAL_SENDER_WAIT_REPLAY and/or the WAIT_EVENT_WAL_SENDER_WAIT_FLUSH wait events thought (and we'd just provide the WAIT_EVENT_WAL_SENDER_WAIT_WAL one) but I'm not sure that's a big issue. What do you think? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-02 00:40 Jeff Davis <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Jeff Davis @ 2023-03-02 00:40 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Wed, 2023-03-01 at 11:51 +0100, Drouvot, Bertrand wrote: > > Why not "simply" call ConditionVariablePrepareToSleep() without any > call to ConditionVariableTimedSleep() later? ConditionVariableSleep() re-inserts itself into the queue if it was previously removed. Without that, a single wakeup could remove it from the wait queue, and the effects of ConditionVariablePrepareToSleep() would be lost. > In that case the walsender will be put in the wait queue (thanks to > ConditionVariablePrepareToSleep()) > and will be waked up by the event on the socket, the timeout or the > CV broadcast I believe it will only be awakened once, and if it enters WalSndWait() again, future ConditionVariableBroadcast/Signal() calls won't wake it up any more. > (since IIUC they all rely on the same latch). Relying on that fact seems like too much action-at-a-distance to me. If we change the implementation of condition variables, then it would stop working. Also, since they are using the same latch, that means we are still waking up too frequently, right? We haven't really solved the problem. > That looks weird to use ConditionVariablePrepareToSleep() without > actually using ConditionVariableTimedSleep() > but it looks to me that it would achieve the same goal: having the > walsender being waked up > by the event on the socket, the timeout or the CV broadcast. I don't think it actually works, because something needs to keep re- inserting it into the queue after it gets removed. You could maybe hack it to put ConditionVariablePrepareToSleep() *in* the loop, and never sleep. But that just seems like too much of a hack, and I didn't really look at the details to see if that would actually work. To use condition variables properly, I think we'd need an API like ConditionVariableEventsSleep(), which takes a WaitEventSet and a timeout. I think this is what Andres was suggesting and seems like a good idea. I looked into it and I don't think it's too hard to implement -- we just need to WaitEventSetWait instead of WaitLatch. There are a few details to sort out, like how to enable callers to easily create the right WaitEventSet (it obviously needs to include MyLatch, for instance) and update it with the right socket events. -- Jeff Davis PostgreSQL Contributor Team - AWS ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-02 09:20 Drouvot, Bertrand <[email protected]> parent: Jeff Davis <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-02 09:20 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/2/23 1:40 AM, Jeff Davis wrote: > On Wed, 2023-03-01 at 11:51 +0100, Drouvot, Bertrand wrote: > >> >> Why not "simply" call ConditionVariablePrepareToSleep() without any >> call to ConditionVariableTimedSleep() later? > > ConditionVariableSleep() re-inserts itself into the queue if it was > previously removed. Without that, a single wakeup could remove it from > the wait queue, and the effects of ConditionVariablePrepareToSleep() > would be lost. Right, but in our case, right after the wakeup (the one due to the CV broadcast, aka the one that will remove it from the wait queue) we'll exit the loop due to: " /* check whether we're done */ if (loc <= RecentFlushPtr) break; " as the CV broadcast means that a flush/replay occurred. So I don't see any issue in this particular case (as we are removed from the queue but we'll not have to wait anymore). > >> In that case the walsender will be put in the wait queue (thanks to >> ConditionVariablePrepareToSleep()) >> and will be waked up by the event on the socket, the timeout or the >> CV broadcast > > I believe it will only be awakened once, and if it enters WalSndWait() > again, future ConditionVariableBroadcast/Signal() calls won't wake it > up any more. I don't think that's right and that's not what my testing shows (please find attached 0004-CV-POC.txt, a .txt file to not break the CF bot), as: - If it is awakened due to the CV broadcast, then we'll right after exit the loop (see above) - If it is awakened due to the timeout or the socket event then we're still in the CV wait queue (as nothing removed it from the CV wait queue). > >> (since IIUC they all rely on the same latch). > > Relying on that fact seems like too much action-at-a-distance to me > If > we change the implementation of condition variables, then it would stop > working. > I'm not sure about this one. I mean it would depend what the implementation changes are. Also the related TAP test (0005) would probably fail or start taking a long time due to the corner case we are trying to solve here coming back (like it was detected in [1]) > Also, since they are using the same latch, that means we are still > waking up too frequently, right? We haven't really solved the problem. > I don't think so as the first CV broadcast will make us exit the loop. So, ISTM that we'll wake up as we currently do, expect when there is a flush/replay which is what we want, right? >> That looks weird to use ConditionVariablePrepareToSleep() without >> actually using ConditionVariableTimedSleep() >> but it looks to me that it would achieve the same goal: having the >> walsender being waked up >> by the event on the socket, the timeout or the CV broadcast. > > I don't think it actually works, because something needs to keep re- > inserting it into the queue after it gets removed. I think that's not needed as we'd exit the loop right after we are awakened by a CV broadcast. > > To use condition variables properly, I think we'd need an API like > ConditionVariableEventsSleep(), which takes a WaitEventSet and a > timeout. I think this is what Andres was suggesting and seems like a > good idea. I looked into it and I don't think it's too hard to > implement -- we just need to WaitEventSetWait instead of WaitLatch. > There are a few details to sort out, like how to enable callers to > easily create the right WaitEventSet (it obviously needs to include > MyLatch, for instance) and update it with the right socket events. > I agree that's a good idea and that it should/would work too. I just wanted to highlight that in this particular case that might not be necessary to build this new API. [1]: https://www.postgresql.org/message-id/47606911-cf44-5a62-21d5-366d3bc6e445%40enterprisedb.com Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..05350bb535 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,17 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); + + /* + * Prepare the replayedCV to sleep. Note that this is enough to be added + * in the wait queue and then waked up (while in WalSndWait() below) + * by ConditionVariableBroadcast() during the WAL replay. Also Note that + * if awakaned by the CV broadcast we'll exit the loop right after due to + * the loc <= RecentFlushPtr test done in the loop. Indeed, CV brodcast + * would mean that a replay occured. + */ + ConditionVariablePrepareToSleep(replayedCV); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1670,6 +1681,7 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); } + ConditionVariableCancelSleep(); /* reactivate latch so WalSndLoop knows to continue */ SetLatch(MyLatch); return RecentFlushPtr; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. Attachments: [text/plain] 0004-CV-POC.txt (4.4K, ../../[email protected]/2-0004-CV-POC.txt) download | inline diff: diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..05350bb535 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,17 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); + + /* + * Prepare the replayedCV to sleep. Note that this is enough to be added + * in the wait queue and then waked up (while in WalSndWait() below) + * by ConditionVariableBroadcast() during the WAL replay. Also Note that + * if awakaned by the CV broadcast we'll exit the loop right after due to + * the loc <= RecentFlushPtr test done in the loop. Indeed, CV brodcast + * would mean that a replay occured. + */ + ConditionVariablePrepareToSleep(replayedCV); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1670,6 +1681,7 @@ WalSndWaitForWal(XLogRecPtr loc) WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); } + ConditionVariableCancelSleep(); /* reactivate latch so WalSndLoop knows to continue */ SetLatch(MyLatch); return RecentFlushPtr; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-02 19:45 Jeff Davis <[email protected]> parent: Drouvot, Bertrand <[email protected]> 0 siblings, 2 replies; 41+ messages in thread From: Jeff Davis @ 2023-03-02 19:45 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Thu, 2023-03-02 at 10:20 +0100, Drouvot, Bertrand wrote: > Right, but in our case, right after the wakeup (the one due to the CV > broadcast, > aka the one that will remove it from the wait queue) we'll exit the > loop due to: > > " > /* check whether we're done */ > if (loc <= RecentFlushPtr) > break; > " > > as the CV broadcast means that a flush/replay occurred. But does it mean that the flush/replay advanced *enough* to be greater than or equal to loc? > - If it is awakened due to the CV broadcast, then we'll right after > exit the loop (see above) ... > I think that's not needed as we'd exit the loop right after we are > awakened by a CV broadcast. See the comment here: * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or * ConditionVariableBroadcast), so we should return to the caller. But * that doesn't guarantee that the exit condition is met, only that we * ought to check it. You seem to be arguing that in this case, it doesn't matter; that walreceiver knows what walsender is waiting for, and will never wake it up before it's ready. I don't think that's true, and even if it is, it needs explanation. > > I agree that's a good idea and that it should/would work too. I just > wanted to highlight that in this particular > case that might not be necessary to build this new API. In this case it looks easier to add the right API than to be sure about whether it's needed or not. Regards, Jeff Davis ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-03 07:58 Jeff Davis <[email protected]> parent: Jeff Davis <[email protected]> 1 sibling, 2 replies; 41+ messages in thread From: Jeff Davis @ 2023-03-03 07:58 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: > In this case it looks easier to add the right API than to be sure > about > whether it's needed or not. I attached a sketch of one approach. I'm not very confident that it's the right API or even that it works as I intended it, but if others like the approach I can work on it some more. -- Jeff Davis PostgreSQL Contributor Team - AWS Attachments: [text/x-patch] v1-0001-Introduce-ConditionVariableEventSleep.patch (8.8K, ../../[email protected]/2-v1-0001-Introduce-ConditionVariableEventSleep.patch) download | inline diff: From ada1c8f373caa971dc0d8ef2144f1e01100d335c Mon Sep 17 00:00:00 2001 From: Jeff Davis <[email protected]> Date: Wed, 1 Mar 2023 20:02:42 -0800 Subject: [PATCH v1] Introduce ConditionVariableEventSleep(). The new API takes a WaitEventSet which can include socket events. The WaitEventSet must have been created by ConditionVariableWaitSetCreate(), another new function, so that it includes the wait events necessary for a condition variable. --- src/backend/storage/lmgr/condition_variable.c | 102 ++++++++++++++++-- src/backend/storage/lmgr/proc.c | 6 ++ src/backend/utils/init/miscinit.c | 1 + src/include/storage/condition_variable.h | 10 ++ 4 files changed, 111 insertions(+), 8 deletions(-) diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c index 7e2bbf46d9..4dc6109786 100644 --- a/src/backend/storage/lmgr/condition_variable.c +++ b/src/backend/storage/lmgr/condition_variable.c @@ -27,9 +27,29 @@ #include "storage/spin.h" #include "utils/memutils.h" +#define ConditionVariableWaitSetLatchPos 0 + /* Initially, we are not prepared to sleep on any condition variable. */ static ConditionVariable *cv_sleep_target = NULL; +/* Used by ConditionVariableSleep() and ConditionVariableTimedSleep(). */ +static WaitEventSet *ConditionVariableWaitSet = NULL; + +/* + * Initialize the process-local condition variable WaitEventSet. + * + * This must be called once during startup of any process that can wait on + * condition variables, before it issues any ConditionVariableInit() calls. + */ +void +InitializeConditionVariableWaitSet(void) +{ + Assert(ConditionVariableWaitSet == NULL); + + ConditionVariableWaitSet = ConditionVariableWaitSetCreate( + TopMemoryContext, 0); +} + /* * Initialize a condition variable. */ @@ -40,6 +60,51 @@ ConditionVariableInit(ConditionVariable *cv) proclist_init(&cv->wakeup); } +/* + * Create a WaitEventSet for ConditionVariableEventSleep(). This should be + * used when the caller of ConditionVariableEventSleep() would like to wake up + * on either the condition variable signal or a socket event. For example: + * + * ConditionVariableInit(&cv); + * waitset = ConditionVariableWaitSetCreate(mcxt, 1); + * event_pos = AddWaitEventToSet(waitset, 0, sock, NULL, NULL); + * ... + * ConditionVariablePrepareToSleep(&cv); + * while (...condition not met...) + * { + * socket_wait_events = ... + * ModifyWaitEvent(waitset, event_pos, socket_wait_events, NULL); + * ConditionVariableEventSleep(&cv, waitset, ...); + * } + * ConditionVariableCancelSleep(); + * + * The waitset is created with the standard events for a condition variable, + * and room for adding n_socket_events additional socket events. The + * initially-filled event positions should not be modified, but added socket + * events can be modified. The same waitset can be used for multiple condition + * variables as long as the callers of ConditionVariableEventSleep() are + * interested in the same sockets. + */ +WaitEventSet * +ConditionVariableWaitSetCreate(MemoryContext mcxt, int n_socket_events) +{ + int latch_pos PG_USED_FOR_ASSERTS_ONLY; + int n_cv_events = IsUnderPostmaster ? 2 : 1; + int nevents = n_cv_events + n_socket_events; + WaitEventSet *waitset = CreateWaitEventSet(mcxt, nevents); + + latch_pos = AddWaitEventToSet(waitset, WL_LATCH_SET, PGINVALID_SOCKET, + MyLatch, NULL); + + if (IsUnderPostmaster) + AddWaitEventToSet(waitset, WL_EXIT_ON_PM_DEATH, PGINVALID_SOCKET, + NULL, NULL); + + Assert(latch_pos == ConditionVariableWaitSetLatchPos); + + return waitset; +} + /* * Prepare to wait on a given condition variable. * @@ -97,7 +162,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, ConditionVariableWaitSet, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +177,27 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, ConditionVariableWaitSet, timeout, + wait_event_info); +} + +/* + * Wait for a condition variable to be signaled, a timeout to be reached, or a + * socket event in the given waitset. The waitset must have been created by + * ConditionVariableWaitSetCreate(). + * + * Returns true when timeout expires, otherwise returns false. + * + * See ConditionVariableSleep() for general usage. + */ +bool +ConditionVariableEventSleep(ConditionVariable *cv, WaitEventSet *waitset, + long timeout, uint32 wait_event_info) { long cur_timeout = -1; instr_time start_time; instr_time cur_time; - int wait_events; /* * If the caller didn't prepare to sleep explicitly, then do so now and @@ -147,24 +229,28 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, INSTR_TIME_SET_CURRENT(start_time); Assert(timeout >= 0 && timeout <= INT_MAX); cur_timeout = timeout; - wait_events = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH; } - else - wait_events = WL_LATCH_SET | WL_EXIT_ON_PM_DEATH; while (true) { bool done = false; + WaitEvent cvEvent; + int nevents; /* - * Wait for latch to be set. (If we're awakened for some other - * reason, the code below will cope anyway.) + * Wait for latch to be set, or other events which will be handled + * below. */ - (void) WaitLatch(MyLatch, wait_events, cur_timeout, wait_event_info); + nevents = WaitEventSetWait(waitset, cur_timeout, &cvEvent, + 1, wait_event_info); /* Reset latch before examining the state of the wait list. */ ResetLatch(MyLatch); + /* If a socket event occurred, no need to check wait list. */ + if (nevents == 1 && (cvEvent.events & WL_SOCKET_MASK) != 0) + return true; + /* * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 22b4278610..ae4a7aecd4 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -440,6 +440,9 @@ InitProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); @@ -596,6 +599,9 @@ InitAuxiliaryProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c index 59532bbd80..8731d076cc 100644 --- a/src/backend/utils/init/miscinit.c +++ b/src/backend/utils/init/miscinit.c @@ -40,6 +40,7 @@ #include "postmaster/interrupt.h" #include "postmaster/pgarch.h" #include "postmaster/postmaster.h" +#include "storage/condition_variable.h" #include "storage/fd.h" #include "storage/ipc.h" #include "storage/latch.h" diff --git a/src/include/storage/condition_variable.h b/src/include/storage/condition_variable.h index 589bdd323c..94adb54b91 100644 --- a/src/include/storage/condition_variable.h +++ b/src/include/storage/condition_variable.h @@ -22,6 +22,7 @@ #ifndef CONDITION_VARIABLE_H #define CONDITION_VARIABLE_H +#include "storage/latch.h" #include "storage/proclist_types.h" #include "storage/spin.h" @@ -42,9 +43,14 @@ typedef union ConditionVariableMinimallyPadded char pad[CV_MINIMAL_SIZE]; } ConditionVariableMinimallyPadded; +extern void InitializeConditionVariableWaitSet(void); + /* Initialize a condition variable. */ extern void ConditionVariableInit(ConditionVariable *cv); +extern WaitEventSet *ConditionVariableWaitSetCreate(MemoryContext mcxt, + int n_socket_events); + /* * To sleep on a condition variable, a process should use a loop which first * checks the condition, exiting the loop if it is met, and then calls @@ -56,6 +62,10 @@ extern void ConditionVariableInit(ConditionVariable *cv); extern void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info); extern bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info); +extern bool ConditionVariableEventSleep(ConditionVariable *cv, + WaitEventSet *cvEventSet, + long timeout, + uint32 wait_event_info); extern void ConditionVariableCancelSleep(void); /* -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-03 16:23 Drouvot, Bertrand <[email protected]> parent: Jeff Davis <[email protected]> 1 sibling, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-03 16:23 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/2/23 8:45 PM, Jeff Davis wrote: > On Thu, 2023-03-02 at 10:20 +0100, Drouvot, Bertrand wrote: >> Right, but in our case, right after the wakeup (the one due to the CV >> broadcast, >> aka the one that will remove it from the wait queue) we'll exit the >> loop due to: >> >> " >> /* check whether we're done */ >> if (loc <= RecentFlushPtr) >> break; >> " >> >> as the CV broadcast means that a flush/replay occurred. > > But does it mean that the flush/replay advanced *enough* to be greater > than or equal to loc? > Yes I think so: loc is when we started waiting initially and RecentFlushPtr is >= to when the broadcast has been sent. >> - If it is awakened due to the CV broadcast, then we'll right after >> exit the loop (see above) > > ... > >> I think that's not needed as we'd exit the loop right after we are >> awakened by a CV broadcast. > > See the comment here: > WalSndWaitForWal > * If this process has been taken out of the wait list, then we know > * that it has been signaled by ConditionVariableSignal (or > * ConditionVariableBroadcast), so we should return to the caller. But > * that doesn't guarantee that the exit condition is met, only that we > * ought to check it. > > You seem to be arguing that in this case, it doesn't matter; that > walreceiver knows what walsender is waiting for, and will never wake it > up before it's ready. I don't think that's true, and even if it is, it > needs explanation. > What I think is that, in this particular case, we are sure that the loop exit condition is met as we know that loc <= RecentFlushPtr. >> >> I agree that's a good idea and that it should/would work too. I just >> wanted to highlight that in this particular >> case that might not be necessary to build this new API. > > In this case it looks easier to add the right API than to be sure about > whether it's needed or not. > What I meant is that of course I might be wrong. If we do not agree that the new API (in this particular case) is not needed then I agree that building the new API is the way to go ;-) (+ it offers the advantage to be able to be more precise while reporting the wait event). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-03 16:26 Drouvot, Bertrand <[email protected]> parent: Jeff Davis <[email protected]> 1 sibling, 2 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-03 16:26 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/3/23 8:58 AM, Jeff Davis wrote: > On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: >> In this case it looks easier to add the right API than to be sure >> about >> whether it's needed or not. > > I attached a sketch of one approach. Oh, that's very cool, thanks a lot! > I'm not very confident that it's > the right API or even that it works as I intended it, but if others > like the approach I can work on it some more. > I'll look at it early next week. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-04 11:19 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 1 sibling, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-04 11:19 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/3/23 5:26 PM, Drouvot, Bertrand wrote: > Hi, > > On 3/3/23 8:58 AM, Jeff Davis wrote: >> On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: >>> In this case it looks easier to add the right API than to be sure >>> about >>> whether it's needed or not. >> >> I attached a sketch of one approach. > > Oh, that's very cool, thanks a lot! > >> I'm not very confident that it's >> the right API or even that it works as I intended it, but if others >> like the approach I can work on it some more. >> > > I'll look at it early next week. > Just attaching a tiny rebase due to ebd551f586 breaking 0001 (did not look at your patch yet). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From fadeda04aefbad8202affeea4167ea915b7e80b3 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v52 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 From 52309916925ef711594834fb13c18c4e329931df Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v52 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/035_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.1% src/backend/ 4.0% src/test/perl/PostgreSQL/Test/ 89.7% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 97b3f1c1a6..bf4ef3fa98 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26557,6 +26557,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index abdc6e23f2..39cb3c0d26 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6393,6 +6393,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 3e2a27fb71..da58257f4f 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3060,6 +3060,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 59465b97f3..e834ad5e0d 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', 't/034_create_database.pl', + 't/035_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/035_standby_logical_decoding.pl b/src/test/recovery/t/035_standby_logical_decoding.pl new file mode 100644 index 0000000000..8c45180c35 --- /dev/null +++ b/src/test/recovery/t/035_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_replay_catchup($node_standby); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_replay_catchup($node_standby); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_replay_catchup($node_standby); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_replay_catchup($node_standby); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_replay_catchup($node_cascading_standby); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 From bd7bcb08bed0df7c80e86bc22327593116986fea Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v52 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..5034194e1b 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..e1b80e6202 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 56a9559555918a99c202a0924f7b2ede9de4e75d Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v52 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index de0cbe5e27..33a1b6f0e4 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4467,6 +4467,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 8fe7bb65f1..8457eec4c4 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index c3ec97a0a6..743d12ba14 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -124,23 +124,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -342,6 +341,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index c2523c5caf..3042e5bd64 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 From 29e5eae2a50d36fe02d38ea0f9db21da5dc5e1ee Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v52 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 6249bb50d0..cdf7c09b4b 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4663,6 +4663,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 6c36b3a326..20fb689e7f 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8677,6 +8677,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8846,6 +8847,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8963,6 +8965,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 87af608d15..de0cbe5e27 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7964,6 +7968,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 34ca0e739f..20c70be5a2 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 75e8363e24..c2523c5caf 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 9a73ae67d0..db5c3333cc 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -466,6 +467,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -491,6 +493,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -499,6 +504,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -517,7 +523,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1478,6 +1486,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..b5f9aa285c 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2488,6 +2488,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3057,6 +3060,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b61a12382b..1196716255 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 505595620e..abdc6e23f2 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10959,9 +10964,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index f43fac09ed..3aa0092751 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -338,6 +338,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e953d1f515..1b6600884e 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 From 9e62983cc174a4f645b47d8716dce5750e7a24f4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v52 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 11 +- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 201 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 4f50e0dd34..6c36b3a326 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6658,6 +6658,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8228,7 +8229,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8240,6 +8241,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 3f7b541e9d..a213407fee 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 02b9601bec..1207a49689 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 7777e7ec77..98a712f4ec 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3365,6 +3365,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 2ce9366277..93fb9d438a 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,11 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TODELETE OFFSET NUMBER ARRAY FOLLOWS */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -98,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index 9894ab9afe..6c5535fe73 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; uint16 ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(uint16)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a2c67d1cd3..08db7e62dd 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,13 +346,15 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* * In payload of blk 0 : FREEZE PLANS and OFFSET NUMBER ARRAY */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -409,7 +413,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd67257f2..823c700dee 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,6 +237,8 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /*---- * In payload of blk 0 : @@ -245,7 +249,7 @@ typedef struct xl_btree_delete */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 67f994cb3e..52845497cc 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 Attachments: [text/plain] v52-0006-Doc-changes-describing-details-about-logical-dec.patch (2.2K, ../../[email protected]/2-v52-0006-Doc-changes-describing-details-about-logical-dec.patch) download | inline diff: From fadeda04aefbad8202affeea4167ea915b7e80b3 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 14:08:11 +0000 Subject: [PATCH v52 6/6] Doc changes describing details about logical decoding. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/logicaldecoding.sgml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) 100.0% doc/src/sgml/ diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 4e912b4bd4..3da254ed1f 100644 --- a/doc/src/sgml/logicaldecoding.sgml +++ b/doc/src/sgml/logicaldecoding.sgml @@ -316,6 +316,28 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU may consume changes from a slot at any given time. </para> + <para> + A logical replication slot can also be created on a hot standby. To prevent + <command>VACUUM</command> from removing required rows from the system + catalogs, <varname>hot_standby_feedback</varname> should be set on the + standby. In spite of that, if any required rows get removed, the slot gets + invalidated. It's highly recommended to use a physical slot between the primary + and the standby. Otherwise, hot_standby_feedback will work, but only while the + connection is alive (for example a node restart would break it). Existing + logical slots on standby also get invalidated if wal_level on primary is reduced to + less than 'logical'. + </para> + + <para> + For a logical slot to be created, it builds a historic snapshot, for which + information of all the currently running transactions is essential. On + primary, this information is available, but on standby, this information + has to be obtained from primary. So, slot creation may wait for some + activity to happen on the primary. If the primary is idle, creating a + logical slot on standby may take a noticeable time. One option to speed it + is to call the <function>pg_log_standby_snapshot</function> on the primary. + </para> + <caution> <para> Replication slots persist across crashes and know nothing about the state -- 2.34.1 [text/plain] v52-0005-New-TAP-test-for-logical-decoding-on-standby.patch (32.9K, ../../[email protected]/3-v52-0005-New-TAP-test-for-logical-decoding-on-standby.patch) download | inline diff: From 52309916925ef711594834fb13c18c4e329931df Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:04:12 +0000 Subject: [PATCH v52 5/6] New TAP test for logical decoding on standby. In addition to the new TAP test, this commit introduces a new pg_log_standby_snapshot() function. The idea is to be able to take a snapshot of running transactions and write this to WAL without requesting for a (costly) checkpoint. Author: Craig Ringer (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- doc/src/sgml/func.sgml | 15 + src/backend/access/transam/xlogfuncs.c | 32 + src/backend/catalog/system_functions.sql | 2 + src/include/catalog/pg_proc.dat | 3 + src/test/perl/PostgreSQL/Test/Cluster.pm | 37 + src/test/recovery/meson.build | 1 + .../t/035_standby_logical_decoding.pl | 710 ++++++++++++++++++ 7 files changed, 800 insertions(+) 3.1% src/backend/ 4.0% src/test/perl/PostgreSQL/Test/ 89.7% src/test/recovery/t/ diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 97b3f1c1a6..bf4ef3fa98 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -26557,6 +26557,21 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset prepared with <xref linkend="sql-prepare-transaction"/>. </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_log_standby_snapshot</primary> + </indexterm> + <function>pg_log_standby_snapshot</function> () + <returnvalue>pg_lsn</returnvalue> + </para> + <para> + Take a snapshot of running transactions and write this to WAL without + having to wait bgwriter or checkpointer to log one. This one is useful for + logical decoding on standby for which logical slot creation is hanging + until such a record is replayed on the standby. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c index c07daa874f..481e9a47da 100644 --- a/src/backend/access/transam/xlogfuncs.c +++ b/src/backend/access/transam/xlogfuncs.c @@ -38,6 +38,7 @@ #include "utils/pg_lsn.h" #include "utils/timestamp.h" #include "utils/tuplestore.h" +#include "storage/standby.h" /* * Backup-related variables. @@ -196,6 +197,37 @@ pg_switch_wal(PG_FUNCTION_ARGS) PG_RETURN_LSN(switchpoint); } +/* + * pg_log_standby_snapshot: call LogStandbySnapshot() + * + * Permission checking for this function is managed through the normal + * GRANT system. + */ +Datum +pg_log_standby_snapshot(PG_FUNCTION_ARGS) +{ + XLogRecPtr recptr; + + if (RecoveryInProgress()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("pg_log_standby_snapshot() cannot be executed during recovery."))); + + if (!XLogStandbyInfoActive()) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("wal_level is not in desired state"), + errhint("wal_level has to be >= WAL_LEVEL_REPLICA."))); + + recptr = LogStandbySnapshot(); + + /* + * As a convenience, return the WAL location of the last inserted record + */ + PG_RETURN_LSN(recptr); +} + /* * pg_create_restore_point: a named point for restore * diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql index 83ca893444..b7c65ea37d 100644 --- a/src/backend/catalog/system_functions.sql +++ b/src/backend/catalog/system_functions.sql @@ -644,6 +644,8 @@ REVOKE EXECUTE ON FUNCTION pg_create_restore_point(text) FROM public; REVOKE EXECUTE ON FUNCTION pg_switch_wal() FROM public; +REVOKE EXECUTE ON FUNCTION pg_log_standby_snapshot() FROM public; + REVOKE EXECUTE ON FUNCTION pg_wal_replay_pause() FROM public; REVOKE EXECUTE ON FUNCTION pg_wal_replay_resume() FROM public; diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index abdc6e23f2..39cb3c0d26 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6393,6 +6393,9 @@ { oid => '2848', descr => 'switch to new wal file', proname => 'pg_switch_wal', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => '', prosrc => 'pg_switch_wal' }, +{ oid => '9658', descr => 'log details of the current snapshot to WAL', + proname => 'pg_log_standby_snapshot', provolatile => 'v', prorettype => 'pg_lsn', + proargtypes => '', prosrc => 'pg_log_standby_snapshot' }, { oid => '3098', descr => 'create a named restore point', proname => 'pg_create_restore_point', provolatile => 'v', prorettype => 'pg_lsn', proargtypes => 'text', diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 3e2a27fb71..da58257f4f 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -3060,6 +3060,43 @@ $SIG{TERM} = $SIG{INT} = sub { =pod +=item $node->create_logical_slot_on_standby(self, primary, slot_name, dbname) + +Create logical replication slot on given standby + +=cut + +sub create_logical_slot_on_standby +{ + my ($self, $primary, $slot_name, $dbname) = @_; + my ($stdout, $stderr); + + my $handle; + + $handle = IPC::Run::start(['pg_recvlogical', '-d', $self->connstr($dbname), '-P', 'test_decoding', '-S', $slot_name, '--create-slot'], '>', \$stdout, '2>', \$stderr); + + # Once slot restart_lsn is created, the standby looks for xl_running_xacts + # WAL record from the restart_lsn onwards. So firstly, wait until the slot + # restart_lsn is evaluated. + + $self->poll_query_until( + 'postgres', qq[ + SELECT restart_lsn IS NOT NULL + FROM pg_catalog.pg_replication_slots WHERE slot_name = '$slot_name' + ]) or die "timed out waiting for logical slot to calculate its restart_lsn"; + + # Now arrange for the xl_running_xacts record for which pg_recvlogical + # is waiting. + $primary->safe_psql('postgres', 'SELECT pg_log_standby_snapshot()'); + + $handle->finish(); + + is($self->slot($slot_name)->{'slot_type'}, 'logical', $slot_name . ' on standby created') + or die "could not create slot" . $slot_name; +} + +=pod + =back =cut diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build index 59465b97f3..e834ad5e0d 100644 --- a/src/test/recovery/meson.build +++ b/src/test/recovery/meson.build @@ -40,6 +40,7 @@ tests += { 't/032_relfilenode_reuse.pl', 't/033_replay_tsp_drops.pl', 't/034_create_database.pl', + 't/035_standby_logical_decoding.pl', ], }, } diff --git a/src/test/recovery/t/035_standby_logical_decoding.pl b/src/test/recovery/t/035_standby_logical_decoding.pl new file mode 100644 index 0000000000..8c45180c35 --- /dev/null +++ b/src/test/recovery/t/035_standby_logical_decoding.pl @@ -0,0 +1,710 @@ +# logical decoding on standby : test logical decoding, +# recovery conflict and standby promotion. + +use strict; +use warnings; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 67; + +my ($stdin, $stdout, $stderr, $cascading_stdout, $cascading_stderr, $ret, $handle, $slot); + +my $node_primary = PostgreSQL::Test::Cluster->new('primary'); +my $node_standby = PostgreSQL::Test::Cluster->new('standby'); +my $node_cascading_standby = PostgreSQL::Test::Cluster->new('cascading_standby'); +my $default_timeout = $PostgreSQL::Test::Utils::timeout_default; +my $res; + +# Name for the physical slot on primary +my $primary_slotname = 'primary_physical'; +my $standby_physical_slotname = 'standby_physical'; + +# find $pat in logfile of $node after $off-th byte +sub find_in_log +{ + my ($node, $pat, $off) = @_; + + $off = 0 unless defined $off; + my $log = PostgreSQL::Test::Utils::slurp_file($node->logfile); + return 0 if (length($log) <= $off); + + $log = substr($log, $off); + + return $log =~ m/$pat/; +} + +# Fetch xmin columns from slot's pg_replication_slots row, after waiting for +# given boolean condition to be true to ensure we've reached a quiescent state. +sub wait_for_xmins +{ + my ($node, $slotname, $check_expr) = @_; + + $node->poll_query_until( + 'postgres', qq[ + SELECT $check_expr + FROM pg_catalog.pg_replication_slots + WHERE slot_name = '$slotname'; + ]) or die "Timed out waiting for slot xmins to advance"; +} + +# Create the required logical slots on standby. +sub create_logical_slots +{ + my ($node) = @_; + $node->create_logical_slot_on_standby($node_primary, 'inactiveslot', 'testdb'); + $node->create_logical_slot_on_standby($node_primary, 'activeslot', 'testdb'); +} + +# Drop the logical slots on standby. +sub drop_logical_slots +{ + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('inactiveslot')]); + $node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('activeslot')]); +} + +# Acquire one of the standby logical slots created by create_logical_slots(). +# In case wait is true we are waiting for an active pid on the 'activeslot' slot. +# If wait is not true it means we are testing a known failure scenario. +sub make_slot_active +{ + my ($node, $wait, $to_stdout, $to_stderr) = @_; + my $slot_user_handle; + + $slot_user_handle = IPC::Run::start(['pg_recvlogical', '-d', $node->connstr('testdb'), '-S', 'activeslot', '-o', 'include-xids=0', '-o', 'skip-empty-xacts=1', '--no-loop', '--start', '-f', '-'], '>', $to_stdout, '2>', $to_stderr); + + if ($wait) + { + # make sure activeslot is in use + $node->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NOT NULL)" + ) or die "slot never became active"; + } + return $slot_user_handle; +} + +# Check pg_recvlogical stderr +sub check_pg_recvlogical_stderr +{ + my ($slot_user_handle, $check_stderr) = @_; + my $return; + + # our client should've terminated in response to the walsender error + $slot_user_handle->finish; + $return = $?; + cmp_ok($return, "!=", 0, "pg_recvlogical exited non-zero"); + if ($return) { + like($stderr, qr/$check_stderr/, 'slot has been invalidated'); + } + + return 0; +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub check_slots_dropped +{ + my ($slot_user_handle) = @_; + + is($node_standby->slot('inactiveslot')->{'slot_type'}, '', 'inactiveslot on standby dropped'); + is($node_standby->slot('activeslot')->{'slot_type'}, '', 'activeslot on standby dropped'); + + check_pg_recvlogical_stderr($slot_user_handle, "conflict with recovery"); +} + +# Check if all the slots on standby are dropped. These include the 'activeslot' +# that was acquired by make_slot_active(), and the non-active 'inactiveslot'. +sub change_hot_standby_feedback_and_wait_for_xmins +{ + my ($hsf, $invalidated) = @_; + + $node_standby->append_conf('postgresql.conf',qq[ + hot_standby_feedback = $hsf + ]); + + $node_standby->reload; + + if ($hsf && $invalidated) + { + # With hot_standby_feedback on, xmin should advance, + # but catalog_xmin should still remain NULL since there is no logical slot. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NULL"); + } + elsif ($hsf) + { + # With hot_standby_feedback on, xmin and catalog_xmin should advance. + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NOT NULL AND catalog_xmin IS NOT NULL"); + } + else + { + # Both should be NULL since hs_feedback is off + wait_for_xmins($node_primary, $primary_slotname, + "xmin IS NULL AND catalog_xmin IS NULL"); + + } +} + +# Check conflicting status in pg_replication_slots. +sub check_slots_conflicting_status +{ + my ($conflicting) = @_; + + if ($conflicting) + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_and(conflicting) from pg_replication_slots;)); + + is($res, 't', + "Logical slots are reported as conflicting"); + } + else + { + $res = $node_standby->safe_psql( + 'postgres', qq( + select bool_or(conflicting) from pg_replication_slots;)); + + is($res, 'f', + "Logical slots are reported as non conflicting"); + } +} + +######################## +# Initialize primary node +######################## + +$node_primary->init(allows_streaming => 1, has_archiving => 1); +$node_primary->append_conf('postgresql.conf', q{ +wal_level = 'logical' +max_replication_slots = 4 +max_wal_senders = 4 +log_min_messages = 'debug2' +log_error_verbosity = verbose +}); +$node_primary->dump_info; +$node_primary->start; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); + +$node_primary->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$primary_slotname');]); + +# Check conflicting is NULL for physical slot +$res = $node_primary->safe_psql( + 'postgres', qq[ + SELECT conflicting is null FROM pg_replication_slots where slot_name = '$primary_slotname';]); + +is($res, 't', + "Physical slot reports conflicting as NULL"); + +my $backup_name = 'b1'; +$node_primary->backup($backup_name); + +####################### +# Initialize standby node +####################### + +$node_standby->init_from_backup( + $node_primary, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$primary_slotname']); +$node_standby->start; +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->safe_psql('testdb', qq[SELECT * FROM pg_create_physical_replication_slot('$standby_physical_slotname');]); + +####################### +# Initialize cascading standby node +####################### +$node_standby->backup($backup_name); +$node_cascading_standby->init_from_backup( + $node_standby, $backup_name, + has_streaming => 1, + has_restoring => 1); +$node_cascading_standby->append_conf('postgresql.conf', + qq[primary_slot_name = '$standby_physical_slotname']); +$node_cascading_standby->start; +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +################################################## +# Test that logical decoding on the standby +# behaves correctly. +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,10) s;]); + +$node_primary->wait_for_replay_catchup($node_standby); + +my $result = $node_standby->safe_psql('testdb', + qq[SELECT pg_logical_slot_get_changes('activeslot', NULL, NULL);]); + +# test if basic decoding works +is(scalar(my @foobar = split /^/m, $result), + 14, 'Decoding produced 14 rows (2 BEGIN/COMMIT and 10 rows)'); + +# Insert some rows and verify that we get the same results from pg_recvlogical +# and the SQL interface. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +my $expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT}; + +$node_primary->wait_for_replay_catchup($node_standby); + +my $stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session'); + +my $endpos = $node_standby->safe_psql('testdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" +); + +# Insert some rows after $endpos, which we won't read. +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,50) s;] +); + +$node_primary->wait_for_catchup($node_standby); + +my $stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +$node_standby->poll_query_until('testdb', + "SELECT EXISTS (SELECT 1 FROM pg_replication_slots WHERE slot_name = 'activeslot' AND active_pid IS NULL)" +) or die "slot never became inactive"; + +$stdout_recv = $node_standby->pg_recvlogical_upto( + 'testdb', 'activeslot', $endpos, $default_timeout, + 'include-xids' => '0', + 'skip-empty-xacts' => '1'); +chomp($stdout_recv); +is($stdout_recv, '', 'pg_recvlogical acknowledged changes'); + +$node_primary->safe_psql('postgres', 'CREATE DATABASE otherdb'); + +is( $node_primary->psql( + 'otherdb', + "SELECT lsn FROM pg_logical_slot_peek_changes('activeslot', NULL, NULL) ORDER BY lsn DESC LIMIT 1;" + ), + 3, + 'replaying logical slot from another database fails'); + +# drop the logical slots +drop_logical_slots(); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 1: hot_standby_feedback off and vacuum FULL +################################################## + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum full on pg_class with hot_standby_feedback turned off on +# the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM full pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery"), + 'inactiveslot slot invalidation is logged with vacuum FULL on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery"), + 'activeslot slot invalidation is logged with vacuum FULL on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 1) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1,1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 2: conflict due to row removal with hot_standby_feedback off. +################################################## + +# get the position to search from in the standby logfile +my $logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to create/drop a relation and +# launch a vacuum on pg_class with hot_standby_feedback turned off on the standby. +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[DROP TABLE conflict_test;]); +$node_primary->safe_psql('testdb', 'VACUUM pg_class;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with vacuum on pg_class'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with vacuum on pg_class'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# Recovery conflict: Same as Scenario 2 but on a non catalog table +# Scenario 3: No conflict expected. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# put hot standby feedback to off +change_hot_standby_feedback_and_wait_for_xmins(0,1); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should not trigger a conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE conflict_test(x integer, y text);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO conflict_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;]); +$node_primary->safe_psql('testdb', qq[UPDATE conflict_test set x=1, y=1;]); +$node_primary->safe_psql('testdb', 'VACUUM conflict_test;'); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should not be issued +ok( !find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is not logged with vacuum on conflict_test'); + +ok( !find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is not logged with vacuum on conflict_test'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has not been updated +# we now still expect 2 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 2) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot not updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as non conflicting in pg_replication_slots +check_slots_conflicting_status(0); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 0); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 4: conflict due to on-access pruning. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +# One way to produce recovery conflict is to trigger an on-access pruning +# on a relation marked as user_catalog_table. +change_hot_standby_feedback_and_wait_for_xmins(0,0); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# This should trigger the conflict +$node_primary->safe_psql('testdb', qq[CREATE TABLE prun(id integer, s char(2000)) WITH (fillfactor = 75, user_catalog_table = true);]); +$node_primary->safe_psql('testdb', qq[INSERT INTO prun VALUES (1, 'A');]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'B';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'C';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'D';]); +$node_primary->safe_psql('testdb', qq[UPDATE prun SET s = 'E';]); + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged with on-access pruning'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged with on-access pruning'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 3) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); + +# We are not able to read from the slot as it has been invalidated +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +# Turn hot_standby_feedback back on +change_hot_standby_feedback_and_wait_for_xmins(1, 1); + +################################################## +# Recovery conflict: Invalidate conflicting slots, including in-use slots +# Scenario 5: incorrect wal_level on primary. +################################################## + +# get the position to search from in the standby logfile +$logstart = -s $node_standby->logfile; + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); + +# Make primary wal_level replica. This will trigger slot conflict. +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'replica' +]); +$node_primary->restart; + +$node_primary->wait_for_replay_catchup($node_standby); + +# message should be issued +ok( find_in_log( + $node_standby, + "invalidating slot \"inactiveslot\" because it conflicts with recovery", $logstart), + 'inactiveslot slot invalidation is logged due to wal_level'); + +ok( find_in_log( + $node_standby, + "invalidating slot \"activeslot\" because it conflicts with recovery", $logstart), + 'activeslot slot invalidation is logged due to wal_level'); + +# Verify that pg_stat_database_conflicts.confl_active_logicalslot has been updated +# we now expect 3 conflicts reported as the counter persist across reloads +ok( $node_standby->poll_query_until( + 'postgres', + "select (confl_active_logicalslot = 4) from pg_stat_database_conflicts where datname = 'testdb'", 't'), + 'confl_active_logicalslot updated') or die "Timed out waiting confl_active_logicalslot to be updated"; + +# Verify slots are reported as conflicting in pg_replication_slots +check_slots_conflicting_status(1); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# We are not able to read from the slot as it requires wal_level at least logical on the primary server +check_pg_recvlogical_stderr($handle, "logical decoding on standby requires wal_level to be at least logical on the primary server"); + +# Restore primary wal_level +$node_primary->append_conf('postgresql.conf',q[ +wal_level = 'logical' +]); +$node_primary->restart; +$node_primary->wait_for_replay_catchup($node_standby); + +$handle = make_slot_active($node_standby, 0, \$stdout, \$stderr); +# as the slot has been invalidated we should not be able to read +check_pg_recvlogical_stderr($handle, "cannot read from logical replication slot \"activeslot\""); + +################################################## +# DROP DATABASE should drops it's slots, including active slots. +################################################## + +# drop the logical slots +drop_logical_slots(); + +# create the logical slots +create_logical_slots($node_standby); + +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +# Create a slot on a database that would not be dropped. This slot should not +# get dropped. +$node_standby->create_logical_slot_on_standby($node_primary, 'otherslot', 'postgres'); + +# dropdb on the primary to verify slots are dropped on standby +$node_primary->safe_psql('postgres', q[DROP DATABASE testdb]); + +$node_primary->wait_for_replay_catchup($node_standby); + +is($node_standby->safe_psql('postgres', + q[SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = 'testdb')]), 'f', + 'database dropped on standby'); + +check_slots_dropped($handle); + +is($node_standby->slot('otherslot')->{'slot_type'}, 'logical', + 'otherslot on standby not dropped'); + +# Cleanup : manually drop the slot that was not dropped. +$node_standby->psql('postgres', q[SELECT pg_drop_replication_slot('otherslot')]); + +################################################## +# Test standby promotion and logical decoding behavior +# after the standby gets promoted. +################################################## + +# reduce wal_sender_timeout to not wait too long after promotion +$node_standby->append_conf('postgresql.conf',qq[ + wal_sender_timeout = 1s +]); + +$node_standby->reload; + +$node_primary->psql('postgres', q[CREATE DATABASE testdb]); +$node_primary->safe_psql('testdb', qq[CREATE TABLE decoding_test(x integer, y text);]); + +# create the logical slots +create_logical_slots($node_standby); + +# create the logical slots on the cascading standby too +create_logical_slots($node_cascading_standby); + +# Make slots actives +$handle = make_slot_active($node_standby, 1, \$stdout, \$stderr); +my $cascading_handle = make_slot_active($node_cascading_standby, 1, \$cascading_stdout, \$cascading_stderr); + +# Insert some rows before the promotion +$node_primary->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(1,4) s;] +); + +# Wait for both standbys to catchup +$node_primary->wait_for_replay_catchup($node_standby); +$node_standby->wait_for_replay_catchup($node_cascading_standby, $node_primary); + +# promote +$node_standby->promote; + +# insert some rows on promoted standby +$node_standby->safe_psql('testdb', + qq[INSERT INTO decoding_test(x,y) SELECT s, s::text FROM generate_series(5,7) s;] +); + +# Wait for the cascading standby to catchup +$node_standby->wait_for_replay_catchup($node_cascading_standby); + +$expected = q{BEGIN +table public.decoding_test: INSERT: x[integer]:1 y[text]:'1' +table public.decoding_test: INSERT: x[integer]:2 y[text]:'2' +table public.decoding_test: INSERT: x[integer]:3 y[text]:'3' +table public.decoding_test: INSERT: x[integer]:4 y[text]:'4' +COMMIT +BEGIN +table public.decoding_test: INSERT: x[integer]:5 y[text]:'5' +table public.decoding_test: INSERT: x[integer]:6 y[text]:'6' +table public.decoding_test: INSERT: x[integer]:7 y[text]:'7' +COMMIT}; + +# check that we are decoding pre and post promotion inserted rows +$stdout_sql = $node_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on promoted standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion +my $pump_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default); + +ok( pump_until( + $handle, $pump_timeout, \$stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($stdout); +is($stdout, $expected, + 'got same expected output from pg_recvlogical decoding session'); + +# check that we are decoding pre and post promotion inserted rows on the cascading standby +$stdout_sql = $node_cascading_standby->safe_psql('testdb', + qq[SELECT data FROM pg_logical_slot_peek_changes('inactiveslot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');] +); + +is($stdout_sql, $expected, 'got expected output from SQL decoding session on cascading standby'); + +# check that we are decoding pre and post promotion inserted rows +# with pg_recvlogical that has started before the promotion on the cascading standby +ok( pump_until( + $cascading_handle, $pump_timeout, \$cascading_stdout, qr/^.*COMMIT.*COMMIT$/s), + 'got 2 COMMIT from pg_recvlogical output'); + +chomp($cascading_stdout); +is($cascading_stdout, $expected, + 'got same expected output from pg_recvlogical decoding session on cascading standby'); -- 2.34.1 [text/plain] v52-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch (7.7K, ../../[email protected]/4-v52-0004-Fixing-Walsender-corner-case-with-logical-decodi.patch) download | inline diff: From bd7bcb08bed0df7c80e86bc22327593116986fea Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 09:00:29 +0000 Subject: [PATCH v52 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable to fix this corner case. --- src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++ src/backend/replication/walsender.c | 34 +++++++++++++++++------ src/backend/utils/activity/wait_event.c | 3 ++ src/include/access/xlogrecovery.h | 3 ++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 6 files changed, 62 insertions(+), 8 deletions(-) 43.2% src/backend/access/transam/ 46.1% src/backend/replication/ 3.8% src/backend/utils/activity/ 3.7% src/include/access/ 3.1% src/include/ diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..5034194e1b 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1552,6 +1552,7 @@ WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *replayedCV = check_for_replay(); /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1566,10 +1567,15 @@ WalSndWaitForWal(XLogRecPtr loc) if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(NULL); else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + /* Prepare the replayedCV to sleep */ + ConditionVariablePrepareToSleep(replayedCV); + } for (;;) { + long sleeptime; /* Clear any already-pending wakeups */ @@ -1653,21 +1659,33 @@ WalSndWaitForWal(XLogRecPtr loc) /* Send keepalive if the time has come */ WalSndKeepaliveIfNecessary(); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); /* - * Sleep until something happens or we time out. Also wait for the - * socket becoming writable, if there's still pending output. + * When not in recovery, sleep until something happens or we time out. + * Also wait for the socket becoming writable, if there's still pending output. * Otherwise we might sit on sendable output data while waiting for * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); - - wakeEvents = WL_SOCKET_READABLE; + if (!RecoveryInProgress()) + { + wakeEvents = WL_SOCKET_READABLE; - if (pq_is_send_pending()) - wakeEvents |= WL_SOCKET_WRITEABLE; + if (pq_is_send_pending()) + wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + } + else + { + /* + * We are in the logical decoding on standby case. + * We are waiting for the startup process to replay wal record(s) using + * a timeout in case we are requested to stop. + */ + ConditionVariableTimedSleep(replayedCV, sleeptime, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY); + } } /* reactivate latch so WalSndLoop knows to continue */ diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..e1b80e6202 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalReceiverWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v52-0003-Allow-logical-decoding-on-standby.patch (11.8K, ../../[email protected]/5-v52-0003-Allow-logical-decoding-on-standby.patch) download | inline diff: From 56a9559555918a99c202a0924f7b2ede9de4e75d Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:59:47 +0000 Subject: [PATCH v52 3/6] Allow logical decoding on standby. Allow a logical slot to be created on standby. Restrict its usage or its creation if wal_level on primary is less than logical. During slot creation, it's restart_lsn is set to the last replayed LSN. Effectively, a logical slot creation on standby waits for an xl_running_xact record to arrive from primary. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello --- src/backend/access/transam/xlog.c | 11 +++++ src/backend/replication/logical/decode.c | 22 ++++++++- src/backend/replication/logical/logical.c | 37 ++++++++------- src/backend/replication/slot.c | 57 ++++++++++++----------- src/backend/replication/walsender.c | 41 ++++++++++------ src/include/access/xlog.h | 1 + 6 files changed, 111 insertions(+), 58 deletions(-) 4.7% src/backend/access/transam/ 38.7% src/backend/replication/logical/ 55.6% src/backend/replication/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index de0cbe5e27..33a1b6f0e4 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4467,6 +4467,17 @@ LocalProcessControlFile(bool reset) ReadControlFile(); } +/* + * Get the wal_level from the control file. For a standby, this value should be + * considered as its active wal_level, because it may be different from what + * was originally configured on standby. + */ +WalLevel +GetActiveWalLevelOnStandby(void) +{ + return ControlFile->wal_level; +} + /* * Initialization of shared memory for XLOG */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 8fe7bb65f1..8457eec4c4 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -152,11 +152,31 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) * can restart from there. */ break; + case XLOG_PARAMETER_CHANGE: + { + xl_parameter_change *xlrec = + (xl_parameter_change *) XLogRecGetData(buf->record); + + /* + * If wal_level on primary is reduced to less than logical, then we + * want to prevent existing logical slots from being used. + * Existing logical slots on standby get invalidated when this WAL + * record is replayed; and further, slot creation fails when the + * wal level is not sufficient; but all these operations are not + * synchronized, so a logical slot may creep in while the wal_level + * is being reduced. Hence this extra check. + */ + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + break; + } case XLOG_NOOP: case XLOG_NEXTOID: case XLOG_SWITCH: case XLOG_BACKUP_END: - case XLOG_PARAMETER_CHANGE: case XLOG_RESTORE_POINT: case XLOG_FPW_CHANGE: case XLOG_FPI_FOR_HINT: diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index c3ec97a0a6..743d12ba14 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -124,23 +124,22 @@ CheckLogicalDecodingRequirements(void) (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("logical decoding requires a database connection"))); - /* ---- - * TODO: We got to change that someday soon... - * - * There's basically three things missing to allow this: - * 1) We need to be able to correctly and quickly identify the timeline a - * LSN belongs to - * 2) We need to force hot_standby_feedback to be enabled at all times so - * the primary cannot remove rows we need. - * 3) support dropping replication slots referring to a database, in - * dbase_redo. There can't be any active ones due to HS recovery - * conflicts, so that should be relatively easy. - * ---- - */ if (RecoveryInProgress()) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("logical decoding cannot be used while in recovery"))); + { + /* + * This check may have race conditions, but whenever + * XLOG_PARAMETER_CHANGE indicates that wal_level has changed, we + * verify that there are no existing logical replication slots. And to + * avoid races around creating a new slot, + * CheckLogicalDecodingRequirements() is called once before creating + * the slot, and once when logical decoding is initially starting up. + */ + if (GetActiveWalLevelOnStandby() < WAL_LEVEL_LOGICAL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("logical decoding on standby requires wal_level " + "to be at least logical on the primary server"))); + } } /* @@ -342,6 +341,12 @@ CreateInitDecodingContext(const char *plugin, LogicalDecodingContext *ctx; MemoryContext old_context; + /* + * On standby, this check is also required while creating the slot. Check + * the comments in this function. + */ + CheckLogicalDecodingRequirements(); + /* shorter lines... */ slot = MyReplicationSlot; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 38c6f18886..290d4b45f4 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -51,6 +51,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "utils/builtins.h" +#include "access/xlogrecovery.h" /* * Replication slot on-disk data structure. @@ -1177,37 +1178,28 @@ ReplicationSlotReserveWal(void) /* * For logical slots log a standby snapshot and start logical decoding * at exactly that position. That allows the slot to start up more - * quickly. + * quickly. But on a standby we cannot do WAL writes, so just use the + * replay pointer; effectively, an attempt to create a logical slot on + * standby will cause it to wait for an xl_running_xact record to be + * logged independently on the primary, so that a snapshot can be built + * using the record. * - * That's not needed (or indeed helpful) for physical slots as they'll - * start replay at the last logged checkpoint anyway. Instead return - * the location of the last redo LSN. While that slightly increases - * the chance that we have to retry, it's where a base backup has to - * start replay at. + * None of this is needed (or indeed helpful) for physical slots as + * they'll start replay at the last logged checkpoint anyway. Instead + * return the location of the last redo LSN. While that slightly + * increases the chance that we have to retry, it's where a base backup + * has to start replay at. */ - if (!RecoveryInProgress() && SlotIsLogical(slot)) - { - XLogRecPtr flushptr; - - /* start at current insert position */ + if (SlotIsPhysical(slot)) + restart_lsn = GetRedoRecPtr(); + else if (RecoveryInProgress()) + restart_lsn = GetXLogReplayRecPtr(NULL); + else restart_lsn = GetXLogInsertRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - - /* make sure we have enough information to start */ - flushptr = LogStandbySnapshot(); - /* and make sure it's fsynced to disk */ - XLogFlush(flushptr); - } - else - { - restart_lsn = GetRedoRecPtr(); - SpinLockAcquire(&slot->mutex); - slot->data.restart_lsn = restart_lsn; - SpinLockRelease(&slot->mutex); - } + SpinLockAcquire(&slot->mutex); + slot->data.restart_lsn = restart_lsn; + SpinLockRelease(&slot->mutex); /* prevent WAL removal as fast as possible */ ReplicationSlotsComputeRequiredLSN(); @@ -1223,6 +1215,17 @@ ReplicationSlotReserveWal(void) if (XLogGetLastRemovedSegno() < segno) break; } + + if (!RecoveryInProgress() && SlotIsLogical(slot)) + { + XLogRecPtr flushptr; + + /* make sure we have enough information to start */ + flushptr = LogStandbySnapshot(); + + /* and make sure it's fsynced to disk */ + XLogFlush(flushptr); + } } /* diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index c2523c5caf..3042e5bd64 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -906,23 +906,31 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req int count; WALReadError errinfo; XLogSegNo segno; - TimeLineID currTLI = GetWALInsertionTimeLine(); + TimeLineID currTLI; /* - * Since logical decoding is only permitted on a primary server, we know - * that the current timeline ID can't be changing any more. If we did this - * on a standby, we'd have to worry about the values we compute here - * becoming invalid due to a promotion or timeline change. + * Since logical decoding is also permitted on a standby server, we need + * to check if the server is in recovery to decide how to get the current + * timeline ID (so that it also cover the promotion or timeline change cases). */ + + /* make sure we have enough WAL available */ + flushptr = WalSndWaitForWal(targetPagePtr + reqLen); + + /* the standby could have been promoted, so check if still in recovery */ + am_cascading_walsender = RecoveryInProgress(); + + if (am_cascading_walsender) + GetXLogReplayRecPtr(&currTLI); + else + currTLI = GetWALInsertionTimeLine(); + XLogReadDetermineTimeline(state, targetPagePtr, reqLen, currTLI); sendTimeLineIsHistoric = (state->currTLI != currTLI); sendTimeLine = state->currTLI; sendTimeLineValidUpto = state->currTLIValidUntil; sendTimeLineNextTLI = state->nextTLI; - /* make sure we have enough WAL available */ - flushptr = WalSndWaitForWal(targetPagePtr + reqLen); - /* fail if not (implies we are going to shut down) */ if (flushptr < targetPagePtr + reqLen) return -1; @@ -937,7 +945,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req cur_page, targetPagePtr, XLOG_BLCKSZ, - state->seg.ws_tli, /* Pass the current TLI because only + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new * TLI is needed. */ &errinfo)) @@ -3074,10 +3082,14 @@ XLogSendLogical(void) * If first time through in this session, initialize flushPtr. Otherwise, * we only need to update flushPtr if EndRecPtr is past it. */ - if (flushPtr == InvalidXLogRecPtr) - flushPtr = GetFlushRecPtr(NULL); - else if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) - flushPtr = GetFlushRecPtr(NULL); + if (flushPtr == InvalidXLogRecPtr || + logical_decoding_ctx->reader->EndRecPtr >= flushPtr) + { + if (am_cascading_walsender) + flushPtr = GetStandbyFlushRecPtr(NULL); + else + flushPtr = GetFlushRecPtr(NULL); + } /* If EndRecPtr is still past our flushPtr, it means we caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= flushPtr) @@ -3168,7 +3180,8 @@ GetStandbyFlushRecPtr(TimeLineID *tli) receivePtr = GetWalRcvFlushRecPtr(NULL, &receiveTLI); replayPtr = GetXLogReplayRecPtr(&replayTLI); - *tli = replayTLI; + if (tli) + *tli = replayTLI; result = replayPtr; if (receiveTLI == replayTLI && receivePtr > replayPtr) diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index cfe5409738..48ca852381 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -230,6 +230,7 @@ extern void XLOGShmemInit(void); extern void BootStrapXLOG(void); extern void InitializeWalConsistencyChecking(void); extern void LocalProcessControlFile(bool reset); +extern WalLevel GetActiveWalLevelOnStandby(void); extern void StartupXLOG(void); extern void ShutdownXLOG(int code, Datum arg); extern void CreateCheckPoint(int flags); -- 2.34.1 [text/plain] v52-0002-Handle-logical-slot-conflicts-on-standby.patch (37.0K, ../../[email protected]/6-v52-0002-Handle-logical-slot-conflicts-on-standby.patch) download | inline diff: From 29e5eae2a50d36fe02d38ea0f9db21da5dc5e1ee Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:57:56 +0000 Subject: [PATCH v52 2/6] Handle logical slot conflicts on standby. During WAL replay on standby, when slot conflict is identified, invalidate such slots. Also do the same thing if wal_level on the primary server is reduced to below logical and there are existing logical slots on standby. Introduce a new ProcSignalReason value for slot conflict recovery. Arrange for a new pg_stat_database_conflicts field: confl_active_logicalslot. Add a new field "conflicting" in pg_replication_slots. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Bharath Rupireddy --- doc/src/sgml/monitoring.sgml | 11 + doc/src/sgml/system-views.sgml | 10 + src/backend/access/gist/gistxlog.c | 2 + src/backend/access/hash/hash_xlog.c | 1 + src/backend/access/heap/heapam.c | 3 + src/backend/access/nbtree/nbtxlog.c | 2 + src/backend/access/spgist/spgxlog.c | 1 + src/backend/access/transam/xlog.c | 24 ++- src/backend/catalog/system_views.sql | 6 +- .../replication/logical/logicalfuncs.c | 13 +- src/backend/replication/slot.c | 198 +++++++++++++----- src/backend/replication/slotfuncs.c | 13 +- src/backend/replication/walsender.c | 8 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/ipc/standby.c | 13 +- src/backend/tcop/postgres.c | 24 +++ src/backend/utils/activity/pgstat_database.c | 4 + src/backend/utils/adt/pgstatfuncs.c | 3 + src/include/catalog/pg_proc.dat | 11 +- src/include/pgstat.h | 1 + src/include/replication/slot.h | 5 +- src/include/storage/procsignal.h | 1 + src/include/storage/standby.h | 2 + src/test/regress/expected/rules.out | 8 +- 24 files changed, 304 insertions(+), 63 deletions(-) 5.4% doc/src/sgml/ 7.2% src/backend/access/transam/ 4.7% src/backend/replication/logical/ 56.8% src/backend/replication/ 4.5% src/backend/storage/ipc/ 6.5% src/backend/tcop/ 5.4% src/backend/ 3.9% src/include/catalog/ 3.0% src/include/replication/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 6249bb50d0..cdf7c09b4b 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -4663,6 +4663,17 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i deadlocks </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>confl_active_logicalslot</structfield> <type>bigint</type> + </para> + <para> + Number of active logical slots in this database that have been + invalidated because they conflict with recovery (note that inactive ones + are also invalidated but do not increment this counter) + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..239f713295 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -2516,6 +2516,16 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx false for physical slots. </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>conflicting</structfield> <type>bool</type> + </para> + <para> + True if this logical slot conflicted with recovery (and so is now + invalidated). Always NULL for physical slots. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b7678f3c14..9a86fb3fef 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -197,6 +197,7 @@ gistRedoDeleteRecord(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } @@ -390,6 +391,7 @@ gistRedoPageReuse(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index 08ceb91288..b856304746 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1003,6 +1003,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, rlocator); } diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 6c36b3a326..20fb689e7f 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -8677,6 +8677,7 @@ heap_xlog_prune(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); /* @@ -8846,6 +8847,7 @@ heap_xlog_visible(XLogReaderState *record) */ if (InHotStandby) ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->flags & VISIBILITYMAP_IS_CATALOG_REL, rlocator); /* @@ -8963,6 +8965,7 @@ heap_xlog_freeze_page(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 414ca4f6de..c87e46ed66 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -669,6 +669,7 @@ btree_xlog_delete(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &rlocator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, rlocator); } @@ -1007,6 +1008,7 @@ btree_xlog_reuse_page(XLogReaderState *record) if (InHotStandby) ResolveRecoveryConflictWithSnapshotFullXid(xlrec->snapshotConflictHorizon, + xlrec->isCatalogRel, xlrec->locator); } diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index b071b59c8a..459ac929ba 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -879,6 +879,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) XLogRecGetBlockTag(record, 0, &locator, NULL, NULL); ResolveRecoveryConflictWithSnapshot(xldata->snapshotConflictHorizon, + xldata->isCatalogRel, locator); } diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 87af608d15..de0cbe5e27 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6447,6 +6447,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + bool invalidated = false; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -6807,7 +6808,8 @@ CreateCheckPoint(int flags) */ XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size); KeepLogSeg(recptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7086,6 +7088,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + bool invalidated = false; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7251,7 +7254,8 @@ CreateRestartPoint(int flags) replayPtr = GetXLogReplayRecPtr(&replayTLI); endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr; KeepLogSeg(endptr, &_logSegNo); - if (InvalidateObsoleteReplicationSlots(_logSegNo)) + InvalidateObsoleteReplicationSlots(_logSegNo, &invalidated, InvalidOid, NULL); + if (invalidated) { /* * Some slots have been invalidated; recalculate the old-segment @@ -7964,6 +7968,22 @@ xlog_redo(XLogReaderState *record) /* Update our copy of the parameters in pg_control */ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_parameter_change)); + /* + * Invalidate logical slots if we are in hot standby and the primary does not + * have a WAL level sufficient for logical decoding. No need to search + * for potentially conflicting logically slots if standby is running + * with wal_level lower than logical, because in that case, we would + * have either disallowed creation of logical slots or invalidated existing + * ones. + */ + if (InRecovery && InHotStandby && + xlrec.wal_level < WAL_LEVEL_LOGICAL && + wal_level >= WAL_LEVEL_LOGICAL) + { + TransactionId ConflictHorizon = InvalidTransactionId; + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, InvalidOid, &ConflictHorizon); + } + LWLockAcquire(ControlFileLock, LW_EXCLUSIVE); ControlFile->MaxConnections = xlrec.MaxConnections; ControlFile->max_worker_processes = xlrec.max_worker_processes; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 34ca0e739f..20c70be5a2 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -997,7 +997,8 @@ CREATE VIEW pg_replication_slots AS L.confirmed_flush_lsn, L.wal_status, L.safe_wal_size, - L.two_phase + L.two_phase, + L.conflicting FROM pg_get_replication_slots() AS L LEFT JOIN pg_database D ON (L.datoid = D.oid); @@ -1065,7 +1066,8 @@ CREATE VIEW pg_stat_database_conflicts AS pg_stat_get_db_conflict_lock(D.oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(D.oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(D.oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(D.oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(D.oid) AS confl_active_logicalslot FROM pg_database D; CREATE VIEW pg_stat_user_functions AS diff --git a/src/backend/replication/logical/logicalfuncs.c b/src/backend/replication/logical/logicalfuncs.c index fa1b641a2b..070fd378e8 100644 --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -216,9 +216,9 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin /* * After the sanity checks in CreateDecodingContext, make sure the - * restart_lsn is valid. Avoid "cannot get changes" wording in this - * errmsg because that'd be confusingly ambiguous about no changes - * being available. + * restart_lsn is valid or both xmin and catalog_xmin are valid. Avoid + * "cannot get changes" wording in this errmsg because that'd be + * confusingly ambiguous about no changes being available. */ if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, @@ -227,6 +227,13 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool confirm, bool bin NameStr(*name)), errdetail("This slot has never previously reserved WAL, or it has been invalidated."))); + if (LogicalReplicationSlotIsInvalid(MyReplicationSlot)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + NameStr(*name)), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + MemoryContextSwitchTo(oldcontext); /* diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index f286918f69..38c6f18886 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -855,8 +855,10 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) SpinLockAcquire(&s->mutex); effective_xmin = s->effective_xmin; effective_catalog_xmin = s->effective_catalog_xmin; - invalidated = (!XLogRecPtrIsInvalid(s->data.invalidated_at) && - XLogRecPtrIsInvalid(s->data.restart_lsn)); + invalidated = ((!XLogRecPtrIsInvalid(s->data.invalidated_at) && + XLogRecPtrIsInvalid(s->data.restart_lsn)) + || (!TransactionIdIsValid(s->data.xmin) && + !TransactionIdIsValid(s->data.catalog_xmin))); SpinLockRelease(&s->mutex); /* invalidated slots need not apply */ @@ -1224,20 +1226,21 @@ ReplicationSlotReserveWal(void) } /* - * Helper for InvalidateObsoleteReplicationSlots -- acquires the given slot - * and mark it invalid, if necessary and possible. + * Helper for InvalidateObsoleteReplicationSlots + * + * Acquires the given slot and mark it invalid, if necessary and possible. * * Returns whether ReplicationSlotControlLock was released in the interim (and * in that case we're not holding the lock at return, otherwise we are). * - * Sets *invalidated true if the slot was invalidated. (Untouched otherwise.) + * Sets *invalidated true if an obsolete slot was invalidated. (Untouched otherwise.) * * This is inherently racy, because we release the LWLock * for syscalls, so caller must restart if we return true. */ static bool -InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, - bool *invalidated) +InvalidatePossiblyObsoleteOrConflictingLogicalSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, + bool *invalidated, TransactionId *xid) { int last_signaled_pid = 0; bool released_lock = false; @@ -1245,6 +1248,9 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, for (;;) { XLogRecPtr restart_lsn; + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + NameData slotname; int active_pid = 0; @@ -1261,18 +1267,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, * Check if the slot needs to be invalidated. If it needs to be * invalidated, and is not currently acquired, acquire it and mark it * as having been invalidated. We do this with the spinlock held to - * avoid race conditions -- for example the restart_lsn could move - * forward, or the slot could be dropped. + * avoid race conditions -- for example the restart_lsn (or the + * xmin(s) could) move forward or the slot could be dropped. */ SpinLockAcquire(&s->mutex); restart_lsn = s->data.restart_lsn; + slot_xmin = s->data.xmin; + slot_catalog_xmin = s->data.catalog_xmin; + + /* slot has been invalidated (logical decoding conflict case) */ + if ((xid && + ((LogicalReplicationSlotIsInvalid(s)) + || /* - * If the slot is already invalid or is fresh enough, we don't need to - * do anything. + * We are not forcing for invalidation because the xid is valid and + * this is a non conflicting slot. */ - if (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN) + (TransactionIdIsValid(*xid) && !( + (TransactionIdIsValid(slot_xmin) && TransactionIdPrecedesOrEquals(slot_xmin, *xid)) + || + (TransactionIdIsValid(slot_catalog_xmin) && TransactionIdPrecedesOrEquals(slot_catalog_xmin, *xid)) + )) + )) + || + /* slot has been invalidated (obsolete LSN case) */ + (!xid && (XLogRecPtrIsInvalid(restart_lsn) || restart_lsn >= oldestLSN))) { SpinLockRelease(&s->mutex); if (released_lock) @@ -1292,9 +1313,16 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, { MyReplicationSlot = s; s->active_pid = MyProcPid; - s->data.invalidated_at = restart_lsn; - s->data.restart_lsn = InvalidXLogRecPtr; - + if (xid) + { + s->data.xmin = InvalidTransactionId; + s->data.catalog_xmin = InvalidTransactionId; + } + else + { + s->data.invalidated_at = restart_lsn; + s->data.restart_lsn = InvalidXLogRecPtr; + } /* Let caller know */ *invalidated = true; } @@ -1327,15 +1355,39 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, */ if (last_signaled_pid != active_pid) { - ereport(LOG, - errmsg("terminating process %d to release replication slot \"%s\"", - active_pid, NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", + *xid)); + } + else + { + ereport(LOG, + errmsg("terminating process %d because replication slot \"%s\" conflicts with recovery", + active_pid, NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + + (void) SendProcSignal(active_pid, PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, InvalidBackendId); + } + else + { + ereport(LOG, + errmsg("terminating process %d to release replication slot \"%s\"", + active_pid, NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + + (void) kill(active_pid, SIGTERM); + } - (void) kill(active_pid, SIGTERM); last_signaled_pid = active_pid; } @@ -1369,13 +1421,33 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, ReplicationSlotSave(); ReplicationSlotRelease(); - ereport(LOG, - errmsg("invalidating obsolete replication slot \"%s\"", - NameStr(slotname)), - errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", - LSN_FORMAT_ARGS(restart_lsn), - (unsigned long long) (oldestLSN - restart_lsn)), - errhint("You might need to increase max_slot_wal_keep_size.")); + if (xid) + { + pgstat_drop_replslot(s); + + if (TransactionIdIsValid(*xid)) + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("The slot conflicted with xid horizon %u.", *xid)); + } + else + { + ereport(LOG, + errmsg("invalidating slot \"%s\" because it conflicts with recovery", NameStr(slotname)), + errdetail("Logical decoding on standby requires wal_level to be at least logical on the primary server")); + } + } + else + { + ereport(LOG, + errmsg("invalidating obsolete replication slot \"%s\"", + NameStr(slotname)), + errdetail("The slot's restart_lsn %X/%X exceeds the limit by %llu bytes.", + LSN_FORMAT_ARGS(restart_lsn), + (unsigned long long) (oldestLSN - restart_lsn)), + errhint("You might need to increase max_slot_wal_keep_size.")); + } /* done with this slot for now */ break; @@ -1388,20 +1460,40 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlot *s, XLogRecPtr oldestLSN, } /* - * Mark any slot that points to an LSN older than the given segment - * as invalid; it requires WAL that's about to be removed. + * Invalidate Obsolete slots or resolve recovery conflicts with logical slots. * - * Returns true when any slot have got invalidated. + * Obsolete case (aka xid is NULL): * - * NB - this runs as part of checkpoint, so avoid raising errors if possible. + * Mark any slot that points to an LSN older than the given segment + * as invalid; it requires WAL that's about to be removed. + * invalidated is set to true when any slot have got invalidated. + * + * Logical replication slot case: + * + * When xid is valid, it means that we are about to remove rows older than xid. + * Therefore we need to invalidate slots that depend on seeing those rows. + * When xid is invalid, invalidate all logical slots. This is required when the + * master wal_level is set back to replica, so existing logical slots need to + * be invalidated. */ -bool -InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno) +void +InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid) { - XLogRecPtr oldestLSN; - bool invalidated = false; - XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + XLogRecPtr oldestLSN = InvalidXLogRecPtr; + bool logical_slot_invalidated = false; + + Assert(max_replication_slots >= 0); + + if (max_replication_slots == 0) + return; + + if (!xid) + { + Assert(invalidated); + *invalidated = false; + XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); + } restart: LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); @@ -1412,24 +1504,36 @@ restart: if (!s->in_use) continue; - if (InvalidatePossiblyObsoleteSlot(s, oldestLSN, &invalidated)) + if (xid) { - /* if the lock was released, start from scratch */ - goto restart; + /* we are only dealing with *logical* slot conflicts */ + if (!SlotIsLogical(s)) + continue; + + /* + * not the database of interest and we don't want all the + * database, skip + */ + if (s->data.database != dboid && TransactionIdIsValid(*xid)) + continue; } + + if (InvalidatePossiblyObsoleteOrConflictingLogicalSlot(s, oldestLSN, invalidated ? invalidated : &logical_slot_invalidated, xid)) + goto restart; } + LWLockRelease(ReplicationSlotControlLock); /* - * If any slots have been invalidated, recalculate the resource limits. + * If any slots have been invalidated, recalculate the required xmin + * and the required lsn (if appropriate). */ - if (invalidated) + if ((!xid && *invalidated) || (xid && logical_slot_invalidated)) { ReplicationSlotsComputeRequiredXmin(false); - ReplicationSlotsComputeRequiredLSN(); + if (!xid && *invalidated) + ReplicationSlotsComputeRequiredLSN(); } - - return invalidated; } /* diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 2f3c964824..44192bc32d 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -232,7 +232,7 @@ pg_drop_replication_slot(PG_FUNCTION_ARGS) Datum pg_get_replication_slots(PG_FUNCTION_ARGS) { -#define PG_GET_REPLICATION_SLOTS_COLS 14 +#define PG_GET_REPLICATION_SLOTS_COLS 15 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; XLogRecPtr currlsn; int slotno; @@ -404,6 +404,17 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) values[i++] = BoolGetDatum(slot_contents.data.two_phase); + if (slot_contents.data.database == InvalidOid) + nulls[i++] = true; + else + { + if (slot_contents.data.xmin == InvalidTransactionId && + slot_contents.data.catalog_xmin == InvalidTransactionId) + values[i++] = BoolGetDatum(true); + else + values[i++] = BoolGetDatum(false); + } + Assert(i == PG_GET_REPLICATION_SLOTS_COLS); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 75e8363e24..c2523c5caf 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1253,6 +1253,14 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotAcquire(cmd->slotname, true); + if (!TransactionIdIsValid(MyReplicationSlot->data.xmin) + && !TransactionIdIsValid(MyReplicationSlot->data.catalog_xmin)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot read from logical replication slot \"%s\"", + cmd->slotname), + errdetail("This slot has been invalidated because it was conflicting with recovery."))); + if (XLogRecPtrIsInvalid(MyReplicationSlot->data.restart_lsn)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 395b2cf690..c85cb5cc18 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -673,6 +673,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_SNAPSHOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT)) + RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 9a73ae67d0..db5c3333cc 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -35,6 +35,7 @@ #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" +#include "replication/slot.h" /* User-settable GUC parameters */ int vacuum_defer_cleanup_age; @@ -466,6 +467,7 @@ ResolveRecoveryConflictWithVirtualXIDs(VirtualTransactionId *waitlist, */ void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { VirtualTransactionId *backends; @@ -491,6 +493,9 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, WAIT_EVENT_RECOVERY_CONFLICT_SNAPSHOT, true); + + if (wal_level >= WAL_LEVEL_LOGICAL && isCatalogRel) + InvalidateObsoleteReplicationSlots(InvalidXLogRecPtr, NULL, locator.dbOid, &snapshotConflictHorizon); } /* @@ -499,6 +504,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, */ void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator) { /* @@ -517,7 +523,9 @@ ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHor TransactionId truncated; truncated = XidFromFullTransactionId(snapshotConflictHorizon); - ResolveRecoveryConflictWithSnapshot(truncated, locator); + ResolveRecoveryConflictWithSnapshot(truncated, + isCatalogRel, + locator); } } @@ -1478,6 +1486,9 @@ get_recovery_conflict_desc(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: reasonDesc = _("recovery conflict on snapshot"); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + reasonDesc = _("recovery conflict on replication slot"); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: reasonDesc = _("recovery conflict on buffer deadlock"); break; diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..b5f9aa285c 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -2488,6 +2488,9 @@ errdetail_recovery_conflict(void) case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: errdetail("User query might have needed to see row versions that must be removed."); break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + errdetail("User was using the logical slot that must be dropped."); + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: errdetail("User transaction caused buffer deadlock with recovery."); break; @@ -3057,6 +3060,27 @@ RecoveryConflictInterrupt(ProcSignalReason reason) case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + + /* + * For conflicts that require a logical slot to be + * invalidated, the requirement is for the signal receiver to + * release the slot, so that it could be invalidated by the + * signal sender. So for normal backends, the transaction + * should be aborted, just like for other recovery conflicts. + * But if it's walsender on standby, we don't want to go + * through the following IsTransactionOrTransactionBlock() + * check, so break here. + */ + if (am_cascading_walsender && + reason == PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT && + MyReplicationSlot && SlotIsLogical(MyReplicationSlot)) + { + RecoveryConflictPending = true; + QueryCancelPending = true; + InterruptPending = true; + break; + } /* * If we aren't in a transaction any longer then ignore. diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index 6e650ceaad..7149f22f72 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -109,6 +109,9 @@ pgstat_report_recovery_conflict(int reason) case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: dbentry->conflict_bufferpin++; break; + case PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT: + dbentry->conflict_logicalslot++; + break; case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: dbentry->conflict_startup_deadlock++; break; @@ -387,6 +390,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PGSTAT_ACCUM_DBCOUNT(conflict_tablespace); PGSTAT_ACCUM_DBCOUNT(conflict_lock); PGSTAT_ACCUM_DBCOUNT(conflict_snapshot); + PGSTAT_ACCUM_DBCOUNT(conflict_logicalslot); PGSTAT_ACCUM_DBCOUNT(conflict_bufferpin); PGSTAT_ACCUM_DBCOUNT(conflict_startup_deadlock); diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b61a12382b..1196716255 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1066,6 +1066,8 @@ PG_STAT_GET_DBENTRY_INT64(xact_commit) /* pg_stat_get_db_xact_rollback */ PG_STAT_GET_DBENTRY_INT64(xact_rollback) +/* pg_stat_get_db_conflict_logicalslot */ +PG_STAT_GET_DBENTRY_INT64(conflict_logicalslot) Datum pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS) @@ -1099,6 +1101,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS) result = (int64) (dbentry->conflict_tablespace + dbentry->conflict_lock + dbentry->conflict_snapshot + + dbentry->conflict_logicalslot + dbentry->conflict_bufferpin + dbentry->conflict_startup_deadlock); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 505595620e..abdc6e23f2 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5577,6 +5577,11 @@ proname => 'pg_stat_get_db_conflict_snapshot', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_db_conflict_snapshot' }, +{ oid => '9901', + descr => 'statistics: recovery conflicts in database caused by logical replication slot', + proname => 'pg_stat_get_db_conflict_logicalslot', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_db_conflict_logicalslot' }, { oid => '3068', descr => 'statistics: recovery conflicts in database caused by shared buffer pin', proname => 'pg_stat_get_db_conflict_bufferpin', provolatile => 's', @@ -10959,9 +10964,9 @@ proname => 'pg_get_replication_slots', prorows => '10', proisstrict => 'f', proretset => 't', provolatile => 's', prorettype => 'record', proargtypes => '', - proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase}', + proallargtypes => '{name,name,text,oid,bool,bool,int4,xid,xid,pg_lsn,pg_lsn,text,int8,bool,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn,wal_status,safe_wal_size,two_phase,conflicting}', prosrc => 'pg_get_replication_slots' }, { oid => '3786', descr => 'set up a logical replication slot', proname => 'pg_create_logical_replication_slot', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index f43fac09ed..3aa0092751 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -338,6 +338,7 @@ typedef struct PgStat_StatDBEntry PgStat_Counter conflict_tablespace; PgStat_Counter conflict_lock; PgStat_Counter conflict_snapshot; + PgStat_Counter conflict_logicalslot; PgStat_Counter conflict_bufferpin; PgStat_Counter conflict_startup_deadlock; PgStat_Counter temp_files; diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 8872c80cdf..236ebcdbdb 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -17,6 +17,8 @@ #include "storage/spin.h" #include "replication/walreceiver.h" +#define LogicalReplicationSlotIsInvalid(s) (!TransactionIdIsValid(s->data.xmin) && \ + !TransactionIdIsValid(s->data.catalog_xmin)) /* * Behaviour of replication slots, upon release or crash. * @@ -215,7 +217,7 @@ extern void ReplicationSlotsComputeRequiredLSN(void); extern XLogRecPtr ReplicationSlotsComputeLogicalRestartLSN(void); extern bool ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive); extern void ReplicationSlotsDropDBSlots(Oid dboid); -extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno); +extern void InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno, bool *invalidated, Oid dboid, TransactionId *xid); extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock); extern int ReplicationSlotIndex(ReplicationSlot *slot); extern bool ReplicationSlotName(int index, Name name); @@ -227,5 +229,6 @@ extern void CheckPointReplicationSlots(void); extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid, TransactionId xid, char *reason); #endif /* SLOT_H */ diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 905af2231b..2f52100b00 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -42,6 +42,7 @@ typedef enum PROCSIG_RECOVERY_CONFLICT_TABLESPACE, PROCSIG_RECOVERY_CONFLICT_LOCK, PROCSIG_RECOVERY_CONFLICT_SNAPSHOT, + PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT, PROCSIG_RECOVERY_CONFLICT_BUFFERPIN, PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK, diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h index 2effdea126..41f4dc372e 100644 --- a/src/include/storage/standby.h +++ b/src/include/storage/standby.h @@ -30,8 +30,10 @@ extern void InitRecoveryTransactionEnvironment(void); extern void ShutdownRecoveryTransactionEnvironment(void); extern void ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithSnapshotFullXid(FullTransactionId snapshotConflictHorizon, + bool isCatalogRel, RelFileLocator locator); extern void ResolveRecoveryConflictWithTablespace(Oid tsid); extern void ResolveRecoveryConflictWithDatabase(Oid dbid); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index e953d1f515..1b6600884e 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1472,8 +1472,9 @@ pg_replication_slots| SELECT l.slot_name, l.confirmed_flush_lsn, l.wal_status, l.safe_wal_size, - l.two_phase - FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase) + l.two_phase, + l.conflicting + FROM (pg_get_replication_slots() l(slot_name, plugin, slot_type, datoid, temporary, active, active_pid, xmin, catalog_xmin, restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size, two_phase, conflicting) LEFT JOIN pg_database d ON ((l.datoid = d.oid))); pg_roles| SELECT pg_authid.rolname, pg_authid.rolsuper, @@ -1868,7 +1869,8 @@ pg_stat_database_conflicts| SELECT oid AS datid, pg_stat_get_db_conflict_lock(oid) AS confl_lock, pg_stat_get_db_conflict_snapshot(oid) AS confl_snapshot, pg_stat_get_db_conflict_bufferpin(oid) AS confl_bufferpin, - pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock + pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock, + pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot FROM pg_database d; pg_stat_gssapi| SELECT pid, gss_auth AS gss_authenticated, -- 2.34.1 [text/plain] v52-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch (76.2K, ../../[email protected]/7-v52-0001-Add-info-in-WAL-records-in-preparation-for-logic.patch) download | inline diff: From 9e62983cc174a4f645b47d8716dce5750e7a24f4 Mon Sep 17 00:00:00 2001 From: bdrouvotAWS <[email protected]> Date: Tue, 7 Feb 2023 08:55:19 +0000 Subject: [PATCH v52 1/6] Add info in WAL records in preparation for logical slot conflict handling. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignement. It's not needed on the others struct where isCatalogRel has been added. Author: Andres Freund (in an older version), Amit Khandekar, Bertrand Drouvot Reviewed-By: Bertrand Drouvot, Andres Freund, Robert Haas, Fabrizio de Royes Mello, Melanie Plageman --- contrib/amcheck/verify_nbtree.c | 15 +-- src/backend/access/gist/gist.c | 5 +- src/backend/access/gist/gistbuild.c | 2 +- src/backend/access/gist/gistutil.c | 4 +- src/backend/access/gist/gistxlog.c | 17 ++-- src/backend/access/hash/hash_xlog.c | 12 +-- src/backend/access/hash/hashinsert.c | 1 + src/backend/access/heap/heapam.c | 5 +- src/backend/access/heap/heapam_handler.c | 9 +- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/vacuumlazy.c | 2 + src/backend/access/heap/visibilitymap.c | 3 +- src/backend/access/nbtree/nbtinsert.c | 91 +++++++++-------- src/backend/access/nbtree/nbtpage.c | 111 +++++++++++---------- src/backend/access/nbtree/nbtree.c | 4 +- src/backend/access/nbtree/nbtsearch.c | 50 ++++++---- src/backend/access/nbtree/nbtsort.c | 2 +- src/backend/access/nbtree/nbtutils.c | 7 +- src/backend/access/spgist/spgvacuum.c | 9 +- src/backend/catalog/index.c | 1 + src/backend/commands/analyze.c | 1 + src/backend/commands/vacuumparallel.c | 6 ++ src/backend/optimizer/util/plancat.c | 2 +- src/backend/utils/sort/tuplesortvariants.c | 5 +- src/include/access/genam.h | 1 + src/include/access/gist_private.h | 7 +- src/include/access/gistxlog.h | 11 +- src/include/access/hash_xlog.h | 8 +- src/include/access/heapam_xlog.h | 10 +- src/include/access/nbtree.h | 37 ++++--- src/include/access/nbtxlog.h | 8 +- src/include/access/spgxlog.h | 2 + src/include/access/visibilitymapdefs.h | 10 +- src/include/utils/rel.h | 1 + src/include/utils/tuplesort.h | 4 +- 35 files changed, 263 insertions(+), 201 deletions(-) 3.3% contrib/amcheck/ 4.7% src/backend/access/gist/ 4.1% src/backend/access/heap/ 59.0% src/backend/access/nbtree/ 3.7% src/backend/access/ 22.0% src/include/access/ diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 257cff671b..eb280d4893 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -183,6 +183,7 @@ static inline bool invariant_l_nontarget_offset(BtreeCheckState *state, OffsetNumber upperbound); static Page palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum); static inline BTScanInsert bt_mkscankey_pivotsearch(Relation rel, + Relation heaprel, IndexTuple itup); static ItemId PageGetItemIdCareful(BtreeCheckState *state, BlockNumber block, Page page, OffsetNumber offset); @@ -331,7 +332,7 @@ bt_index_check_internal(Oid indrelid, bool parentcheck, bool heapallindexed, RelationGetRelationName(indrel)))); /* Extract metadata from metapage, and sanitize it in passing */ - _bt_metaversion(indrel, &heapkeyspace, &allequalimage); + _bt_metaversion(indrel, heaprel, &heapkeyspace, &allequalimage); if (allequalimage && !heapkeyspace) ereport(ERROR, (errcode(ERRCODE_INDEX_CORRUPTED), @@ -1258,7 +1259,7 @@ bt_target_page_check(BtreeCheckState *state) } /* Build insertion scankey for current page offset */ - skey = bt_mkscankey_pivotsearch(state->rel, itup); + skey = bt_mkscankey_pivotsearch(state->rel, state->heaprel, itup); /* * Make sure tuple size does not exceed the relevant BTREE_VERSION @@ -1768,7 +1769,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * memory remaining allocated. */ firstitup = (IndexTuple) PageGetItem(rightpage, rightitem); - return bt_mkscankey_pivotsearch(state->rel, firstitup); + return bt_mkscankey_pivotsearch(state->rel, state->heaprel, firstitup); } /* @@ -2681,7 +2682,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) Buffer lbuf; bool exists; - key = _bt_mkscankey(state->rel, itup); + key = _bt_mkscankey(state->rel, state->heaprel, itup); Assert(key->heapkeyspace && key->scantid != NULL); /* @@ -2694,7 +2695,7 @@ bt_rootdescend(BtreeCheckState *state, IndexTuple itup) */ Assert(state->readonly && state->rootdescend); exists = false; - stack = _bt_search(state->rel, key, &lbuf, BT_READ, NULL); + stack = _bt_search(state->rel, state->heaprel, key, &lbuf, BT_READ, NULL); if (BufferIsValid(lbuf)) { @@ -3133,11 +3134,11 @@ palloc_btree_page(BtreeCheckState *state, BlockNumber blocknum) * the scankey is greater. */ static inline BTScanInsert -bt_mkscankey_pivotsearch(Relation rel, IndexTuple itup) +bt_mkscankey_pivotsearch(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert skey; - skey = _bt_mkscankey(rel, itup); + skey = _bt_mkscankey(rel, heaprel, itup); skey->pivotsearch = true; return skey; diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c index ba394f08f6..3ac68ec3b4 100644 --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -348,7 +348,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (; ptr; ptr = ptr->next) { /* Allocate new page */ - ptr->buffer = gistNewBuffer(rel); + ptr->buffer = gistNewBuffer(rel, heapRel); GISTInitBuffer(ptr->buffer, (is_leaf) ? F_LEAF : 0); ptr->page = BufferGetPage(ptr->buffer); ptr->block.blkno = BufferGetBlockNumber(ptr->buffer); @@ -1694,7 +1694,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel) recptr = gistXLogDelete(buffer, deletable, ndeletable, - snapshotConflictHorizon); + snapshotConflictHorizon, + heapRel); PageSetLSN(page, recptr); } diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c index 7a6d93bb87..1f044840d4 100644 --- a/src/backend/access/gist/gistbuild.c +++ b/src/backend/access/gist/gistbuild.c @@ -298,7 +298,7 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo) Page page; /* initialize the root page */ - buffer = gistNewBuffer(index); + buffer = gistNewBuffer(index, heap); Assert(BufferGetBlockNumber(buffer) == GIST_ROOT_BLKNO); page = BufferGetPage(buffer); diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index b4d843a0ff..a607464b97 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -821,7 +821,7 @@ gistcheckpage(Relation rel, Buffer buf) * Caller is responsible for initializing the page by calling GISTInitBuffer */ Buffer -gistNewBuffer(Relation r) +gistNewBuffer(Relation r, Relation heaprel) { Buffer buffer; bool needLock; @@ -865,7 +865,7 @@ gistNewBuffer(Relation r) * page's deleteXid. */ if (XLogStandbyInfoActive() && RelationNeedsWAL(r)) - gistXLogPageReuse(r, blkno, GistPageGetDeleteXid(page)); + gistXLogPageReuse(r, heaprel, blkno, GistPageGetDeleteXid(page)); return buffer; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index f65864254a..b7678f3c14 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -177,6 +177,7 @@ gistRedoDeleteRecord(XLogReaderState *record) gistxlogDelete *xldata = (gistxlogDelete *) XLogRecGetData(record); Buffer buffer; Page page; + OffsetNumber *toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -203,14 +204,7 @@ gistRedoDeleteRecord(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfGistxlogDelete) - { - OffsetNumber *todelete; - - todelete = (OffsetNumber *) ((char *) xldata + SizeOfGistxlogDelete); - - PageIndexMultiDelete(page, todelete, xldata->ntodelete); - } + PageIndexMultiDelete(page, toDelete, xldata->ntodelete); GistClearPageHasGarbage(page); GistMarkTuplesDeleted(page); @@ -597,7 +591,8 @@ gistXLogAssignLSN(void) * Write XLOG record about reuse of a deleted page. */ void -gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) +gistXLogPageReuse(Relation rel, Relation heaprel, + BlockNumber blkno, FullTransactionId deleteXid) { gistxlogPageReuse xlrec_reuse; @@ -608,6 +603,7 @@ gistXLogPageReuse(Relation rel, BlockNumber blkno, FullTransactionId deleteXid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = deleteXid; @@ -672,11 +668,12 @@ gistXLogUpdate(Buffer buffer, */ XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, int ntodelete, - TransactionId snapshotConflictHorizon) + TransactionId snapshotConflictHorizon, Relation heaprel) { gistxlogDelete xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntodelete = ntodelete; diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index f38b42efb9..08ceb91288 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -980,8 +980,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) Page page; XLogRedoAction action; HashPageOpaque pageopaque; + OffsetNumber *toDelete; xldata = (xl_hash_vacuum_one_page *) XLogRecGetData(record); + toDelete = xldata->offsets; /* * If we have any conflict processing to do, it must happen before we @@ -1010,15 +1012,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) { page = (Page) BufferGetPage(buffer); - if (XLogRecGetDataLen(record) > SizeOfHashVacuumOnePage) - { - OffsetNumber *unused; - - unused = (OffsetNumber *) ((char *) xldata + SizeOfHashVacuumOnePage); - - PageIndexMultiDelete(page, unused, xldata->ntuples); - } - + PageIndexMultiDelete(page, toDelete, xldata->ntuples); /* * Mark the page as not containing any LP_DEAD items. See comments in * _hash_vacuum_one_page() for details. diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c index a604e31891..22656b24e2 100644 --- a/src/backend/access/hash/hashinsert.c +++ b/src/backend/access/hash/hashinsert.c @@ -432,6 +432,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf) xl_hash_vacuum_one_page xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(hrel); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.ntuples = ndeletable; diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 4f50e0dd34..6c36b3a326 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -6658,6 +6658,7 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer, nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets); xlrec.snapshotConflictHorizon = snapshotConflictHorizon; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel); xlrec.nplans = nplans; XLogBeginInsert(); @@ -8228,7 +8229,7 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate) * update the heap page's LSN. */ XLogRecPtr -log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, +log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags) { xl_heap_visible xlrec; @@ -8240,6 +8241,8 @@ log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, Buffer vm_buffer, xlrec.snapshotConflictHorizon = snapshotConflictHorizon; xlrec.flags = vmflags; + if (RelationIsAccessibleInLogicalDecoding(rel)) + xlrec.flags |= VISIBILITYMAP_IS_CATALOG_REL; XLogBeginInsert(); XLogRegisterData((char *) &xlrec, SizeOfHeapVisible); diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index c4b1916d36..392c6e659c 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -720,9 +720,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, *multi_cutoff); - /* Set up sorting if wanted */ + /* + * Set up sorting if wanted. NewHeap is being passed to + * tuplesort_begin_cluster(), it could have been OldHeap too. It does not + * really matter, as the goal is to have a heap relation being passed to + * _bt_log_reuse_page() (which should not be called from this code path). + */ if (use_sort) - tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, + tuplesort = tuplesort_begin_cluster(oldTupDesc, OldIndex, NewHeap, maintenance_work_mem, NULL, TUPLESORT_NONE); else diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 4e65cbcadf..3f0342351f 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -418,6 +418,7 @@ heap_page_prune(Relation relation, Buffer buffer, xl_heap_prune xlrec; XLogRecPtr recptr; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation); xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon; xlrec.nredirected = prstate.nredirected; xlrec.ndead = prstate.ndead; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8f14cf85f3..ae628d747d 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -2710,6 +2710,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. @@ -2759,6 +2760,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat, ivinfo.num_heap_tuples = reltuples; ivinfo.strategy = vacrel->bstrategy; + ivinfo.heaprel = vacrel->rel; /* * Update error traceback information. diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c index 74ff01bb17..d1ba859851 100644 --- a/src/backend/access/heap/visibilitymap.c +++ b/src/backend/access/heap/visibilitymap.c @@ -288,8 +288,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf, if (XLogRecPtrIsInvalid(recptr)) { Assert(!InRecovery); - recptr = log_heap_visible(rel->rd_locator, heapBuf, vmBuf, - cutoff_xid, flags); + recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags); /* * If data checksums are enabled (or wal_log_hints=on), we diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index f4c1a974ef..8c6e867c61 100644 --- a/src/backend/access/nbtree/nbtinsert.c +++ b/src/backend/access/nbtree/nbtinsert.c @@ -30,7 +30,8 @@ #define BTREE_FASTPATH_MIN_LEVEL 2 -static BTStack _bt_search_insert(Relation rel, BTInsertState insertstate); +static BTStack _bt_search_insert(Relation rel, Relation heaprel, + BTInsertState insertstate); static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel, IndexUniqueCheck checkUnique, bool *is_unique, @@ -41,8 +42,9 @@ static OffsetNumber _bt_findinsertloc(Relation rel, bool indexUnchanged, BTStack stack, Relation heapRel); -static void _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack); -static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, +static void _bt_stepright(Relation rel, Relation heaprel, + BTInsertState insertstate, BTStack stack); +static void _bt_insertonpg(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, BTStack stack, @@ -51,13 +53,13 @@ static void _bt_insertonpg(Relation rel, BTScanInsert itup_key, OffsetNumber newitemoff, int postingoff, bool split_only_page); -static Buffer _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, - Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, - IndexTuple newitem, IndexTuple orignewitem, +static Buffer _bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, + Buffer buf, Buffer cbuf, OffsetNumber newitemoff, + Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff); -static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf, - BTStack stack, bool isroot, bool isonly); -static Buffer _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf); +static void _bt_insert_parent(Relation rel, Relation heaprel, Buffer buf, + Buffer rbuf, BTStack stack, bool isroot, bool isonly); +static Buffer _bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf); static inline bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup, OffsetNumber itup_off, bool newfirstdataitem); static void _bt_delete_or_dedup_one_page(Relation rel, Relation heapRel, @@ -108,7 +110,7 @@ _bt_doinsert(Relation rel, IndexTuple itup, bool checkingunique = (checkUnique != UNIQUE_CHECK_NO); /* we need an insertion scan key to do our search, so build one */ - itup_key = _bt_mkscankey(rel, itup); + itup_key = _bt_mkscankey(rel, heapRel, itup); if (checkingunique) { @@ -162,7 +164,7 @@ search: * searching from the root page. insertstate.buf will hold a buffer that * is locked in exclusive mode afterwards. */ - stack = _bt_search_insert(rel, &insertstate); + stack = _bt_search_insert(rel, heapRel, &insertstate); /* * checkingunique inserts are not allowed to go ahead when two tuples with @@ -255,8 +257,8 @@ search: */ newitemoff = _bt_findinsertloc(rel, &insertstate, checkingunique, indexUnchanged, stack, heapRel); - _bt_insertonpg(rel, itup_key, insertstate.buf, InvalidBuffer, stack, - itup, insertstate.itemsz, newitemoff, + _bt_insertonpg(rel, heapRel, itup_key, insertstate.buf, InvalidBuffer, + stack, itup, insertstate.itemsz, newitemoff, insertstate.postingoff, false); } else @@ -312,7 +314,7 @@ search: * since each per-backend cache won't stay valid for long. */ static BTStack -_bt_search_insert(Relation rel, BTInsertState insertstate) +_bt_search_insert(Relation rel, Relation heaprel, BTInsertState insertstate) { Assert(insertstate->buf == InvalidBuffer); Assert(!insertstate->bounds_valid); @@ -375,8 +377,8 @@ _bt_search_insert(Relation rel, BTInsertState insertstate) } /* Cannot use optimization -- descend tree, return proper descent stack */ - return _bt_search(rel, insertstate->itup_key, &insertstate->buf, BT_WRITE, - NULL); + return _bt_search(rel, heaprel, insertstate->itup_key, &insertstate->buf, + BT_WRITE, NULL); } /* @@ -885,7 +887,7 @@ _bt_findinsertloc(Relation rel, _bt_compare(rel, itup_key, page, P_HIKEY) <= 0) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -969,7 +971,7 @@ _bt_findinsertloc(Relation rel, pg_prng_uint32(&pg_global_prng_state) <= (PG_UINT32_MAX / 100)) break; - _bt_stepright(rel, insertstate, stack); + _bt_stepright(rel, heapRel, insertstate, stack); /* Update local state after stepping right */ page = BufferGetPage(insertstate->buf); opaque = BTPageGetOpaque(page); @@ -1022,7 +1024,7 @@ _bt_findinsertloc(Relation rel, * indexes. */ static void -_bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) +_bt_stepright(Relation rel, Relation heaprel, BTInsertState insertstate, BTStack stack) { Page page; BTPageOpaque opaque; @@ -1048,7 +1050,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, rbuf, stack); + _bt_finish_split(rel, heaprel, rbuf, stack); rbuf = InvalidBuffer; continue; } @@ -1099,6 +1101,7 @@ _bt_stepright(Relation rel, BTInsertState insertstate, BTStack stack) */ static void _bt_insertonpg(Relation rel, + Relation heaprel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, @@ -1209,8 +1212,8 @@ _bt_insertonpg(Relation rel, Assert(!split_only_page); /* split the buffer into left and right halves */ - rbuf = _bt_split(rel, itup_key, buf, cbuf, newitemoff, itemsz, itup, - origitup, nposting, postingoff); + rbuf = _bt_split(rel, heaprel, itup_key, buf, cbuf, newitemoff, itemsz, + itup, origitup, nposting, postingoff); PredicateLockPageSplit(rel, BufferGetBlockNumber(buf), BufferGetBlockNumber(rbuf)); @@ -1233,7 +1236,7 @@ _bt_insertonpg(Relation rel, * page. *---------- */ - _bt_insert_parent(rel, buf, rbuf, stack, isroot, isonly); + _bt_insert_parent(rel, heaprel, buf, rbuf, stack, isroot, isonly); } else { @@ -1254,7 +1257,7 @@ _bt_insertonpg(Relation rel, Assert(!isleaf); Assert(BufferIsValid(cbuf)); - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -1418,7 +1421,7 @@ _bt_insertonpg(Relation rel, * call _bt_getrootheight while holding a buffer lock. */ if (BlockNumberIsValid(blockcache) && - _bt_getrootheight(rel) >= BTREE_FASTPATH_MIN_LEVEL) + _bt_getrootheight(rel, heaprel) >= BTREE_FASTPATH_MIN_LEVEL) RelationSetTargetBlock(rel, blockcache); } @@ -1459,8 +1462,8 @@ _bt_insertonpg(Relation rel, * The pin and lock on buf are maintained. */ static Buffer -_bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, - OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, +_bt_split(Relation rel, Relation heaprel, BTScanInsert itup_key, Buffer buf, + Buffer cbuf, OffsetNumber newitemoff, Size newitemsz, IndexTuple newitem, IndexTuple orignewitem, IndexTuple nposting, uint16 postingoff) { Buffer rbuf; @@ -1712,7 +1715,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, * way because it avoids an unnecessary PANIC when either origpage or its * existing sibling page are corrupt. */ - rbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rightpage = BufferGetPage(rbuf); rightpagenumber = BufferGetBlockNumber(rbuf); /* rightpage was initialized by _bt_getbuf */ @@ -1885,7 +1888,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ if (!isrightmost) { - sbuf = _bt_getbuf(rel, oopaque->btpo_next, BT_WRITE); + sbuf = _bt_getbuf(rel, heaprel, oopaque->btpo_next, BT_WRITE); spage = BufferGetPage(sbuf); sopaque = BTPageGetOpaque(spage); if (sopaque->btpo_prev != origpagenumber) @@ -2092,6 +2095,7 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf, */ static void _bt_insert_parent(Relation rel, + Relation heaprel, Buffer buf, Buffer rbuf, BTStack stack, @@ -2118,7 +2122,7 @@ _bt_insert_parent(Relation rel, Assert(stack == NULL); Assert(isonly); /* create a new root node and update the metapage */ - rootbuf = _bt_newroot(rel, buf, rbuf); + rootbuf = _bt_newroot(rel, heaprel, buf, rbuf); /* release the split buffers */ _bt_relbuf(rel, rootbuf); _bt_relbuf(rel, rbuf); @@ -2157,7 +2161,8 @@ _bt_insert_parent(Relation rel, BlockNumberIsValid(RelationGetTargetBlock(rel)))); /* Find the leftmost page at the next level up */ - pbuf = _bt_get_endpoint(rel, opaque->btpo_level + 1, false, NULL); + pbuf = _bt_get_endpoint(rel, heaprel, opaque->btpo_level + 1, false, + NULL); /* Set up a phony stack entry pointing there */ stack = &fakestack; stack->bts_blkno = BufferGetBlockNumber(pbuf); @@ -2183,7 +2188,7 @@ _bt_insert_parent(Relation rel, * new downlink will be inserted at the correct offset. Even buf's * parent may have changed. */ - pbuf = _bt_getstackbuf(rel, stack, bknum); + pbuf = _bt_getstackbuf(rel, heaprel, stack, bknum); /* * Unlock the right child. The left child will be unlocked in @@ -2207,7 +2212,7 @@ _bt_insert_parent(Relation rel, RelationGetRelationName(rel), bknum, rbknum))); /* Recursively insert into the parent */ - _bt_insertonpg(rel, NULL, pbuf, buf, stack->bts_parent, + _bt_insertonpg(rel, heaprel, NULL, pbuf, buf, stack->bts_parent, new_item, MAXALIGN(IndexTupleSize(new_item)), stack->bts_offset + 1, 0, isonly); @@ -2227,7 +2232,7 @@ _bt_insert_parent(Relation rel, * and unpinned. */ void -_bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) +_bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, BTStack stack) { Page lpage = BufferGetPage(lbuf); BTPageOpaque lpageop = BTPageGetOpaque(lpage); @@ -2240,7 +2245,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) Assert(P_INCOMPLETE_SPLIT(lpageop)); /* Lock right sibling, the one missing the downlink */ - rbuf = _bt_getbuf(rel, lpageop->btpo_next, BT_WRITE); + rbuf = _bt_getbuf(rel, heaprel, lpageop->btpo_next, BT_WRITE); rpage = BufferGetPage(rbuf); rpageop = BTPageGetOpaque(rpage); @@ -2252,7 +2257,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) BTMetaPageData *metad; /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2269,7 +2274,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) elog(DEBUG1, "finishing incomplete split of %u/%u", BufferGetBlockNumber(lbuf), BufferGetBlockNumber(rbuf)); - _bt_insert_parent(rel, lbuf, rbuf, stack, wasroot, wasonly); + _bt_insert_parent(rel, heaprel, lbuf, rbuf, stack, wasroot, wasonly); } /* @@ -2304,7 +2309,7 @@ _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack) * offset number bts_offset + 1. */ Buffer -_bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) +_bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, BlockNumber child) { BlockNumber blkno; OffsetNumber start; @@ -2318,13 +2323,13 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) Page page; BTPageOpaque opaque; - buf = _bt_getbuf(rel, blkno, BT_WRITE); + buf = _bt_getbuf(rel, heaprel, blkno, BT_WRITE); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); if (P_INCOMPLETE_SPLIT(opaque)) { - _bt_finish_split(rel, buf, stack->bts_parent); + _bt_finish_split(rel, heaprel, buf, stack->bts_parent); continue; } @@ -2428,7 +2433,7 @@ _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child) * lbuf, rbuf & rootbuf. */ static Buffer -_bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) +_bt_newroot(Relation rel, Relation heaprel, Buffer lbuf, Buffer rbuf) { Buffer rootbuf; Page lpage, @@ -2454,12 +2459,12 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf) lopaque = BTPageGetOpaque(lpage); /* get a new root page */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootpage = BufferGetPage(rootbuf); rootblknum = BufferGetBlockNumber(rootbuf); /* acquire lock on the metapage */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 3feee28d19..151ad37a54 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -38,25 +38,24 @@ #include "utils/snapmgr.h" static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf); -static void _bt_log_reuse_page(Relation rel, BlockNumber blkno, +static void _bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId safexid); -static void _bt_delitems_delete(Relation rel, Buffer buf, +static void _bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable); static char *_bt_delitems_update(BTVacuumPosting *updatable, int nupdatable, OffsetNumber *updatedoffsets, Size *updatedbuflen, bool needswal); -static bool _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, - BTStack stack); +static bool _bt_mark_page_halfdead(Relation rel, Relation heaprel, + Buffer leafbuf, BTStack stack); static bool _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, bool *rightsib_empty, BTVacState *vstate); -static bool _bt_lock_subtree_parent(Relation rel, BlockNumber child, - BTStack stack, - Buffer *subtreeparent, - OffsetNumber *poffset, +static bool _bt_lock_subtree_parent(Relation rel, Relation heaprel, + BlockNumber child, BTStack stack, + Buffer *subtreeparent, OffsetNumber *poffset, BlockNumber *topparent, BlockNumber *topparentrightsib); static void _bt_pendingfsm_add(BTVacState *vstate, BlockNumber target, @@ -178,7 +177,7 @@ _bt_getmeta(Relation rel, Buffer metabuf) * index tuples needed to be deleted. */ bool -_bt_vacuum_needs_cleanup(Relation rel) +_bt_vacuum_needs_cleanup(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -191,7 +190,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * * Note that we deliberately avoid using cached version of metapage here. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); btm_version = metad->btm_version; @@ -231,7 +230,7 @@ _bt_vacuum_needs_cleanup(Relation rel) * finalized. */ void -_bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) +_bt_set_cleanup_info(Relation rel, Relation heaprel, BlockNumber num_delpages) { Buffer metabuf; Page metapg; @@ -255,7 +254,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * no longer used as of PostgreSQL 14. We set it to -1.0 on rewrite, just * to be consistent. */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -340,7 +339,7 @@ _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages) * The metadata page is not locked or pinned on exit. */ Buffer -_bt_getroot(Relation rel, int access) +_bt_getroot(Relation rel, Relation heaprel, int access) { Buffer metabuf; Buffer rootbuf; @@ -370,7 +369,7 @@ _bt_getroot(Relation rel, int access) Assert(rootblkno != P_NONE); rootlevel = metad->btm_fastlevel; - rootbuf = _bt_getbuf(rel, rootblkno, BT_READ); + rootbuf = _bt_getbuf(rel, heaprel, rootblkno, BT_READ); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -396,7 +395,7 @@ _bt_getroot(Relation rel, int access) rel->rd_amcache = NULL; } - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* if no root page initialized yet, do it */ @@ -429,7 +428,7 @@ _bt_getroot(Relation rel, int access) * to optimize this case.) */ _bt_relbuf(rel, metabuf); - return _bt_getroot(rel, access); + return _bt_getroot(rel, heaprel, access); } /* @@ -437,7 +436,7 @@ _bt_getroot(Relation rel, int access) * the new root page. Since this is the first page in the tree, it's * a leaf as well as the root. */ - rootbuf = _bt_getbuf(rel, P_NEW, BT_WRITE); + rootbuf = _bt_getbuf(rel, heaprel, P_NEW, BT_WRITE); rootblkno = BufferGetBlockNumber(rootbuf); rootpage = BufferGetPage(rootbuf); rootopaque = BTPageGetOpaque(rootpage); @@ -574,7 +573,7 @@ _bt_getroot(Relation rel, int access) * moving to the root --- that'd deadlock against any concurrent root split.) */ Buffer -_bt_gettrueroot(Relation rel) +_bt_gettrueroot(Relation rel, Relation heaprel) { Buffer metabuf; Page metapg; @@ -596,7 +595,7 @@ _bt_gettrueroot(Relation rel) pfree(rel->rd_amcache); rel->rd_amcache = NULL; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metapg = BufferGetPage(metabuf); metaopaque = BTPageGetOpaque(metapg); metad = BTPageGetMeta(metapg); @@ -669,7 +668,7 @@ _bt_gettrueroot(Relation rel) * about updating previously cached data. */ int -_bt_getrootheight(Relation rel) +_bt_getrootheight(Relation rel, Relation heaprel) { BTMetaPageData *metad; @@ -677,7 +676,7 @@ _bt_getrootheight(Relation rel) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -733,7 +732,7 @@ _bt_getrootheight(Relation rel) * pg_upgrade'd from Postgres 12. */ void -_bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) +_bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage) { BTMetaPageData *metad; @@ -741,7 +740,7 @@ _bt_metaversion(Relation rel, bool *heapkeyspace, bool *allequalimage) { Buffer metabuf; - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ); + metabuf = _bt_getbuf(rel, heaprel, BTREE_METAPAGE, BT_READ); metad = _bt_getmeta(rel, metabuf); /* @@ -825,7 +824,8 @@ _bt_checkpage(Relation rel, Buffer buf) * Log the reuse of a page from the FSM. */ static void -_bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) +_bt_log_reuse_page(Relation rel, Relation heaprel, BlockNumber blkno, + FullTransactionId safexid) { xl_btree_reuse_page xlrec_reuse; @@ -836,6 +836,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) */ /* XLOG stuff */ + xlrec_reuse.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_reuse.locator = rel->rd_locator; xlrec_reuse.block = blkno; xlrec_reuse.snapshotConflictHorizon = safexid; @@ -868,7 +869,7 @@ _bt_log_reuse_page(Relation rel, BlockNumber blkno, FullTransactionId safexid) * as _bt_lockbuf(). */ Buffer -_bt_getbuf(Relation rel, BlockNumber blkno, int access) +_bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, int access) { Buffer buf; @@ -943,7 +944,7 @@ _bt_getbuf(Relation rel, BlockNumber blkno, int access) * than safexid value */ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel)) - _bt_log_reuse_page(rel, blkno, + _bt_log_reuse_page(rel, heaprel, blkno, BTPageGetDeleteXid(page)); /* Okay to use page. Re-initialize and return it. */ @@ -1293,7 +1294,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf, * clear page's VACUUM cycle ID. */ static void -_bt_delitems_delete(Relation rel, Buffer buf, +_bt_delitems_delete(Relation rel, Relation heaprel, Buffer buf, TransactionId snapshotConflictHorizon, OffsetNumber *deletable, int ndeletable, BTVacuumPosting *updatable, int nupdatable) @@ -1358,6 +1359,7 @@ _bt_delitems_delete(Relation rel, Buffer buf, XLogRecPtr recptr; xl_btree_delete xlrec_delete; + xlrec_delete.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec_delete.snapshotConflictHorizon = snapshotConflictHorizon; xlrec_delete.ndeleted = ndeletable; xlrec_delete.nupdated = nupdatable; @@ -1684,8 +1686,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, } /* Physically delete tuples (or TIDs) using deletable (or updatable) */ - _bt_delitems_delete(rel, buf, snapshotConflictHorizon, - deletable, ndeletable, updatable, nupdatable); + _bt_delitems_delete(rel, heapRel, buf, snapshotConflictHorizon, deletable, + ndeletable, updatable, nupdatable); /* be tidy */ for (int i = 0; i < nupdatable; i++) @@ -1706,7 +1708,8 @@ _bt_delitems_delete_check(Relation rel, Buffer buf, Relation heapRel, * same level must always be locked left to right to avoid deadlocks. */ static bool -_bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) +_bt_leftsib_splitflag(Relation rel, Relation heaprel, BlockNumber leftsib, + BlockNumber target) { Buffer buf; Page page; @@ -1717,7 +1720,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) if (leftsib == P_NONE) return false; - buf = _bt_getbuf(rel, leftsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leftsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1763,7 +1766,7 @@ _bt_leftsib_splitflag(Relation rel, BlockNumber leftsib, BlockNumber target) * to-be-deleted subtree.) */ static bool -_bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) +_bt_rightsib_halfdeadflag(Relation rel, Relation heaprel, BlockNumber leafrightsib) { Buffer buf; Page page; @@ -1772,7 +1775,7 @@ _bt_rightsib_halfdeadflag(Relation rel, BlockNumber leafrightsib) Assert(leafrightsib != P_NONE); - buf = _bt_getbuf(rel, leafrightsib, BT_READ); + buf = _bt_getbuf(rel, heaprel, leafrightsib, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); @@ -1961,17 +1964,18 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * marked with INCOMPLETE_SPLIT flag before proceeding */ Assert(leafblkno == scanblkno); - if (_bt_leftsib_splitflag(rel, leftsib, leafblkno)) + if (_bt_leftsib_splitflag(rel, vstate->info->heaprel, leftsib, leafblkno)) { ReleaseBuffer(leafbuf); return; } /* we need an insertion scan key for the search, so build one */ - itup_key = _bt_mkscankey(rel, targetkey); + itup_key = _bt_mkscankey(rel, vstate->info->heaprel, targetkey); /* find the leftmost leaf page with matching pivot/high key */ itup_key->pivotsearch = true; - stack = _bt_search(rel, itup_key, &sleafbuf, BT_READ, NULL); + stack = _bt_search(rel, vstate->info->heaprel, itup_key, + &sleafbuf, BT_READ, NULL); /* won't need a second lock or pin on leafbuf */ _bt_relbuf(rel, sleafbuf); @@ -2002,7 +2006,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * leafbuf page half-dead. */ Assert(P_ISLEAF(opaque) && !P_IGNORE(opaque)); - if (!_bt_mark_page_halfdead(rel, leafbuf, stack)) + if (!_bt_mark_page_halfdead(rel, vstate->info->heaprel, leafbuf, stack)) { _bt_relbuf(rel, leafbuf); return; @@ -2065,7 +2069,7 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) if (!rightsib_empty) break; - leafbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + leafbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); } } @@ -2084,7 +2088,8 @@ _bt_pagedel(Relation rel, Buffer leafbuf, BTVacState *vstate) * successfully. */ static bool -_bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) +_bt_mark_page_halfdead(Relation rel, Relation heaprel, Buffer leafbuf, + BTStack stack) { BlockNumber leafblkno; BlockNumber leafrightsib; @@ -2119,7 +2124,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) * delete the downlink. It would fail the "right sibling of target page * is also the next child in parent page" cross-check below. */ - if (_bt_rightsib_halfdeadflag(rel, leafrightsib)) + if (_bt_rightsib_halfdeadflag(rel, heaprel, leafrightsib)) { elog(DEBUG1, "could not delete page %u because its right sibling %u is half-dead", leafblkno, leafrightsib); @@ -2143,7 +2148,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack) */ topparent = leafblkno; topparentrightsib = leafrightsib; - if (!_bt_lock_subtree_parent(rel, leafblkno, stack, + if (!_bt_lock_subtree_parent(rel, heaprel, leafblkno, stack, &subtreeparent, &poffset, &topparent, &topparentrightsib)) return false; @@ -2363,7 +2368,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, Assert(target != leafblkno); /* Fetch the block number of the target's left sibling */ - buf = _bt_getbuf(rel, target, BT_READ); + buf = _bt_getbuf(rel, vstate->info->heaprel, target, BT_READ); page = BufferGetPage(buf); opaque = BTPageGetOpaque(page); leftsib = opaque->btpo_prev; @@ -2390,7 +2395,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, _bt_lockbuf(rel, leafbuf, BT_WRITE); if (leftsib != P_NONE) { - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); while (P_ISDELETED(opaque) || opaque->btpo_next != target) @@ -2440,7 +2445,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, CHECK_FOR_INTERRUPTS(); /* step right one page */ - lbuf = _bt_getbuf(rel, leftsib, BT_WRITE); + lbuf = _bt_getbuf(rel, vstate->info->heaprel, leftsib, BT_WRITE); page = BufferGetPage(lbuf); opaque = BTPageGetOpaque(page); } @@ -2504,7 +2509,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * And next write-lock the (current) right sibling. */ rightsib = opaque->btpo_next; - rbuf = _bt_getbuf(rel, rightsib, BT_WRITE); + rbuf = _bt_getbuf(rel, vstate->info->heaprel, rightsib, BT_WRITE); page = BufferGetPage(rbuf); opaque = BTPageGetOpaque(page); if (opaque->btpo_prev != target) @@ -2533,7 +2538,8 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, if (P_RIGHTMOST(opaque)) { /* rightsib will be the only one left on the level */ - metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_WRITE); + metabuf = _bt_getbuf(rel, vstate->info->heaprel, BTREE_METAPAGE, + BT_WRITE); metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); @@ -2773,9 +2779,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * parent block in the leafbuf page using BTreeTupleSetTopParent()). */ static bool -_bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, - Buffer *subtreeparent, OffsetNumber *poffset, - BlockNumber *topparent, BlockNumber *topparentrightsib) +_bt_lock_subtree_parent(Relation rel, Relation heaprel, BlockNumber child, + BTStack stack, Buffer *subtreeparent, + OffsetNumber *poffset, BlockNumber *topparent, + BlockNumber *topparentrightsib) { BlockNumber parent, leftsibparent; @@ -2789,7 +2796,7 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * Locate the pivot tuple whose downlink points to "child". Write lock * the parent page itself. */ - pbuf = _bt_getstackbuf(rel, stack, child); + pbuf = _bt_getstackbuf(rel, heaprel, stack, child); if (pbuf == InvalidBuffer) { /* @@ -2889,11 +2896,11 @@ _bt_lock_subtree_parent(Relation rel, BlockNumber child, BTStack stack, * * Note: We deliberately avoid completing incomplete splits here. */ - if (_bt_leftsib_splitflag(rel, leftsibparent, parent)) + if (_bt_leftsib_splitflag(rel, heaprel, leftsibparent, parent)) return false; /* Recurse to examine child page's grandparent page */ - return _bt_lock_subtree_parent(rel, parent, stack->bts_parent, + return _bt_lock_subtree_parent(rel, heaprel, parent, stack->bts_parent, subtreeparent, poffset, topparent, topparentrightsib); } diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 3f7b541e9d..a213407fee 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -834,7 +834,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) if (stats == NULL) { /* Check if VACUUM operation can entirely avoid btvacuumscan() call */ - if (!_bt_vacuum_needs_cleanup(info->index)) + if (!_bt_vacuum_needs_cleanup(info->index, info->heaprel)) return NULL; /* @@ -870,7 +870,7 @@ btvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) */ Assert(stats->pages_deleted >= stats->pages_free); num_delpages = stats->pages_deleted - stats->pages_free; - _bt_set_cleanup_info(info->index, num_delpages); + _bt_set_cleanup_info(info->index, info->heaprel, num_delpages); /* * It's quite possible for us to be fooled by concurrent page splits into diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index c43c1a2830..5c728e353d 100644 --- a/src/backend/access/nbtree/nbtsearch.c +++ b/src/backend/access/nbtree/nbtsearch.c @@ -42,7 +42,8 @@ static bool _bt_steppage(IndexScanDesc scan, ScanDirection dir); static bool _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); static bool _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir); -static Buffer _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot); +static Buffer _bt_walk_left(Relation rel, Relation heaprel, Buffer buf, + Snapshot snapshot); static bool _bt_endpoint(IndexScanDesc scan, ScanDirection dir); static inline void _bt_initialize_more_data(BTScanOpaque so, ScanDirection dir); @@ -93,14 +94,14 @@ _bt_drop_lock_and_maybe_pin(IndexScanDesc scan, BTScanPos sp) * during the search will be finished. */ BTStack -_bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, - Snapshot snapshot) +_bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP, + int access, Snapshot snapshot) { BTStack stack_in = NULL; int page_access = BT_READ; /* Get the root page to start with */ - *bufP = _bt_getroot(rel, access); + *bufP = _bt_getroot(rel, heaprel, access); /* If index is empty and access = BT_READ, no root page is created. */ if (!BufferIsValid(*bufP)) @@ -129,8 +130,8 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * also taken care of in _bt_getstackbuf). But this is a good * opportunity to finish splits of internal pages too. */ - *bufP = _bt_moveright(rel, key, *bufP, (access == BT_WRITE), stack_in, - page_access, snapshot); + *bufP = _bt_moveright(rel, heaprel, key, *bufP, (access == BT_WRITE), + stack_in, page_access, snapshot); /* if this is a leaf page, we're done */ page = BufferGetPage(*bufP); @@ -190,7 +191,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, * but before we acquired a write lock. If it has, we may need to * move right to its new sibling. Do that. */ - *bufP = _bt_moveright(rel, key, *bufP, true, stack_in, BT_WRITE, + *bufP = _bt_moveright(rel, heaprel, key, *bufP, true, stack_in, BT_WRITE, snapshot); } @@ -234,6 +235,7 @@ _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, int access, */ Buffer _bt_moveright(Relation rel, + Relation heaprel, BTScanInsert key, Buffer buf, bool forupdate, @@ -288,12 +290,12 @@ _bt_moveright(Relation rel, } if (P_INCOMPLETE_SPLIT(opaque)) - _bt_finish_split(rel, buf, stack); + _bt_finish_split(rel, heaprel, buf, stack); else _bt_relbuf(rel, buf); /* re-acquire the lock in the right mode, and re-check */ - buf = _bt_getbuf(rel, blkno, access); + buf = _bt_getbuf(rel, heaprel, blkno, access); continue; } @@ -860,6 +862,7 @@ bool _bt_first(IndexScanDesc scan, ScanDirection dir) { Relation rel = scan->indexRelation; + Relation heaprel = scan->heapRelation; BTScanOpaque so = (BTScanOpaque) scan->opaque; Buffer buf; BTStack stack; @@ -1352,7 +1355,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) } /* Initialize remaining insertion scan key fields */ - _bt_metaversion(rel, &inskey.heapkeyspace, &inskey.allequalimage); + _bt_metaversion(rel, heaprel, &inskey.heapkeyspace, &inskey.allequalimage); inskey.anynullkeys = false; /* unused */ inskey.nextkey = nextkey; inskey.pivotsearch = false; @@ -1363,7 +1366,7 @@ _bt_first(IndexScanDesc scan, ScanDirection dir) * Use the manufactured insertion scan key to descend the tree and * position ourselves on the target leaf page. */ - stack = _bt_search(rel, &inskey, &buf, BT_READ, scan->xs_snapshot); + stack = _bt_search(rel, heaprel, &inskey, &buf, BT_READ, scan->xs_snapshot); /* don't need to keep the stack around... */ _bt_freestack(stack); @@ -2004,7 +2007,7 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); /* step right one page */ - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, BT_READ); page = BufferGetPage(so->currPos.buf); TestForOldSnapshot(scan->xs_snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2078,7 +2081,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) if (BTScanPosIsPinned(so->currPos)) _bt_lockbuf(rel, so->currPos.buf, BT_READ); else - so->currPos.buf = _bt_getbuf(rel, so->currPos.currPage, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, + so->currPos.currPage, BT_READ); for (;;) { @@ -2092,8 +2096,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) } /* Step to next physical page */ - so->currPos.buf = _bt_walk_left(rel, so->currPos.buf, - scan->xs_snapshot); + so->currPos.buf = _bt_walk_left(rel, scan->heapRelation, + so->currPos.buf, scan->xs_snapshot); /* if we're physically at end of index, return failure */ if (so->currPos.buf == InvalidBuffer) @@ -2140,7 +2144,8 @@ _bt_readnextpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) BTScanPosInvalidate(so->currPos); return false; } - so->currPos.buf = _bt_getbuf(rel, blkno, BT_READ); + so->currPos.buf = _bt_getbuf(rel, scan->heapRelation, blkno, + BT_READ); } } } @@ -2185,7 +2190,7 @@ _bt_parallel_readpage(IndexScanDesc scan, BlockNumber blkno, ScanDirection dir) * again if it's important. */ static Buffer -_bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) +_bt_walk_left(Relation rel, Relation heaprel, Buffer buf, Snapshot snapshot) { Page page; BTPageOpaque opaque; @@ -2213,7 +2218,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) _bt_relbuf(rel, buf); /* check for interrupts while we're not holding any buffer lock */ CHECK_FOR_INTERRUPTS(); - buf = _bt_getbuf(rel, blkno, BT_READ); + buf = _bt_getbuf(rel, heaprel, blkno, BT_READ); page = BufferGetPage(buf); TestForOldSnapshot(snapshot, rel, page); opaque = BTPageGetOpaque(page); @@ -2304,7 +2309,7 @@ _bt_walk_left(Relation rel, Buffer buf, Snapshot snapshot) * The returned buffer is pinned and read-locked. */ Buffer -_bt_get_endpoint(Relation rel, uint32 level, bool rightmost, +_bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, bool rightmost, Snapshot snapshot) { Buffer buf; @@ -2320,9 +2325,9 @@ _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, * smarter about intermediate levels.) */ if (level == 0) - buf = _bt_getroot(rel, BT_READ); + buf = _bt_getroot(rel, heaprel, BT_READ); else - buf = _bt_gettrueroot(rel); + buf = _bt_gettrueroot(rel, heaprel); if (!BufferIsValid(buf)) return InvalidBuffer; @@ -2403,7 +2408,8 @@ _bt_endpoint(IndexScanDesc scan, ScanDirection dir) * version of _bt_search(). We don't maintain a stack since we know we * won't need it. */ - buf = _bt_get_endpoint(rel, 0, ScanDirectionIsBackward(dir), scan->xs_snapshot); + buf = _bt_get_endpoint(rel, scan->heapRelation, 0, + ScanDirectionIsBackward(dir), scan->xs_snapshot); if (!BufferIsValid(buf)) { diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c index 02b9601bec..1207a49689 100644 --- a/src/backend/access/nbtree/nbtsort.c +++ b/src/backend/access/nbtree/nbtsort.c @@ -566,7 +566,7 @@ _bt_leafbuild(BTSpool *btspool, BTSpool *btspool2) wstate.heap = btspool->heap; wstate.index = btspool->index; - wstate.inskey = _bt_mkscankey(wstate.index, NULL); + wstate.inskey = _bt_mkscankey(wstate.index, btspool->heap, NULL); /* _bt_mkscankey() won't set allequalimage without metapage */ wstate.inskey->allequalimage = _bt_allequalimage(wstate.index, true); wstate.btws_use_wal = RelationNeedsWAL(wstate.index); diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index 7da499c4dd..05abf36032 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c @@ -87,7 +87,7 @@ static int _bt_keep_natts(Relation rel, IndexTuple lastleft, * field themselves. */ BTScanInsert -_bt_mkscankey(Relation rel, IndexTuple itup) +_bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup) { BTScanInsert key; ScanKey skey; @@ -112,7 +112,7 @@ _bt_mkscankey(Relation rel, IndexTuple itup) key = palloc(offsetof(BTScanInsertData, scankeys) + sizeof(ScanKeyData) * indnkeyatts); if (itup) - _bt_metaversion(rel, &key->heapkeyspace, &key->allequalimage); + _bt_metaversion(rel, heaprel, &key->heapkeyspace, &key->allequalimage); else { /* Utility statement callers can set these fields themselves */ @@ -1761,7 +1761,8 @@ _bt_killitems(IndexScanDesc scan) droppedpin = true; /* Attempt to re-read the buffer, getting pin and lock. */ - buf = _bt_getbuf(scan->indexRelation, so->currPos.currPage, BT_READ); + buf = _bt_getbuf(scan->indexRelation, scan->heapRelation, + so->currPos.currPage, BT_READ); page = BufferGetPage(buf); if (BufferGetLSNAtomic(buf) == so->currPos.lsn) diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index 3adb18f2d8..2f4a4aad24 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -489,7 +489,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer) * Unlike the routines above, this works on both leaf and inner pages. */ static void -vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) +vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer) { Page page = BufferGetPage(buffer); SpGistPageOpaque opaque = SpGistPageGetOpaque(page); @@ -503,6 +503,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) spgxlogVacuumRedirect xlrec; GlobalVisState *vistest; + xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(heaprel); xlrec.nToPlaceholder = 0; xlrec.snapshotConflictHorizon = InvalidTransactionId; @@ -643,13 +644,13 @@ spgvacuumpage(spgBulkDeleteState *bds, BlockNumber blkno) else { vacuumLeafPage(bds, index, buffer, false); - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } } else { /* inner page */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); } /* @@ -719,7 +720,7 @@ spgprocesspending(spgBulkDeleteState *bds) /* deal with any deletable tuples */ vacuumLeafPage(bds, index, buffer, true); /* might as well do this while we are here */ - vacuumRedirectAndPlaceholder(index, buffer); + vacuumRedirectAndPlaceholder(index, bds->info->heaprel, buffer); SpGistSetLastUsedPage(index, buffer); diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 7777e7ec77..98a712f4ec 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3365,6 +3365,7 @@ validate_index(Oid heapId, Oid indexId, Snapshot snapshot) ivinfo.message_level = DEBUG2; ivinfo.num_heap_tuples = heapRelation->rd_rel->reltuples; ivinfo.strategy = NULL; + ivinfo.heaprel = heapRelation; /* * Encode TIDs as int8 values for the sort, rather than directly sorting diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 65750958bb..0178186d38 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -712,6 +712,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, ivinfo.message_level = elevel; ivinfo.num_heap_tuples = onerel->rd_rel->reltuples; ivinfo.strategy = vac_strategy; + ivinfo.heaprel = onerel; stats = index_vacuum_cleanup(&ivinfo, NULL); diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c index bcd40c80a1..2cdbd182b6 100644 --- a/src/backend/commands/vacuumparallel.c +++ b/src/backend/commands/vacuumparallel.c @@ -148,6 +148,9 @@ struct ParallelVacuumState /* NULL for worker processes */ ParallelContext *pcxt; + /* Parent Heap Relation */ + Relation heaprel; + /* Target indexes */ Relation *indrels; int nindexes; @@ -266,6 +269,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, pvs->nindexes = nindexes; pvs->will_parallel_vacuum = will_parallel_vacuum; pvs->bstrategy = bstrategy; + pvs->heaprel = rel; EnterParallelMode(); pcxt = CreateParallelContext("postgres", "parallel_vacuum_main", @@ -838,6 +842,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel, ivinfo.estimated_count = pvs->shared->estimated_count; ivinfo.num_heap_tuples = pvs->shared->reltuples; ivinfo.strategy = pvs->bstrategy; + ivinfo.heaprel = pvs->heaprel; /* Update error traceback information */ pvs->indname = pstrdup(RelationGetRelationName(indrel)); @@ -1007,6 +1012,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc) pvs.dead_items = dead_items; pvs.relnamespace = get_namespace_name(RelationGetNamespace(rel)); pvs.relname = pstrdup(RelationGetRelationName(rel)); + pvs.heaprel = rel; /* These fields will be filled during index vacuum or cleanup */ pvs.indname = NULL; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d58c4a1078..e3824efe9b 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -462,7 +462,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, * For btrees, get tree height while we have the index * open */ - info->tree_height = _bt_getrootheight(indexRelation); + info->tree_height = _bt_getrootheight(indexRelation, relation); } else { diff --git a/src/backend/utils/sort/tuplesortvariants.c b/src/backend/utils/sort/tuplesortvariants.c index eb6cfcfd00..0188106925 100644 --- a/src/backend/utils/sort/tuplesortvariants.c +++ b/src/backend/utils/sort/tuplesortvariants.c @@ -207,6 +207,7 @@ tuplesort_begin_heap(TupleDesc tupDesc, Tuplesortstate * tuplesort_begin_cluster(TupleDesc tupDesc, Relation indexRel, + Relation heaprel, int workMem, SortCoordinate coordinate, int sortopt) { @@ -260,7 +261,7 @@ tuplesort_begin_cluster(TupleDesc tupDesc, arg->tupDesc = tupDesc; /* assume we need not copy tupDesc */ - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heaprel, NULL); if (arg->indexInfo->ii_Expressions != NULL) { @@ -361,7 +362,7 @@ tuplesort_begin_index_btree(Relation heapRel, arg->enforceUnique = enforceUnique; arg->uniqueNullsNotDistinct = uniqueNullsNotDistinct; - indexScanKey = _bt_mkscankey(indexRel, NULL); + indexScanKey = _bt_mkscankey(indexRel, heapRel, NULL); /* Prepare SortSupport data for each column */ base->sortKeys = (SortSupport) palloc0(base->nKeys * diff --git a/src/include/access/genam.h b/src/include/access/genam.h index 83dbee0fe6..7708b82d7d 100644 --- a/src/include/access/genam.h +++ b/src/include/access/genam.h @@ -50,6 +50,7 @@ typedef struct IndexVacuumInfo int message_level; /* ereport level for progress messages */ double num_heap_tuples; /* tuples remaining in heap */ BufferAccessStrategy strategy; /* access strategy for reads */ + Relation heaprel; /* the heap relation the index belongs to */ } IndexVacuumInfo; /* diff --git a/src/include/access/gist_private.h b/src/include/access/gist_private.h index 8af33d7b40..ee275650bd 100644 --- a/src/include/access/gist_private.h +++ b/src/include/access/gist_private.h @@ -440,7 +440,7 @@ extern XLogRecPtr gistXLogPageDelete(Buffer buffer, FullTransactionId xid, Buffer parentBuffer, OffsetNumber downlinkOffset); -extern void gistXLogPageReuse(Relation rel, BlockNumber blkno, +extern void gistXLogPageReuse(Relation rel, Relation heaprel, BlockNumber blkno, FullTransactionId deleteXid); extern XLogRecPtr gistXLogUpdate(Buffer buffer, @@ -449,7 +449,8 @@ extern XLogRecPtr gistXLogUpdate(Buffer buffer, Buffer leftchildbuf); extern XLogRecPtr gistXLogDelete(Buffer buffer, OffsetNumber *todelete, - int ntodelete, TransactionId snapshotConflictHorizon); + int ntodelete, TransactionId snapshotConflictHorizon, + Relation heaprel); extern XLogRecPtr gistXLogSplit(bool page_is_leaf, SplitedPageLayout *dist, @@ -485,7 +486,7 @@ extern bool gistproperty(Oid index_oid, int attno, extern bool gistfitpage(IndexTuple *itvec, int len); extern bool gistnospace(Page page, IndexTuple *itvec, int len, OffsetNumber todelete, Size freespace); extern void gistcheckpage(Relation rel, Buffer buf); -extern Buffer gistNewBuffer(Relation r); +extern Buffer gistNewBuffer(Relation r, Relation heaprel); extern bool gistPageRecyclable(Page page); extern void gistfillbuffer(Page page, IndexTuple *itup, int len, OffsetNumber off); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index 2ce9366277..93fb9d438a 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -51,11 +51,14 @@ typedef struct gistxlogDelete { TransactionId snapshotConflictHorizon; uint16 ntodelete; /* number of deleted offsets */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TODELETE OFFSET NUMBER ARRAY FOLLOWS */ + /* TODELETE OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } gistxlogDelete; -#define SizeOfGistxlogDelete (offsetof(gistxlogDelete, ntodelete) + sizeof(uint16)) +#define SizeOfGistxlogDelete offsetof(gistxlogDelete, offsets) /* * Backup Blk 0: If this operation completes a page split, by inserting a @@ -98,9 +101,11 @@ typedef struct gistxlogPageReuse RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } gistxlogPageReuse; -#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, snapshotConflictHorizon) + sizeof(FullTransactionId)) +#define SizeOfGistxlogPageReuse (offsetof(gistxlogPageReuse, isCatalogRel) + sizeof(bool)) extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index 9894ab9afe..6c5535fe73 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -252,12 +252,14 @@ typedef struct xl_hash_vacuum_one_page { TransactionId snapshotConflictHorizon; uint16 ntuples; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ - /* TARGET OFFSET NUMBERS FOLLOW AT THE END */ + /* TARGET OFFSET NUMBERS */ + OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; } xl_hash_vacuum_one_page; -#define SizeOfHashVacuumOnePage \ - (offsetof(xl_hash_vacuum_one_page, ntuples) + sizeof(uint16)) +#define SizeOfHashVacuumOnePage offsetof(xl_hash_vacuum_one_page, offsets) extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a2c67d1cd3..08db7e62dd 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -245,10 +245,12 @@ typedef struct xl_heap_prune TransactionId snapshotConflictHorizon; uint16 nredirected; uint16 ndead; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* OFFSET NUMBERS are in the block reference 0 */ } xl_heap_prune; -#define SizeOfHeapPrune (offsetof(xl_heap_prune, ndead) + sizeof(uint16)) +#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool)) /* * The vacuum page record is similar to the prune record, but can only mark @@ -344,13 +346,15 @@ typedef struct xl_heap_freeze_page { TransactionId snapshotConflictHorizon; uint16 nplans; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* * In payload of blk 0 : FREEZE PLANS and OFFSET NUMBER ARRAY */ } xl_heap_freeze_page; -#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, nplans) + sizeof(uint16)) +#define SizeOfHeapFreezePage (offsetof(xl_heap_freeze_page, isCatalogRel) + sizeof(bool)) /* * This is what we need to know about setting a visibility map bit @@ -409,7 +413,7 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record); extern const char *heap2_identify(uint8 info); extern void heap_xlog_logical_rewrite(XLogReaderState *r); -extern XLogRecPtr log_heap_visible(RelFileLocator rlocator, Buffer heap_buffer, +extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer, TransactionId snapshotConflictHorizon, uint8 vmflags); diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h index 8f48960f9d..6dee307042 100644 --- a/src/include/access/nbtree.h +++ b/src/include/access/nbtree.h @@ -1182,8 +1182,10 @@ extern IndexTuple _bt_swap_posting(IndexTuple newitem, IndexTuple oposting, extern bool _bt_doinsert(Relation rel, IndexTuple itup, IndexUniqueCheck checkUnique, bool indexUnchanged, Relation heapRel); -extern void _bt_finish_split(Relation rel, Buffer lbuf, BTStack stack); -extern Buffer _bt_getstackbuf(Relation rel, BTStack stack, BlockNumber child); +extern void _bt_finish_split(Relation rel, Relation heaprel, Buffer lbuf, + BTStack stack); +extern Buffer _bt_getstackbuf(Relation rel, Relation heaprel, BTStack stack, + BlockNumber child); /* * prototypes for functions in nbtsplitloc.c @@ -1197,16 +1199,18 @@ extern OffsetNumber _bt_findsplitloc(Relation rel, Page origpage, */ extern void _bt_initmetapage(Page page, BlockNumber rootbknum, uint32 level, bool allequalimage); -extern bool _bt_vacuum_needs_cleanup(Relation rel); -extern void _bt_set_cleanup_info(Relation rel, BlockNumber num_delpages); +extern bool _bt_vacuum_needs_cleanup(Relation rel, Relation heaprel); +extern void _bt_set_cleanup_info(Relation rel, Relation heaprel, + BlockNumber num_delpages); extern void _bt_upgrademetapage(Page page); -extern Buffer _bt_getroot(Relation rel, int access); -extern Buffer _bt_gettrueroot(Relation rel); -extern int _bt_getrootheight(Relation rel); -extern void _bt_metaversion(Relation rel, bool *heapkeyspace, +extern Buffer _bt_getroot(Relation rel, Relation heaprel, int access); +extern Buffer _bt_gettrueroot(Relation rel, Relation heaprel); +extern int _bt_getrootheight(Relation rel, Relation heaprel); +extern void _bt_metaversion(Relation rel, Relation heaprel, bool *heapkeyspace, bool *allequalimage); extern void _bt_checkpage(Relation rel, Buffer buf); -extern Buffer _bt_getbuf(Relation rel, BlockNumber blkno, int access); +extern Buffer _bt_getbuf(Relation rel, Relation heaprel, BlockNumber blkno, + int access); extern Buffer _bt_relandgetbuf(Relation rel, Buffer obuf, BlockNumber blkno, int access); extern void _bt_relbuf(Relation rel, Buffer buf); @@ -1229,21 +1233,22 @@ extern void _bt_pendingfsm_finalize(Relation rel, BTVacState *vstate); /* * prototypes for functions in nbtsearch.c */ -extern BTStack _bt_search(Relation rel, BTScanInsert key, Buffer *bufP, - int access, Snapshot snapshot); -extern Buffer _bt_moveright(Relation rel, BTScanInsert key, Buffer buf, - bool forupdate, BTStack stack, int access, Snapshot snapshot); +extern BTStack _bt_search(Relation rel, Relation heaprel, BTScanInsert key, + Buffer *bufP, int access, Snapshot snapshot); +extern Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, + Buffer buf, bool forupdate, BTStack stack, + int access, Snapshot snapshot); extern OffsetNumber _bt_binsrch_insert(Relation rel, BTInsertState insertstate); extern int32 _bt_compare(Relation rel, BTScanInsert key, Page page, OffsetNumber offnum); extern bool _bt_first(IndexScanDesc scan, ScanDirection dir); extern bool _bt_next(IndexScanDesc scan, ScanDirection dir); -extern Buffer _bt_get_endpoint(Relation rel, uint32 level, bool rightmost, - Snapshot snapshot); +extern Buffer _bt_get_endpoint(Relation rel, Relation heaprel, uint32 level, + bool rightmost, Snapshot snapshot); /* * prototypes for functions in nbtutils.c */ -extern BTScanInsert _bt_mkscankey(Relation rel, IndexTuple itup); +extern BTScanInsert _bt_mkscankey(Relation rel, Relation heaprel, IndexTuple itup); extern void _bt_freestack(BTStack stack); extern void _bt_preprocess_array_keys(IndexScanDesc scan); extern void _bt_start_array_keys(IndexScanDesc scan, ScanDirection dir); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd67257f2..823c700dee 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -188,9 +188,11 @@ typedef struct xl_btree_reuse_page RelFileLocator locator; BlockNumber block; FullTransactionId snapshotConflictHorizon; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ } xl_btree_reuse_page; -#define SizeOfBtreeReusePage (sizeof(xl_btree_reuse_page)) +#define SizeOfBtreeReusePage (offsetof(xl_btree_reuse_page, isCatalogRel) + sizeof(bool)) /* * xl_btree_vacuum and xl_btree_delete records describe deletion of index @@ -235,6 +237,8 @@ typedef struct xl_btree_delete TransactionId snapshotConflictHorizon; uint16 ndeleted; uint16 nupdated; + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /*---- * In payload of blk 0 : @@ -245,7 +249,7 @@ typedef struct xl_btree_delete */ } xl_btree_delete; -#define SizeOfBtreeDelete (offsetof(xl_btree_delete, nupdated) + sizeof(uint16)) +#define SizeOfBtreeDelete (offsetof(xl_btree_delete, isCatalogRel) + sizeof(bool)) /* * The offsets that appear in xl_btree_update metadata are offsets into the diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index b9d6753533..75267a4914 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -240,6 +240,8 @@ typedef struct spgxlogVacuumRedirect uint16 nToPlaceholder; /* number of redirects to make placeholders */ OffsetNumber firstPlaceholder; /* first placeholder tuple to remove */ TransactionId snapshotConflictHorizon; /* newest XID of removed redirects */ + bool isCatalogRel; /* to handle recovery conflict during logical + * decoding on standby */ /* offsets of redirect tuples to make placeholders follow */ OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER]; diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h index 9165b9456b..7306a1c3ee 100644 --- a/src/include/access/visibilitymapdefs.h +++ b/src/include/access/visibilitymapdefs.h @@ -17,9 +17,11 @@ #define BITS_PER_HEAPBLOCK 2 /* Flags for bit map */ -#define VISIBILITYMAP_ALL_VISIBLE 0x01 -#define VISIBILITYMAP_ALL_FROZEN 0x02 -#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap - * flags bits */ +#define VISIBILITYMAP_ALL_VISIBLE 0x01 +#define VISIBILITYMAP_ALL_FROZEN 0x02 +#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap + * flags bits */ +#define VISIBILITYMAP_IS_CATALOG_REL 0x04 /* to handle recovery conflict during logical + * decoding on standby */ #endif /* VISIBILITYMAPDEFS_H */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 67f994cb3e..52845497cc 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -27,6 +27,7 @@ #include "storage/smgr.h" #include "utils/relcache.h" #include "utils/reltrigger.h" +#include "catalog/catalog.h" /* diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h index 12578e42bc..395abfe596 100644 --- a/src/include/utils/tuplesort.h +++ b/src/include/utils/tuplesort.h @@ -399,7 +399,9 @@ extern Tuplesortstate *tuplesort_begin_heap(TupleDesc tupDesc, int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_cluster(TupleDesc tupDesc, - Relation indexRel, int workMem, + Relation indexRel, + Relation heaprel, + int workMem, SortCoordinate coordinate, int sortopt); extern Tuplesortstate *tuplesort_begin_index_btree(Relation heapRel, -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-08 10:25 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 1 sibling, 2 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-08 10:25 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/3/23 5:26 PM, Drouvot, Bertrand wrote: > Hi, > > On 3/3/23 8:58 AM, Jeff Davis wrote: >> On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: >>> In this case it looks easier to add the right API than to be sure >>> about >>> whether it's needed or not. >> >> I attached a sketch of one approach. > > Oh, that's very cool, thanks a lot! > >> I'm not very confident that it's >> the right API or even that it works as I intended it, but if others >> like the approach I can work on it some more. >> > > I'll look at it early next week. > So, I took your patch and as an example I tried a quick integration in 0004, (see 0004_new_API.txt attached) to put it in the logical decoding on standby context. Based on this, I've 3 comments: - Maybe ConditionVariableEventSleep() should take care of the “WaitEventSetWait returns 1 and cvEvent.event == WL_POSTMASTER_DEATH” case? - Maybe ConditionVariableEventSleep() could accept and deal with the CV being NULL? I used it in the POC attached to handle logical decoding on the primary server case. One option should be to create a dedicated CV for that case though. - In the POC attached I had to add this extra condition “(cv && !RecoveryInProgress())” to avoid waiting on the timeout when there is a promotion. That makes me think that we may want to add 2 extra parameters (as 2 functions returning a bool?) to ConditionVariableEventSleep() to check whether or not we still want to test the socket or the CV wake up in each loop iteration. Also 3 additional remarks: 1) About InitializeConditionVariableWaitSet() and ConditionVariableWaitSetCreate(): I'm not sure about the naming as there is no CV yet (they "just" deal with WaitEventSet). So, what about renaming? +static WaitEventSet *ConditionVariableWaitSet = NULL; to say, "LocalWaitSet" and then rename ConditionVariableWaitSetLatchPos, InitializeConditionVariableWaitSet() and ConditionVariableWaitSetCreate() accordingly? But it might be not needed (see 3) below). 2) /* * Prepare to wait on a given condition variable. * @@ -97,7 +162,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, ConditionVariableWaitSet, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +177,27 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, ConditionVariableWaitSet, timeout, + wait_event_info); +} + I like the idea of making use of the new ConditionVariableEventSleep() here, but on the other hand... 3) I wonder if there is no race conditions: ConditionVariableWaitSet is being initialized with PGINVALID_SOCKET as WL_LATCH_SET and might be also (if IsUnderPostmaster) be initialized with PGINVALID_SOCKET as WL_EXIT_ON_PM_DEATH. So IIUC, the patch is introducing 2 new possible source of wake up. Then, what about? - not create ConditionVariableWaitSet, ConditionVariableWaitSetLatchPos, InitializeConditionVariableWaitSet() and ConditionVariableWaitSetCreate() at all? - call ConditionVariableEventSleep() with a NULL parameter in ConditionVariableSleep() and ConditionVariableTimedSleep()? - handle the case where the WaitEventSet parameter is NULL in ConditionVariableEventSleep()? (That could also make sense if we handle the case of the CV being NULL as proposed above) Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From d4423857bd73c4d87b17a0dac74388f664421e18 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Mon, 6 Mar 2023 08:17:52 +0000 Subject: [PATCH v99 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable and a new API ConditionVariableEventSleep() to fix this corner case. --- doc/src/sgml/monitoring.sgml | 4 + src/backend/access/transam/xlogrecovery.c | 28 ++++ src/backend/libpq/pqcomm.c | 14 +- src/backend/replication/walsender.c | 17 ++- src/backend/storage/lmgr/condition_variable.c | 124 +++++++++++++++--- src/backend/storage/lmgr/proc.c | 6 + src/backend/utils/activity/wait_event.c | 3 + src/backend/utils/init/miscinit.c | 1 + src/include/access/xlogrecovery.h | 3 + src/include/libpq/libpq.h | 6 +- src/include/replication/walsender.h | 1 + src/include/storage/condition_variable.h | 10 ++ src/include/utils/wait_event.h | 1 + 13 files changed, 189 insertions(+), 29 deletions(-) 13.5% src/backend/access/transam/ 8.5% src/backend/libpq/ 6.5% src/backend/replication/ 58.6% src/backend/storage/lmgr/ 4.4% src/include/storage/ 4.4% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index cdf7c09b4b..9af8d58da2 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1857,6 +1857,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser <entry>Waiting for startup process to send initial data for streaming replication.</entry> </row> + <row> + <entry><literal>WalSenderWaitReplay</literal></entry> + <entry>Waiting for startup process to replay write-ahead log.</entry> + </row> <row> <entry><literal>XactGroupUpdate</literal></entry> <entry>Waiting for the group leader to update transaction status at diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c index da5bb5fc5d..babd0b6c4e 100644 --- a/src/backend/libpq/pqcomm.c +++ b/src/backend/libpq/pqcomm.c @@ -80,6 +80,7 @@ #include "storage/ipc.h" #include "utils/guc_hooks.h" #include "utils/memutils.h" +#include "storage/condition_variable.h" /* * Cope with the various platform-specific ways to spell TCP keepalive socket @@ -172,7 +173,6 @@ void pq_init(void) { int socket_pos PG_USED_FOR_ASSERTS_ONLY; - int latch_pos PG_USED_FOR_ASSERTS_ONLY; /* initialize state variables */ PqSendBufferSize = PQ_SEND_BUFFER_SIZE; @@ -207,20 +207,14 @@ pq_init(void) elog(FATAL, "fcntl(F_SETFD) failed on socket: %m"); #endif - FeBeWaitSet = CreateWaitEventSet(TopMemoryContext, FeBeWaitSetNEvents); + FeBeWaitSet = ConditionVariableWaitSetCreate(TopMemoryContext, FeBeWaitSetNEvents); socket_pos = AddWaitEventToSet(FeBeWaitSet, WL_SOCKET_WRITEABLE, MyProcPort->sock, NULL, NULL); - latch_pos = AddWaitEventToSet(FeBeWaitSet, WL_LATCH_SET, PGINVALID_SOCKET, - MyLatch, NULL); - AddWaitEventToSet(FeBeWaitSet, WL_POSTMASTER_DEATH, PGINVALID_SOCKET, - NULL, NULL); - /* - * The event positions match the order we added them, but let's sanity - * check them to be sure. + * The socket_pos matches the order we added it, but let's sanity + * check it to be sure. */ Assert(socket_pos == FeBeWaitSetSocketPos); - Assert(latch_pos == FeBeWaitSetLatchPos); } /* -------------------------------- diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..89d1a36e6a 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1551,7 +1551,9 @@ static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; + uint32 wait_event; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *cv = NULL; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1564,9 +1566,20 @@ WalSndWaitForWal(XLogRecPtr loc) /* Get a more recent flush pointer. */ if (!RecoveryInProgress()) + { RecentFlushPtr = GetFlushRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_WAL; + } else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_REPLAY; + cv = check_for_replay(); + } + + /* Prepare the cv to sleep */ + if (cv) + ConditionVariablePrepareToSleep(cv); for (;;) { @@ -1667,9 +1680,11 @@ WalSndWaitForWal(XLogRecPtr loc) if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + ModifyWaitEvent(FeBeWaitSet, FeBeWaitSetSocketPos, wakeEvents, NULL); + ConditionVariableEventSleep(cv, FeBeWaitSet, sleeptime, wait_event); } + ConditionVariableCancelSleep(); /* reactivate latch so WalSndLoop knows to continue */ SetLatch(MyLatch); return RecentFlushPtr; diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c index 7e2bbf46d9..766f1bd7b2 100644 --- a/src/backend/storage/lmgr/condition_variable.c +++ b/src/backend/storage/lmgr/condition_variable.c @@ -27,9 +27,29 @@ #include "storage/spin.h" #include "utils/memutils.h" +#define ConditionVariableWaitSetLatchPos 0 + /* Initially, we are not prepared to sleep on any condition variable. */ static ConditionVariable *cv_sleep_target = NULL; +/* Used by ConditionVariableSleep() and ConditionVariableTimedSleep(). */ +static WaitEventSet *ConditionVariableWaitSet = NULL; + +/* + * Initialize the process-local condition variable WaitEventSet. + * + * This must be called once during startup of any process that can wait on + * condition variables, before it issues any ConditionVariableInit() calls. + */ +void +InitializeConditionVariableWaitSet(void) +{ + Assert(ConditionVariableWaitSet == NULL); + + ConditionVariableWaitSet = ConditionVariableWaitSetCreate( + TopMemoryContext, 0); +} + /* * Initialize a condition variable. */ @@ -40,6 +60,51 @@ ConditionVariableInit(ConditionVariable *cv) proclist_init(&cv->wakeup); } +/* + * Create a WaitEventSet for ConditionVariableEventSleep(). This should be + * used when the caller of ConditionVariableEventSleep() would like to wake up + * on either the condition variable signal or a socket event. For example: + * + * ConditionVariableInit(&cv); + * waitset = ConditionVariableWaitSetCreate(mcxt, 1); + * event_pos = AddWaitEventToSet(waitset, 0, sock, NULL, NULL); + * ... + * ConditionVariablePrepareToSleep(&cv); + * while (...condition not met...) + * { + * socket_wait_events = ... + * ModifyWaitEvent(waitset, event_pos, socket_wait_events, NULL); + * ConditionVariableEventSleep(&cv, waitset, ...); + * } + * ConditionVariableCancelSleep(); + * + * The waitset is created with the standard events for a condition variable, + * and room for adding n_socket_events additional socket events. The + * initially-filled event positions should not be modified, but added socket + * events can be modified. The same waitset can be used for multiple condition + * variables as long as the callers of ConditionVariableEventSleep() are + * interested in the same sockets. + */ +WaitEventSet * +ConditionVariableWaitSetCreate(MemoryContext mcxt, int n_socket_events) +{ + int latch_pos PG_USED_FOR_ASSERTS_ONLY; + int n_cv_events = IsUnderPostmaster ? 2 : 1; + int nevents = n_cv_events + n_socket_events; + WaitEventSet *waitset = CreateWaitEventSet(mcxt, nevents); + + latch_pos = AddWaitEventToSet(waitset, WL_LATCH_SET, PGINVALID_SOCKET, + MyLatch, NULL); + + if (IsUnderPostmaster) + AddWaitEventToSet(waitset, WL_EXIT_ON_PM_DEATH, PGINVALID_SOCKET, + NULL, NULL); + + Assert(latch_pos == ConditionVariableWaitSetLatchPos); + + return waitset; +} + /* * Prepare to wait on a given condition variable. * @@ -97,7 +162,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, ConditionVariableWaitSet, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +177,27 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, ConditionVariableWaitSet, timeout, + wait_event_info); +} + +/* + * Wait for a condition variable to be signaled, a timeout to be reached, or a + * socket event in the given waitset. The waitset must have been created by + * ConditionVariableWaitSetCreate(). + * + * Returns true when timeout expires, otherwise returns false. + * + * See ConditionVariableSleep() for general usage. + */ +bool +ConditionVariableEventSleep(ConditionVariable *cv, WaitEventSet *waitset, + long timeout, uint32 wait_event_info) { long cur_timeout = -1; instr_time start_time; instr_time cur_time; - int wait_events; /* * If the caller didn't prepare to sleep explicitly, then do so now and @@ -132,7 +214,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * If we are currently prepared to sleep on some other CV, we just cancel * that and prepare this one; see ConditionVariablePrepareToSleep. */ - if (cv_sleep_target != cv) + if (cv && cv_sleep_target != cv) { ConditionVariablePrepareToSleep(cv); return false; @@ -147,24 +229,28 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, INSTR_TIME_SET_CURRENT(start_time); Assert(timeout >= 0 && timeout <= INT_MAX); cur_timeout = timeout; - wait_events = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH; } - else - wait_events = WL_LATCH_SET | WL_EXIT_ON_PM_DEATH; while (true) { bool done = false; + WaitEvent cvEvent; + int nevents; /* - * Wait for latch to be set. (If we're awakened for some other - * reason, the code below will cope anyway.) + * Wait for latch to be set, or other events which will be handled + * below. */ - (void) WaitLatch(MyLatch, wait_events, cur_timeout, wait_event_info); + nevents = WaitEventSetWait(waitset, cur_timeout, &cvEvent, + 1, wait_event_info); /* Reset latch before examining the state of the wait list. */ ResetLatch(MyLatch); + /* If a socket event occurred, no need to check wait list. */ + if (nevents == 1 && (cvEvent.events & WL_SOCKET_MASK) != 0) + return true; + /* * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or @@ -180,13 +266,21 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * by something other than ConditionVariableSignal; though we don't * guarantee not to return spuriously, we'll avoid this obvious case. */ - SpinLockAcquire(&cv->mutex); - if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + + if (cv) { - done = true; - proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + SpinLockAcquire(&cv->mutex); + if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + { + done = true; + proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + } + SpinLockRelease(&cv->mutex); } - SpinLockRelease(&cv->mutex); + + /* Note for the POC: If we are not waiting on a CV or have just been promoted. */ + if (!cv || (cv && !RecoveryInProgress())) + done = true; /* * Check for interrupts, and return spuriously if that caused the @@ -194,7 +288,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * waited for a different condition variable). */ CHECK_FOR_INTERRUPTS(); - if (cv != cv_sleep_target) + if (cv && cv != cv_sleep_target) done = true; /* We were signaled, so return */ diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 22b4278610..ae4a7aecd4 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -440,6 +440,9 @@ InitProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); @@ -596,6 +599,9 @@ InitAuxiliaryProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..a10dcd4e61 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalSenderWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c index 7eb7fe87f6..d07d24bc45 100644 --- a/src/backend/utils/init/miscinit.c +++ b/src/backend/utils/init/miscinit.c @@ -40,6 +40,7 @@ #include "postmaster/interrupt.h" #include "postmaster/pgarch.h" #include "postmaster/postmaster.h" +#include "storage/condition_variable.h" #include "storage/fd.h" #include "storage/ipc.h" #include "storage/latch.h" diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h index 50fc781f47..33eddc7d40 100644 --- a/src/include/libpq/libpq.h +++ b/src/include/libpq/libpq.h @@ -60,9 +60,9 @@ extern const PGDLLIMPORT PQcommMethods *PqCommMethods; */ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet; -#define FeBeWaitSetSocketPos 0 -#define FeBeWaitSetLatchPos 1 -#define FeBeWaitSetNEvents 3 +#define FeBeWaitSetLatchPos 0 +#define FeBeWaitSetSocketPos 2 +#define FeBeWaitSetNEvents 1 extern int StreamServerPort(int family, const char *hostName, unsigned short portNumber, const char *unixSocketDir, diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/storage/condition_variable.h b/src/include/storage/condition_variable.h index 589bdd323c..94adb54b91 100644 --- a/src/include/storage/condition_variable.h +++ b/src/include/storage/condition_variable.h @@ -22,6 +22,7 @@ #ifndef CONDITION_VARIABLE_H #define CONDITION_VARIABLE_H +#include "storage/latch.h" #include "storage/proclist_types.h" #include "storage/spin.h" @@ -42,9 +43,14 @@ typedef union ConditionVariableMinimallyPadded char pad[CV_MINIMAL_SIZE]; } ConditionVariableMinimallyPadded; +extern void InitializeConditionVariableWaitSet(void); + /* Initialize a condition variable. */ extern void ConditionVariableInit(ConditionVariable *cv); +extern WaitEventSet *ConditionVariableWaitSetCreate(MemoryContext mcxt, + int n_socket_events); + /* * To sleep on a condition variable, a process should use a loop which first * checks the condition, exiting the loop if it is met, and then calls @@ -56,6 +62,10 @@ extern void ConditionVariableInit(ConditionVariable *cv); extern void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info); extern bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info); +extern bool ConditionVariableEventSleep(ConditionVariable *cv, + WaitEventSet *cvEventSet, + long timeout, + uint32 wait_event_info); extern void ConditionVariableCancelSleep(void); /* diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 Attachments: [text/plain] 0004_new_API.txt (19.7K, ../../[email protected]/2-0004_new_API.txt) download | inline diff: From d4423857bd73c4d87b17a0dac74388f664421e18 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Mon, 6 Mar 2023 08:17:52 +0000 Subject: [PATCH v99 4/6] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable and a new API ConditionVariableEventSleep() to fix this corner case. --- doc/src/sgml/monitoring.sgml | 4 + src/backend/access/transam/xlogrecovery.c | 28 ++++ src/backend/libpq/pqcomm.c | 14 +- src/backend/replication/walsender.c | 17 ++- src/backend/storage/lmgr/condition_variable.c | 124 +++++++++++++++--- src/backend/storage/lmgr/proc.c | 6 + src/backend/utils/activity/wait_event.c | 3 + src/backend/utils/init/miscinit.c | 1 + src/include/access/xlogrecovery.h | 3 + src/include/libpq/libpq.h | 6 +- src/include/replication/walsender.h | 1 + src/include/storage/condition_variable.h | 10 ++ src/include/utils/wait_event.h | 1 + 13 files changed, 189 insertions(+), 29 deletions(-) 13.5% src/backend/access/transam/ 8.5% src/backend/libpq/ 6.5% src/backend/replication/ 58.6% src/backend/storage/lmgr/ 4.4% src/include/storage/ 4.4% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index cdf7c09b4b..9af8d58da2 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1857,6 +1857,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser <entry>Waiting for startup process to send initial data for streaming replication.</entry> </row> + <row> + <entry><literal>WalSenderWaitReplay</literal></entry> + <entry>Waiting for startup process to replay write-ahead log.</entry> + </row> <row> <entry><literal>XactGroupUpdate</literal></entry> <entry>Waiting for the group leader to update transaction status at diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c index da5bb5fc5d..babd0b6c4e 100644 --- a/src/backend/libpq/pqcomm.c +++ b/src/backend/libpq/pqcomm.c @@ -80,6 +80,7 @@ #include "storage/ipc.h" #include "utils/guc_hooks.h" #include "utils/memutils.h" +#include "storage/condition_variable.h" /* * Cope with the various platform-specific ways to spell TCP keepalive socket @@ -172,7 +173,6 @@ void pq_init(void) { int socket_pos PG_USED_FOR_ASSERTS_ONLY; - int latch_pos PG_USED_FOR_ASSERTS_ONLY; /* initialize state variables */ PqSendBufferSize = PQ_SEND_BUFFER_SIZE; @@ -207,20 +207,14 @@ pq_init(void) elog(FATAL, "fcntl(F_SETFD) failed on socket: %m"); #endif - FeBeWaitSet = CreateWaitEventSet(TopMemoryContext, FeBeWaitSetNEvents); + FeBeWaitSet = ConditionVariableWaitSetCreate(TopMemoryContext, FeBeWaitSetNEvents); socket_pos = AddWaitEventToSet(FeBeWaitSet, WL_SOCKET_WRITEABLE, MyProcPort->sock, NULL, NULL); - latch_pos = AddWaitEventToSet(FeBeWaitSet, WL_LATCH_SET, PGINVALID_SOCKET, - MyLatch, NULL); - AddWaitEventToSet(FeBeWaitSet, WL_POSTMASTER_DEATH, PGINVALID_SOCKET, - NULL, NULL); - /* - * The event positions match the order we added them, but let's sanity - * check them to be sure. + * The socket_pos matches the order we added it, but let's sanity + * check it to be sure. */ Assert(socket_pos == FeBeWaitSetSocketPos); - Assert(latch_pos == FeBeWaitSetLatchPos); } /* -------------------------------- diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..89d1a36e6a 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1551,7 +1551,9 @@ static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; + uint32 wait_event; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *cv = NULL; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1564,9 +1566,20 @@ WalSndWaitForWal(XLogRecPtr loc) /* Get a more recent flush pointer. */ if (!RecoveryInProgress()) + { RecentFlushPtr = GetFlushRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_WAL; + } else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_REPLAY; + cv = check_for_replay(); + } + + /* Prepare the cv to sleep */ + if (cv) + ConditionVariablePrepareToSleep(cv); for (;;) { @@ -1667,9 +1680,11 @@ WalSndWaitForWal(XLogRecPtr loc) if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + ModifyWaitEvent(FeBeWaitSet, FeBeWaitSetSocketPos, wakeEvents, NULL); + ConditionVariableEventSleep(cv, FeBeWaitSet, sleeptime, wait_event); } + ConditionVariableCancelSleep(); /* reactivate latch so WalSndLoop knows to continue */ SetLatch(MyLatch); return RecentFlushPtr; diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c index 7e2bbf46d9..766f1bd7b2 100644 --- a/src/backend/storage/lmgr/condition_variable.c +++ b/src/backend/storage/lmgr/condition_variable.c @@ -27,9 +27,29 @@ #include "storage/spin.h" #include "utils/memutils.h" +#define ConditionVariableWaitSetLatchPos 0 + /* Initially, we are not prepared to sleep on any condition variable. */ static ConditionVariable *cv_sleep_target = NULL; +/* Used by ConditionVariableSleep() and ConditionVariableTimedSleep(). */ +static WaitEventSet *ConditionVariableWaitSet = NULL; + +/* + * Initialize the process-local condition variable WaitEventSet. + * + * This must be called once during startup of any process that can wait on + * condition variables, before it issues any ConditionVariableInit() calls. + */ +void +InitializeConditionVariableWaitSet(void) +{ + Assert(ConditionVariableWaitSet == NULL); + + ConditionVariableWaitSet = ConditionVariableWaitSetCreate( + TopMemoryContext, 0); +} + /* * Initialize a condition variable. */ @@ -40,6 +60,51 @@ ConditionVariableInit(ConditionVariable *cv) proclist_init(&cv->wakeup); } +/* + * Create a WaitEventSet for ConditionVariableEventSleep(). This should be + * used when the caller of ConditionVariableEventSleep() would like to wake up + * on either the condition variable signal or a socket event. For example: + * + * ConditionVariableInit(&cv); + * waitset = ConditionVariableWaitSetCreate(mcxt, 1); + * event_pos = AddWaitEventToSet(waitset, 0, sock, NULL, NULL); + * ... + * ConditionVariablePrepareToSleep(&cv); + * while (...condition not met...) + * { + * socket_wait_events = ... + * ModifyWaitEvent(waitset, event_pos, socket_wait_events, NULL); + * ConditionVariableEventSleep(&cv, waitset, ...); + * } + * ConditionVariableCancelSleep(); + * + * The waitset is created with the standard events for a condition variable, + * and room for adding n_socket_events additional socket events. The + * initially-filled event positions should not be modified, but added socket + * events can be modified. The same waitset can be used for multiple condition + * variables as long as the callers of ConditionVariableEventSleep() are + * interested in the same sockets. + */ +WaitEventSet * +ConditionVariableWaitSetCreate(MemoryContext mcxt, int n_socket_events) +{ + int latch_pos PG_USED_FOR_ASSERTS_ONLY; + int n_cv_events = IsUnderPostmaster ? 2 : 1; + int nevents = n_cv_events + n_socket_events; + WaitEventSet *waitset = CreateWaitEventSet(mcxt, nevents); + + latch_pos = AddWaitEventToSet(waitset, WL_LATCH_SET, PGINVALID_SOCKET, + MyLatch, NULL); + + if (IsUnderPostmaster) + AddWaitEventToSet(waitset, WL_EXIT_ON_PM_DEATH, PGINVALID_SOCKET, + NULL, NULL); + + Assert(latch_pos == ConditionVariableWaitSetLatchPos); + + return waitset; +} + /* * Prepare to wait on a given condition variable. * @@ -97,7 +162,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, ConditionVariableWaitSet, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +177,27 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, ConditionVariableWaitSet, timeout, + wait_event_info); +} + +/* + * Wait for a condition variable to be signaled, a timeout to be reached, or a + * socket event in the given waitset. The waitset must have been created by + * ConditionVariableWaitSetCreate(). + * + * Returns true when timeout expires, otherwise returns false. + * + * See ConditionVariableSleep() for general usage. + */ +bool +ConditionVariableEventSleep(ConditionVariable *cv, WaitEventSet *waitset, + long timeout, uint32 wait_event_info) { long cur_timeout = -1; instr_time start_time; instr_time cur_time; - int wait_events; /* * If the caller didn't prepare to sleep explicitly, then do so now and @@ -132,7 +214,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * If we are currently prepared to sleep on some other CV, we just cancel * that and prepare this one; see ConditionVariablePrepareToSleep. */ - if (cv_sleep_target != cv) + if (cv && cv_sleep_target != cv) { ConditionVariablePrepareToSleep(cv); return false; @@ -147,24 +229,28 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, INSTR_TIME_SET_CURRENT(start_time); Assert(timeout >= 0 && timeout <= INT_MAX); cur_timeout = timeout; - wait_events = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH; } - else - wait_events = WL_LATCH_SET | WL_EXIT_ON_PM_DEATH; while (true) { bool done = false; + WaitEvent cvEvent; + int nevents; /* - * Wait for latch to be set. (If we're awakened for some other - * reason, the code below will cope anyway.) + * Wait for latch to be set, or other events which will be handled + * below. */ - (void) WaitLatch(MyLatch, wait_events, cur_timeout, wait_event_info); + nevents = WaitEventSetWait(waitset, cur_timeout, &cvEvent, + 1, wait_event_info); /* Reset latch before examining the state of the wait list. */ ResetLatch(MyLatch); + /* If a socket event occurred, no need to check wait list. */ + if (nevents == 1 && (cvEvent.events & WL_SOCKET_MASK) != 0) + return true; + /* * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or @@ -180,13 +266,21 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * by something other than ConditionVariableSignal; though we don't * guarantee not to return spuriously, we'll avoid this obvious case. */ - SpinLockAcquire(&cv->mutex); - if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + + if (cv) { - done = true; - proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + SpinLockAcquire(&cv->mutex); + if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + { + done = true; + proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + } + SpinLockRelease(&cv->mutex); } - SpinLockRelease(&cv->mutex); + + /* Note for the POC: If we are not waiting on a CV or have just been promoted. */ + if (!cv || (cv && !RecoveryInProgress())) + done = true; /* * Check for interrupts, and return spuriously if that caused the @@ -194,7 +288,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * waited for a different condition variable). */ CHECK_FOR_INTERRUPTS(); - if (cv != cv_sleep_target) + if (cv && cv != cv_sleep_target) done = true; /* We were signaled, so return */ diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 22b4278610..ae4a7aecd4 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -440,6 +440,9 @@ InitProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); @@ -596,6 +599,9 @@ InitAuxiliaryProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..a10dcd4e61 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalSenderWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c index 7eb7fe87f6..d07d24bc45 100644 --- a/src/backend/utils/init/miscinit.c +++ b/src/backend/utils/init/miscinit.c @@ -40,6 +40,7 @@ #include "postmaster/interrupt.h" #include "postmaster/pgarch.h" #include "postmaster/postmaster.h" +#include "storage/condition_variable.h" #include "storage/fd.h" #include "storage/ipc.h" #include "storage/latch.h" diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h index 50fc781f47..33eddc7d40 100644 --- a/src/include/libpq/libpq.h +++ b/src/include/libpq/libpq.h @@ -60,9 +60,9 @@ extern const PGDLLIMPORT PQcommMethods *PqCommMethods; */ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet; -#define FeBeWaitSetSocketPos 0 -#define FeBeWaitSetLatchPos 1 -#define FeBeWaitSetNEvents 3 +#define FeBeWaitSetLatchPos 0 +#define FeBeWaitSetSocketPos 2 +#define FeBeWaitSetNEvents 1 extern int StreamServerPort(int family, const char *hostName, unsigned short portNumber, const char *unixSocketDir, diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/storage/condition_variable.h b/src/include/storage/condition_variable.h index 589bdd323c..94adb54b91 100644 --- a/src/include/storage/condition_variable.h +++ b/src/include/storage/condition_variable.h @@ -22,6 +22,7 @@ #ifndef CONDITION_VARIABLE_H #define CONDITION_VARIABLE_H +#include "storage/latch.h" #include "storage/proclist_types.h" #include "storage/spin.h" @@ -42,9 +43,14 @@ typedef union ConditionVariableMinimallyPadded char pad[CV_MINIMAL_SIZE]; } ConditionVariableMinimallyPadded; +extern void InitializeConditionVariableWaitSet(void); + /* Initialize a condition variable. */ extern void ConditionVariableInit(ConditionVariable *cv); +extern WaitEventSet *ConditionVariableWaitSetCreate(MemoryContext mcxt, + int n_socket_events); + /* * To sleep on a condition variable, a process should use a loop which first * checks the condition, exiting the loop if it is met, and then calls @@ -56,6 +62,10 @@ extern void ConditionVariableInit(ConditionVariable *cv); extern void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info); extern bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info); +extern bool ConditionVariableEventSleep(ConditionVariable *cv, + WaitEventSet *cvEventSet, + long timeout, + uint32 wait_event_info); extern void ConditionVariableCancelSleep(void); /* diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-10 11:33 Drouvot, Bertrand <[email protected]> parent: Drouvot, Bertrand <[email protected]> 1 sibling, 0 replies; 41+ messages in thread From: Drouvot, Bertrand @ 2023-03-10 11:33 UTC (permalink / raw) To: Jeff Davis <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 3/8/23 11:25 AM, Drouvot, Bertrand wrote: > Hi, > > On 3/3/23 5:26 PM, Drouvot, Bertrand wrote: >> Hi, >> >> On 3/3/23 8:58 AM, Jeff Davis wrote: >>> On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: >>>> In this case it looks easier to add the right API than to be sure >>>> about >>>> whether it's needed or not. >>> >>> I attached a sketch of one approach. >> >> Oh, that's very cool, thanks a lot! >> >>> I'm not very confident that it's >>> the right API or even that it works as I intended it, but if others >>> like the approach I can work on it some more. >>> >> >> I'll look at it early next week. >> > > So, I took your patch and as an example I tried a quick integration in 0004, > (see 0004_new_API.txt attached) to put it in the logical decoding on standby context. > > Based on this, I've 3 comments: > > - Maybe ConditionVariableEventSleep() should take care of the “WaitEventSetWait returns 1 and cvEvent.event == WL_POSTMASTER_DEATH” case? > > - Maybe ConditionVariableEventSleep() could accept and deal with the CV being NULL? > I used it in the POC attached to handle logical decoding on the primary server case. > One option should be to create a dedicated CV for that case though. > > - In the POC attached I had to add this extra condition “(cv && !RecoveryInProgress())” to avoid waiting on the timeout when there is a promotion. > That makes me think that we may want to add 2 extra parameters (as 2 functions returning a bool?) to ConditionVariableEventSleep() > to check whether or not we still want to test the socket or the CV wake up in each loop iteration. > > Also 3 additional remarks: > > 1) About InitializeConditionVariableWaitSet() and ConditionVariableWaitSetCreate(): I'm not sure about the naming as there is no CV yet (they "just" deal with WaitEventSet). > > So, what about renaming? > > +static WaitEventSet *ConditionVariableWaitSet = NULL; > > to say, "LocalWaitSet" and then rename ConditionVariableWaitSetLatchPos, InitializeConditionVariableWaitSet() and ConditionVariableWaitSetCreate() accordingly? > > But it might be not needed (see 3) below). > > 2) > > /* > * Prepare to wait on a given condition variable. > * > @@ -97,7 +162,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) > void > ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) > { > - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , > + (void) ConditionVariableEventSleep(cv, ConditionVariableWaitSet, > + -1 /* no timeout */ , > wait_event_info); > } > > @@ -111,11 +177,27 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) > bool > ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, > uint32 wait_event_info) > +{ > + return ConditionVariableEventSleep(cv, ConditionVariableWaitSet, timeout, > + wait_event_info); > +} > + > > I like the idea of making use of the new ConditionVariableEventSleep() here, but on the other hand... > > 3) > > I wonder if there is no race conditions: ConditionVariableWaitSet is being initialized with PGINVALID_SOCKET > as WL_LATCH_SET and might be also (if IsUnderPostmaster) be initialized with PGINVALID_SOCKET as WL_EXIT_ON_PM_DEATH. > > So IIUC, the patch is introducing 2 new possible source of wake up. > > Then, what about? > > - not create ConditionVariableWaitSet, ConditionVariableWaitSetLatchPos, InitializeConditionVariableWaitSet() and ConditionVariableWaitSetCreate() at all? > - call ConditionVariableEventSleep() with a NULL parameter in ConditionVariableSleep() and ConditionVariableTimedSleep()? > - handle the case where the WaitEventSet parameter is NULL in ConditionVariableEventSleep()? (That could also make sense if we handle the case of the CV being NULL as proposed above) > I gave it a try, so please find attached v2-0001-Introduce-ConditionVariableEventSleep.txt (implementing the comments above) and 0004_new_API.txt to put the new API in the logical decoding on standby context. There is no change in v2-0001-Introduce-ConditionVariableEventSleep.txt regarding the up-thread comment related to WL_POSTMASTER_DEATH. What do you think? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com From 9a820140b7356ab94479499a80fc4742403f3ca5 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Fri, 10 Mar 2023 10:58:23 +0000 Subject: [PATCH v99 5/7] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable and a new API ConditionVariableEventSleep() to fix this corner case. --- doc/src/sgml/monitoring.sgml | 4 ++++ src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++++++ src/backend/replication/walsender.c | 18 ++++++++++++++- src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 7 files changed, 57 insertions(+), 1 deletion(-) 7.8% doc/src/sgml/ 52.1% src/backend/access/transam/ 27.1% src/backend/replication/ 4.5% src/backend/utils/activity/ 4.5% src/include/access/ 3.7% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index cdf7c09b4b..9af8d58da2 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1857,6 +1857,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser <entry>Waiting for startup process to send initial data for streaming replication.</entry> </row> + <row> + <entry><literal>WalSenderWaitReplay</literal></entry> + <entry>Waiting for startup process to replay write-ahead log.</entry> + </row> <row> <entry><literal>XactGroupUpdate</literal></entry> <entry>Waiting for the group leader to update transaction status at diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..8ef22616bb 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1551,7 +1551,9 @@ static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; + uint32 wait_event; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *cv = NULL; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1564,9 +1566,20 @@ WalSndWaitForWal(XLogRecPtr loc) /* Get a more recent flush pointer. */ if (!RecoveryInProgress()) + { RecentFlushPtr = GetFlushRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_WAL; + } else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_REPLAY; + cv = check_for_replay(); + } + + /* Prepare the cv to sleep */ + if (cv) + ConditionVariablePrepareToSleep(cv); for (;;) { @@ -1667,9 +1680,12 @@ WalSndWaitForWal(XLogRecPtr loc) if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + ModifyWaitEvent(FeBeWaitSet, FeBeWaitSetSocketPos, wakeEvents, NULL); + ConditionVariableEventSleep(cv, RecoveryInProgress, FeBeWaitSet, NULL, + sleeptime, wait_event); } + ConditionVariableCancelSleep(); /* reactivate latch so WalSndLoop knows to continue */ SetLatch(MyLatch); return RecentFlushPtr; diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..a10dcd4e61 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalSenderWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 From 0044078a540fcb2b5f5c728dcb7e4911b000d6d5 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Fri, 10 Mar 2023 10:57:22 +0000 Subject: [PATCH v99 4/7] Introduce-ConditionVariableEventSleep --- src/backend/storage/lmgr/condition_variable.c | 65 ++++++++++++++----- src/include/storage/condition_variable.h | 7 ++ 2 files changed, 57 insertions(+), 15 deletions(-) 89.0% src/backend/storage/lmgr/ 10.9% src/include/storage/ diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c index 7e2bbf46d9..af241e7317 100644 --- a/src/backend/storage/lmgr/condition_variable.c +++ b/src/backend/storage/lmgr/condition_variable.c @@ -97,7 +97,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, NULL, NULL, NULL, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +112,28 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, NULL, NULL, NULL, timeout, + wait_event_info); +} + +/* + * Wait for a condition variable to be signaled, a timeout to be reached, or a + * socket event in the given waitset. + * + * Returns true when timeout expires, otherwise returns false. + * + * See ConditionVariableSleep() for general usage. + */ +bool +ConditionVariableEventSleep(ConditionVariable *cv, bool (*cv_resume_waiting)(void), + WaitEventSet *waitset, + bool (*waitset_resume_waiting)(void), + long timeout, uint32 wait_event_info) { long cur_timeout = -1; instr_time start_time; instr_time cur_time; - int wait_events; /* * If the caller didn't prepare to sleep explicitly, then do so now and @@ -132,7 +150,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * If we are currently prepared to sleep on some other CV, we just cancel * that and prepare this one; see ConditionVariablePrepareToSleep. */ - if (cv_sleep_target != cv) + if (cv && cv_sleep_target != cv) { ConditionVariablePrepareToSleep(cv); return false; @@ -147,24 +165,29 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, INSTR_TIME_SET_CURRENT(start_time); Assert(timeout >= 0 && timeout <= INT_MAX); cur_timeout = timeout; - wait_events = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH; } - else - wait_events = WL_LATCH_SET | WL_EXIT_ON_PM_DEATH; while (true) { bool done = false; + WaitEvent cvEvent; + int nevents = 0; /* - * Wait for latch to be set. (If we're awakened for some other - * reason, the code below will cope anyway.) + * Wait for latch to be set, or other events which will be handled + * below. */ - (void) WaitLatch(MyLatch, wait_events, cur_timeout, wait_event_info); + if (waitset) + nevents = WaitEventSetWait(waitset, cur_timeout, &cvEvent, + 1, wait_event_info); /* Reset latch before examining the state of the wait list. */ ResetLatch(MyLatch); + /* If a socket event occurred, no need to check wait list. */ + if (nevents == 1 && (cvEvent.events & WL_SOCKET_MASK) != 0) + return true; + /* * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or @@ -180,13 +203,25 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * by something other than ConditionVariableSignal; though we don't * guarantee not to return spuriously, we'll avoid this obvious case. */ - SpinLockAcquire(&cv->mutex); - if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + + if (cv) { - done = true; - proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + SpinLockAcquire(&cv->mutex); + if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + { + done = true; + proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + } + SpinLockRelease(&cv->mutex); } - SpinLockRelease(&cv->mutex); + + /* If we are not waiting on a CV or don't want to wait anymore */ + if (!cv || (cv && cv_resume_waiting && !cv_resume_waiting())) + done = true; + + /* If we don't want to wait on the waitset anymore */ + if (waitset && waitset_resume_waiting && !waitset_resume_waiting()) + done = true; /* * Check for interrupts, and return spuriously if that caused the @@ -194,7 +229,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * waited for a different condition variable). */ CHECK_FOR_INTERRUPTS(); - if (cv != cv_sleep_target) + if (cv && cv != cv_sleep_target) done = true; /* We were signaled, so return */ diff --git a/src/include/storage/condition_variable.h b/src/include/storage/condition_variable.h index 589bdd323c..b9510caa17 100644 --- a/src/include/storage/condition_variable.h +++ b/src/include/storage/condition_variable.h @@ -22,6 +22,7 @@ #ifndef CONDITION_VARIABLE_H #define CONDITION_VARIABLE_H +#include "storage/latch.h" #include "storage/proclist_types.h" #include "storage/spin.h" @@ -56,6 +57,12 @@ extern void ConditionVariableInit(ConditionVariable *cv); extern void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info); extern bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info); +extern bool ConditionVariableEventSleep(ConditionVariable *cv, + bool (*cv_resume_waiting)(void), + WaitEventSet *cvEventSet, + bool (*waitset_resume_waiting)(void), + long timeout, + uint32 wait_event_info); extern void ConditionVariableCancelSleep(void); /* -- 2.34.1 Attachments: [text/plain] 0004_new_API.txt (7.7K, ../../[email protected]/2-0004_new_API.txt) download | inline diff: From 9a820140b7356ab94479499a80fc4742403f3ca5 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Fri, 10 Mar 2023 10:58:23 +0000 Subject: [PATCH v99 5/7] Fixing Walsender corner case with logical decoding on standby. The problem is that WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up by walreceiver when new WAL has been flushed. Which means that typically walsenders will get woken up at the same time that the startup process will be - which means that by the time the logical walsender checks GetXLogReplayRecPtr() it's unlikely that the startup process already replayed the record and updated XLogCtl->lastReplayedEndRecPtr. Introducing a new condition variable and a new API ConditionVariableEventSleep() to fix this corner case. --- doc/src/sgml/monitoring.sgml | 4 ++++ src/backend/access/transam/xlogrecovery.c | 28 +++++++++++++++++++++++ src/backend/replication/walsender.c | 18 ++++++++++++++- src/backend/utils/activity/wait_event.c | 3 +++ src/include/access/xlogrecovery.h | 3 +++ src/include/replication/walsender.h | 1 + src/include/utils/wait_event.h | 1 + 7 files changed, 57 insertions(+), 1 deletion(-) 7.8% doc/src/sgml/ 52.1% src/backend/access/transam/ 27.1% src/backend/replication/ 4.5% src/backend/utils/activity/ 4.5% src/include/access/ 3.7% src/include/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index cdf7c09b4b..9af8d58da2 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1857,6 +1857,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser <entry>Waiting for startup process to send initial data for streaming replication.</entry> </row> + <row> + <entry><literal>WalSenderWaitReplay</literal></entry> + <entry>Waiting for startup process to replay write-ahead log.</entry> + </row> <row> <entry><literal>XactGroupUpdate</literal></entry> <entry>Waiting for the group leader to update transaction status at diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..8a9505a52d 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -358,6 +358,9 @@ typedef struct XLogRecoveryCtlData RecoveryPauseState recoveryPauseState; ConditionVariable recoveryNotPausedCV; + /* Replay state (see check_for_replay() for more explanation) */ + ConditionVariable replayedCV; + slock_t info_lck; /* locks shared variables shown above */ } XLogRecoveryCtlData; @@ -468,6 +471,7 @@ XLogRecoveryShmemInit(void) SpinLockInit(&XLogRecoveryCtl->info_lck); InitSharedLatch(&XLogRecoveryCtl->recoveryWakeupLatch); ConditionVariableInit(&XLogRecoveryCtl->recoveryNotPausedCV); + ConditionVariableInit(&XLogRecoveryCtl->replayedCV); } /* @@ -1935,6 +1939,11 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl XLogRecoveryCtl->lastReplayedTLI = *replayTLI; SpinLockRelease(&XLogRecoveryCtl->info_lck); + /* + * wake up walsender(s) used by logical decoding on standby. + */ + ConditionVariableBroadcast(&XLogRecoveryCtl->replayedCV); + /* * If rm_redo called XLogRequestWalReceiverReply, then we wake up the * receiver so that it notices the updated lastReplayedEndRecPtr and sends @@ -4942,3 +4951,22 @@ assign_recovery_target_xid(const char *newval, void *extra) else recoveryTarget = RECOVERY_TARGET_UNSET; } + +/* + * Return the ConditionVariable indicating that a replay has been done. + * + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that typically + * walsenders will get woken up at the same time that the startup process + * will be - which means that by the time the logical walsender checks + * GetXLogReplayRecPtr() it's unlikely that the startup process already replayed + * the record and updated XLogCtl->lastReplayedEndRecPtr. + * + * The ConditionVariable XLogRecoveryCtl->replayedCV solves this corner case. + */ +ConditionVariable * +check_for_replay(void) +{ + return &XLogRecoveryCtl->replayedCV; +} diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3042e5bd64..8ef22616bb 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1551,7 +1551,9 @@ static XLogRecPtr WalSndWaitForWal(XLogRecPtr loc) { int wakeEvents; + uint32 wait_event; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; + ConditionVariable *cv = NULL; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1564,9 +1566,20 @@ WalSndWaitForWal(XLogRecPtr loc) /* Get a more recent flush pointer. */ if (!RecoveryInProgress()) + { RecentFlushPtr = GetFlushRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_WAL; + } else + { RecentFlushPtr = GetXLogReplayRecPtr(NULL); + wait_event = WAIT_EVENT_WAL_SENDER_WAIT_REPLAY; + cv = check_for_replay(); + } + + /* Prepare the cv to sleep */ + if (cv) + ConditionVariablePrepareToSleep(cv); for (;;) { @@ -1667,9 +1680,12 @@ WalSndWaitForWal(XLogRecPtr loc) if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + ModifyWaitEvent(FeBeWaitSet, FeBeWaitSetSocketPos, wakeEvents, NULL); + ConditionVariableEventSleep(cv, RecoveryInProgress, FeBeWaitSet, NULL, + sleeptime, wait_event); } + ConditionVariableCancelSleep(); /* reactivate latch so WalSndLoop knows to continue */ SetLatch(MyLatch); return RecentFlushPtr; diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c index cb99cc6339..a10dcd4e61 100644 --- a/src/backend/utils/activity/wait_event.c +++ b/src/backend/utils/activity/wait_event.c @@ -466,6 +466,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) case WAIT_EVENT_WAL_RECEIVER_WAIT_START: event_name = "WalReceiverWaitStart"; break; + case WAIT_EVENT_WAL_SENDER_WAIT_REPLAY: + event_name = "WalSenderWaitReplay"; + break; case WAIT_EVENT_XACT_GROUP_UPDATE: event_name = "XactGroupUpdate"; break; diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 47c29350f5..2bfeaaa00f 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -15,6 +15,7 @@ #include "catalog/pg_control.h" #include "lib/stringinfo.h" #include "utils/timestamp.h" +#include "storage/condition_variable.h" /* * Recovery target type. @@ -155,4 +156,6 @@ extern void RecoveryRequiresIntParameter(const char *param_name, int currValue, extern void xlog_outdesc(StringInfo buf, XLogReaderState *record); +extern ConditionVariable *check_for_replay(void); + #endif /* XLOGRECOVERY_H */ diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 52bb3e2aae..2fd745fe72 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -13,6 +13,7 @@ #define _WALSENDER_H #include <signal.h> +#include "storage/condition_variable.h" /* * What to do with a snapshot in create replication slot command. diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h index 9ab23e1c4a..548ef41dca 100644 --- a/src/include/utils/wait_event.h +++ b/src/include/utils/wait_event.h @@ -131,6 +131,7 @@ typedef enum WAIT_EVENT_SYNC_REP, WAIT_EVENT_WAL_RECEIVER_EXIT, WAIT_EVENT_WAL_RECEIVER_WAIT_START, + WAIT_EVENT_WAL_SENDER_WAIT_REPLAY, WAIT_EVENT_XACT_GROUP_UPDATE } WaitEventIPC; -- 2.34.1 [text/plain] v2-0001-Introduce-ConditionVariableEventSleep.txt (5.8K, ../../[email protected]/3-v2-0001-Introduce-ConditionVariableEventSleep.txt) download | inline diff: From 0044078a540fcb2b5f5c728dcb7e4911b000d6d5 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Fri, 10 Mar 2023 10:57:22 +0000 Subject: [PATCH v99 4/7] Introduce-ConditionVariableEventSleep --- src/backend/storage/lmgr/condition_variable.c | 65 ++++++++++++++----- src/include/storage/condition_variable.h | 7 ++ 2 files changed, 57 insertions(+), 15 deletions(-) 89.0% src/backend/storage/lmgr/ 10.9% src/include/storage/ diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c index 7e2bbf46d9..af241e7317 100644 --- a/src/backend/storage/lmgr/condition_variable.c +++ b/src/backend/storage/lmgr/condition_variable.c @@ -97,7 +97,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, NULL, NULL, NULL, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +112,28 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, NULL, NULL, NULL, timeout, + wait_event_info); +} + +/* + * Wait for a condition variable to be signaled, a timeout to be reached, or a + * socket event in the given waitset. + * + * Returns true when timeout expires, otherwise returns false. + * + * See ConditionVariableSleep() for general usage. + */ +bool +ConditionVariableEventSleep(ConditionVariable *cv, bool (*cv_resume_waiting)(void), + WaitEventSet *waitset, + bool (*waitset_resume_waiting)(void), + long timeout, uint32 wait_event_info) { long cur_timeout = -1; instr_time start_time; instr_time cur_time; - int wait_events; /* * If the caller didn't prepare to sleep explicitly, then do so now and @@ -132,7 +150,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * If we are currently prepared to sleep on some other CV, we just cancel * that and prepare this one; see ConditionVariablePrepareToSleep. */ - if (cv_sleep_target != cv) + if (cv && cv_sleep_target != cv) { ConditionVariablePrepareToSleep(cv); return false; @@ -147,24 +165,29 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, INSTR_TIME_SET_CURRENT(start_time); Assert(timeout >= 0 && timeout <= INT_MAX); cur_timeout = timeout; - wait_events = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH; } - else - wait_events = WL_LATCH_SET | WL_EXIT_ON_PM_DEATH; while (true) { bool done = false; + WaitEvent cvEvent; + int nevents = 0; /* - * Wait for latch to be set. (If we're awakened for some other - * reason, the code below will cope anyway.) + * Wait for latch to be set, or other events which will be handled + * below. */ - (void) WaitLatch(MyLatch, wait_events, cur_timeout, wait_event_info); + if (waitset) + nevents = WaitEventSetWait(waitset, cur_timeout, &cvEvent, + 1, wait_event_info); /* Reset latch before examining the state of the wait list. */ ResetLatch(MyLatch); + /* If a socket event occurred, no need to check wait list. */ + if (nevents == 1 && (cvEvent.events & WL_SOCKET_MASK) != 0) + return true; + /* * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or @@ -180,13 +203,25 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * by something other than ConditionVariableSignal; though we don't * guarantee not to return spuriously, we'll avoid this obvious case. */ - SpinLockAcquire(&cv->mutex); - if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + + if (cv) { - done = true; - proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + SpinLockAcquire(&cv->mutex); + if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + { + done = true; + proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + } + SpinLockRelease(&cv->mutex); } - SpinLockRelease(&cv->mutex); + + /* If we are not waiting on a CV or don't want to wait anymore */ + if (!cv || (cv && cv_resume_waiting && !cv_resume_waiting())) + done = true; + + /* If we don't want to wait on the waitset anymore */ + if (waitset && waitset_resume_waiting && !waitset_resume_waiting()) + done = true; /* * Check for interrupts, and return spuriously if that caused the @@ -194,7 +229,7 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * waited for a different condition variable). */ CHECK_FOR_INTERRUPTS(); - if (cv != cv_sleep_target) + if (cv && cv != cv_sleep_target) done = true; /* We were signaled, so return */ diff --git a/src/include/storage/condition_variable.h b/src/include/storage/condition_variable.h index 589bdd323c..b9510caa17 100644 --- a/src/include/storage/condition_variable.h +++ b/src/include/storage/condition_variable.h @@ -22,6 +22,7 @@ #ifndef CONDITION_VARIABLE_H #define CONDITION_VARIABLE_H +#include "storage/latch.h" #include "storage/proclist_types.h" #include "storage/spin.h" @@ -56,6 +57,12 @@ extern void ConditionVariableInit(ConditionVariable *cv); extern void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info); extern bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info); +extern bool ConditionVariableEventSleep(ConditionVariable *cv, + bool (*cv_resume_waiting)(void), + WaitEventSet *cvEventSet, + bool (*waitset_resume_waiting)(void), + long timeout, + uint32 wait_event_info); extern void ConditionVariableCancelSleep(void); /* -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-30 05:45 Jeff Davis <[email protected]> parent: Jeff Davis <[email protected]> 1 sibling, 1 reply; 41+ messages in thread From: Jeff Davis @ 2023-03-30 05:45 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Thu, 2023-03-02 at 23:58 -0800, Jeff Davis wrote: > On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: > > In this case it looks easier to add the right API than to be sure > > about > > whether it's needed or not. > > I attached a sketch of one approach. I'm not very confident that it's > the right API or even that it works as I intended it, but if others > like the approach I can work on it some more. Another approach might be to extend WaitEventSets() to be able to wait on Condition Variables, rather than Condition Variables waiting on WaitEventSets. Thoughts? Regards, Jeff Davis ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-30 16:31 Masahiko Sawada <[email protected]> parent: Jeff Davis <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Masahiko Sawada @ 2023-03-30 16:31 UTC (permalink / raw) To: Jeff Davis <[email protected]>; +Cc: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On Thu, Mar 30, 2023 at 2:45 PM Jeff Davis <[email protected]> wrote: > > On Thu, 2023-03-02 at 23:58 -0800, Jeff Davis wrote: > > On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote: > > > In this case it looks easier to add the right API than to be sure > > > about > > > whether it's needed or not. > > > > I attached a sketch of one approach. I'm not very confident that it's > > the right API or even that it works as I intended it, but if others > > like the approach I can work on it some more. > > Another approach might be to extend WaitEventSets() to be able to wait > on Condition Variables, rather than Condition Variables waiting on > WaitEventSets. Thoughts? > +1 to extend CV. If we extend WaitEventSet() to be able to wait on CV, it would be able to make the code simple, but we would need to change both CV and WaitEventSet(). On Fri, Mar 10, 2023 at 8:34 PM Drouvot, Bertrand <[email protected]> wrote: > > I gave it a try, so please find attached v2-0001-Introduce-ConditionVariableEventSleep.txt (implementing the comments above) and 0004_new_API.txt to put the new API in the logical decoding on standby context. @@ -180,13 +203,25 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, * by something other than ConditionVariableSignal; though we don't * guarantee not to return spuriously, we'll avoid this obvious case. */ - SpinLockAcquire(&cv->mutex); - if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + + if (cv) { - done = true; - proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + SpinLockAcquire(&cv->mutex); + if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink)) + { + done = true; + proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink); + } + SpinLockRelease(&cv->mutex); } This change looks odd to me since it accepts cv being NULL in spite of calling ConditionVariableEventSleep() for cv. I think that this is because in 0004_new_API.txt, we use ConditionVariableEventSleep() in both not-in-recovery case and recovery-in-progress cases in WalSndWaitForWal() as follows: - WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_WAIT_WAL); + ModifyWaitEvent(FeBeWaitSet, FeBeWaitSetSocketPos, wakeEvents, NULL); + ConditionVariableEventSleep(cv, RecoveryInProgress, FeBeWaitSet, NULL, + sleeptime, wait_event); } But I don't think we need to use ConditionVariableEventSleep() in not-in-recovery cases. If I correctly understand the problem this patch wants to deal with, in logical decoding on standby cases, the walsender needs to be woken up on the following events: * condition variable * timeout * socket writable (if pq_is_send_pending() is true) (socket readable event should also be included to avoid wal_receiver_timeout BTW?) On the other hand, in not-in-recovery case, the events are: * socket readable * socket writable (if pq_is_send_pending() is true) * latch * timeout I think that we don't need to change for the latter case as WalSndWait() perfectly works. As for the former cases, since we need to wait for CV, timeout, or socket writable we can use ConditionVariableEventSleep(). Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-03-31 09:44 Jeff Davis <[email protected]> parent: Drouvot, Bertrand <[email protected]> 1 sibling, 2 replies; 41+ messages in thread From: Jeff Davis @ 2023-03-31 09:44 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Wed, 2023-03-08 at 11:25 +0100, Drouvot, Bertrand wrote: > - Maybe ConditionVariableEventSleep() should take care of the > “WaitEventSetWait returns 1 and cvEvent.event == WL_POSTMASTER_DEATH” > case? Thank you, done. I think the nearby line was also wrong, returning true when there was no timeout. I combined the lines and got rid of the early return so it can check the list and timeout condition like normal. Attached. > - Maybe ConditionVariableEventSleep() could accept and deal with the > CV being NULL? > I used it in the POC attached to handle logical decoding on the > primary server case. > One option should be to create a dedicated CV for that case though. I don't think it's a good idea to have a CV-based API that doesn't need a CV. Wouldn't that just be a normal WaitEventSet? > - In the POC attached I had to add this extra condition “(cv && > !RecoveryInProgress())” to avoid waiting on the timeout when there is > a promotion. > That makes me think that we may want to add 2 extra parameters (as 2 > functions returning a bool?) to ConditionVariableEventSleep() > to check whether or not we still want to test the socket or the CV > wake up in each loop iteration. That seems like a complex API. Would it work to signal the CV during promotion instead? > Also 3 additional remarks: > > 1) About InitializeConditionVariableWaitSet() and > ConditionVariableWaitSetCreate(): I'm not sure about the naming as > there is no CV yet (they "just" deal with WaitEventSet). It's a WaitEventSet that contains the conditions always required for any CV, and allows you to add in more. > 3) > > I wonder if there is no race conditions: ConditionVariableWaitSet is > being initialized with PGINVALID_SOCKET > as WL_LATCH_SET and might be also (if IsUnderPostmaster) be > initialized with PGINVALID_SOCKET as WL_EXIT_ON_PM_DEATH. > > So IIUC, the patch is introducing 2 new possible source of wake up. Those should be the same conditions already required by ConditionVariableTimedSleep() in master, right? Regards, Jeff Davis Attachments: [text/x-patch] v2-0001-Introduce-ConditionVariableEventSleep.patch (8.8K, ../../[email protected]/2-v2-0001-Introduce-ConditionVariableEventSleep.patch) download | inline diff: From 2f05cab9012950ae9290fccbb6366d50fc01553e Mon Sep 17 00:00:00 2001 From: Jeff Davis <[email protected]> Date: Wed, 1 Mar 2023 20:02:42 -0800 Subject: [PATCH v2] Introduce ConditionVariableEventSleep(). The new API takes a WaitEventSet which can include socket events. The WaitEventSet must have been created by ConditionVariableWaitSetCreate(), another new function, so that it includes the wait events necessary for a condition variable. --- src/backend/storage/lmgr/condition_variable.c | 106 ++++++++++++++++-- src/backend/storage/lmgr/proc.c | 6 + src/backend/utils/init/miscinit.c | 1 + src/include/storage/condition_variable.h | 10 ++ 4 files changed, 115 insertions(+), 8 deletions(-) diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c index 7e2bbf46d9..3dbfa7468b 100644 --- a/src/backend/storage/lmgr/condition_variable.c +++ b/src/backend/storage/lmgr/condition_variable.c @@ -27,9 +27,29 @@ #include "storage/spin.h" #include "utils/memutils.h" +#define ConditionVariableWaitSetLatchPos 0 + /* Initially, we are not prepared to sleep on any condition variable. */ static ConditionVariable *cv_sleep_target = NULL; +/* Used by ConditionVariableSleep() and ConditionVariableTimedSleep(). */ +static WaitEventSet *ConditionVariableWaitSet = NULL; + +/* + * Initialize the process-local condition variable WaitEventSet. + * + * This must be called once during startup of any process that can wait on + * condition variables, before it issues any ConditionVariableInit() calls. + */ +void +InitializeConditionVariableWaitSet(void) +{ + Assert(ConditionVariableWaitSet == NULL); + + ConditionVariableWaitSet = ConditionVariableWaitSetCreate( + TopMemoryContext, 0); +} + /* * Initialize a condition variable. */ @@ -40,6 +60,51 @@ ConditionVariableInit(ConditionVariable *cv) proclist_init(&cv->wakeup); } +/* + * Create a WaitEventSet for ConditionVariableEventSleep(). This should be + * used when the caller of ConditionVariableEventSleep() would like to wake up + * on either the condition variable signal or a socket event. For example: + * + * ConditionVariableInit(&cv); + * waitset = ConditionVariableWaitSetCreate(mcxt, 1); + * event_pos = AddWaitEventToSet(waitset, 0, sock, NULL, NULL); + * ... + * ConditionVariablePrepareToSleep(&cv); + * while (...condition not met...) + * { + * socket_wait_events = ... + * ModifyWaitEvent(waitset, event_pos, socket_wait_events, NULL); + * ConditionVariableEventSleep(&cv, waitset, ...); + * } + * ConditionVariableCancelSleep(); + * + * The waitset is created with the standard events for a condition variable, + * and room for adding n_socket_events additional socket events. The + * initially-filled event positions should not be modified, but added socket + * events can be modified. The same waitset can be used for multiple condition + * variables as long as the callers of ConditionVariableEventSleep() are + * interested in the same sockets. + */ +WaitEventSet * +ConditionVariableWaitSetCreate(MemoryContext mcxt, int n_socket_events) +{ + int latch_pos PG_USED_FOR_ASSERTS_ONLY; + int n_cv_events = IsUnderPostmaster ? 2 : 1; + int nevents = n_cv_events + n_socket_events; + WaitEventSet *waitset = CreateWaitEventSet(mcxt, nevents); + + latch_pos = AddWaitEventToSet(waitset, WL_LATCH_SET, PGINVALID_SOCKET, + MyLatch, NULL); + + if (IsUnderPostmaster) + AddWaitEventToSet(waitset, WL_EXIT_ON_PM_DEATH, PGINVALID_SOCKET, + NULL, NULL); + + Assert(latch_pos == ConditionVariableWaitSetLatchPos); + + return waitset; +} + /* * Prepare to wait on a given condition variable. * @@ -97,7 +162,8 @@ ConditionVariablePrepareToSleep(ConditionVariable *cv) void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) { - (void) ConditionVariableTimedSleep(cv, -1 /* no timeout */ , + (void) ConditionVariableEventSleep(cv, ConditionVariableWaitSet, + -1 /* no timeout */ , wait_event_info); } @@ -111,11 +177,27 @@ ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info) bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info) +{ + return ConditionVariableEventSleep(cv, ConditionVariableWaitSet, timeout, + wait_event_info); +} + +/* + * Wait for a condition variable to be signaled, a timeout to be reached, or a + * socket event in the given waitset. The waitset must have been created by + * ConditionVariableWaitSetCreate(). + * + * Returns true when timeout expires, otherwise returns false. + * + * See ConditionVariableSleep() for general usage. + */ +bool +ConditionVariableEventSleep(ConditionVariable *cv, WaitEventSet *waitset, + long timeout, uint32 wait_event_info) { long cur_timeout = -1; instr_time start_time; instr_time cur_time; - int wait_events; /* * If the caller didn't prepare to sleep explicitly, then do so now and @@ -147,24 +229,32 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, INSTR_TIME_SET_CURRENT(start_time); Assert(timeout >= 0 && timeout <= INT_MAX); cur_timeout = timeout; - wait_events = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH; } - else - wait_events = WL_LATCH_SET | WL_EXIT_ON_PM_DEATH; while (true) { bool done = false; + WaitEvent cvEvent; + int nevents; /* - * Wait for latch to be set. (If we're awakened for some other - * reason, the code below will cope anyway.) + * Wait for latch to be set, or other events which will be handled + * below. */ - (void) WaitLatch(MyLatch, wait_events, cur_timeout, wait_event_info); + nevents = WaitEventSetWait(waitset, cur_timeout, &cvEvent, + 1, wait_event_info); /* Reset latch before examining the state of the wait list. */ ResetLatch(MyLatch); + /* + * If the wakeup was due to a socket event or postmaster death, then + * we must return to the caller. + */ + if (nevents == 1 && + (cvEvent.events & (WL_SOCKET_MASK | WL_POSTMASTER_DEATH)) != 0) + done = true; + /* * If this process has been taken out of the wait list, then we know * that it has been signaled by ConditionVariableSignal (or diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 22b4278610..ae4a7aecd4 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -440,6 +440,9 @@ InitProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); @@ -596,6 +599,9 @@ InitAuxiliaryProcess(void) OwnLatch(&MyProc->procLatch); SwitchToSharedLatch(); + /* Initialize process-local condition variable support */ + InitializeConditionVariableWaitSet(); + /* now that we have a proc, report wait events to shared memory */ pgstat_set_wait_event_storage(&MyProc->wait_event_info); diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c index a604432126..0545044225 100644 --- a/src/backend/utils/init/miscinit.c +++ b/src/backend/utils/init/miscinit.c @@ -40,6 +40,7 @@ #include "postmaster/interrupt.h" #include "postmaster/pgarch.h" #include "postmaster/postmaster.h" +#include "storage/condition_variable.h" #include "storage/fd.h" #include "storage/ipc.h" #include "storage/latch.h" diff --git a/src/include/storage/condition_variable.h b/src/include/storage/condition_variable.h index 589bdd323c..94adb54b91 100644 --- a/src/include/storage/condition_variable.h +++ b/src/include/storage/condition_variable.h @@ -22,6 +22,7 @@ #ifndef CONDITION_VARIABLE_H #define CONDITION_VARIABLE_H +#include "storage/latch.h" #include "storage/proclist_types.h" #include "storage/spin.h" @@ -42,9 +43,14 @@ typedef union ConditionVariableMinimallyPadded char pad[CV_MINIMAL_SIZE]; } ConditionVariableMinimallyPadded; +extern void InitializeConditionVariableWaitSet(void); + /* Initialize a condition variable. */ extern void ConditionVariableInit(ConditionVariable *cv); +extern WaitEventSet *ConditionVariableWaitSetCreate(MemoryContext mcxt, + int n_socket_events); + /* * To sleep on a condition variable, a process should use a loop which first * checks the condition, exiting the loop if it is met, and then calls @@ -56,6 +62,10 @@ extern void ConditionVariableInit(ConditionVariable *cv); extern void ConditionVariableSleep(ConditionVariable *cv, uint32 wait_event_info); extern bool ConditionVariableTimedSleep(ConditionVariable *cv, long timeout, uint32 wait_event_info); +extern bool ConditionVariableEventSleep(ConditionVariable *cv, + WaitEventSet *cvEventSet, + long timeout, + uint32 wait_event_info); extern void ConditionVariableCancelSleep(void); /* -- 2.34.1 ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-04-02 21:25 Jeff Davis <[email protected]> parent: Jeff Davis <[email protected]> 1 sibling, 0 replies; 41+ messages in thread From: Jeff Davis @ 2023-04-02 21:25 UTC (permalink / raw) To: Drouvot, Bertrand <[email protected]>; Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Fri, 2023-03-31 at 02:44 -0700, Jeff Davis wrote: > Thank you, done. I think the nearby line was also wrong, returning > true > when there was no timeout. I combined the lines and got rid of the > early return so it can check the list and timeout condition like > normal. Attached. On second (third?) thought, I think I was right the first time. It passes the flag WL_EXIT_ON_PM_DEATH (included in the ConditionVariableWaitSet), so a WL_POSTMASTER_DEATH event should not be returned. Also, I think the early return is correct. The current code in ConditionVariableTimedSleep() still checks the wait list even if WaitLatch() returns WL_TIMEOUT (it ignores the return), but I don't see why it can't early return true. For a socket event in ConditionVariableEventSleep() I think it should early return false. Regards, Jeff Davis ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-04-02 21:35 Andres Freund <[email protected]> parent: Jeff Davis <[email protected]> 1 sibling, 1 reply; 41+ messages in thread From: Andres Freund @ 2023-04-02 21:35 UTC (permalink / raw) To: Jeff Davis <[email protected]>; +Cc: Drouvot, Bertrand <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2023-03-31 02:44:33 -0700, Jeff Davis wrote: > From 2f05cab9012950ae9290fccbb6366d50fc01553e Mon Sep 17 00:00:00 2001 > From: Jeff Davis <[email protected]> > Date: Wed, 1 Mar 2023 20:02:42 -0800 > Subject: [PATCH v2] Introduce ConditionVariableEventSleep(). > > The new API takes a WaitEventSet which can include socket events. The > WaitEventSet must have been created by > ConditionVariableWaitSetCreate(), another new function, so that it > includes the wait events necessary for a condition variable. Why not offer a function to add a CV to a WES? It seems somehow odd to require going through condition_variable.c to create a WES. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-04-02 22:15 Jeff Davis <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 1 reply; 41+ messages in thread From: Jeff Davis @ 2023-04-02 22:15 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Drouvot, Bertrand <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers On Sun, 2023-04-02 at 14:35 -0700, Andres Freund wrote: > Why not offer a function to add a CV to a WES? It seems somehow odd > to require > going through condition_variable.c to create a WES. I agree that it's a bit odd, but remember that after waiting on a CV's latch, it needs to re-insert itself into the CV's wait list. A WaitEventSetWait() can't do that, unless we move the details of re- adding to the wait list into latch.c. I considered that, but latch.c already implements the APIs for WaitEventSet and Latch, so it felt complex to also make it responsible for ConditionVariable. I'm open to suggestion. Regards, Jeff Davis ^ permalink raw reply [nested|flat] 41+ messages in thread
* Re: Minimal logical decoding on standbys @ 2023-04-02 22:29 Andres Freund <[email protected]> parent: Jeff Davis <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Andres Freund @ 2023-04-02 22:29 UTC (permalink / raw) To: Jeff Davis <[email protected]>; +Cc: Drouvot, Bertrand <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Alvaro Herrera <[email protected]>; Ibrar Ahmed <[email protected]>; Amit Khandekar <[email protected]>; [email protected]; tushar <[email protected]>; Rahila Syed <[email protected]>; pgsql-hackers Hi, On 2023-04-02 15:15:44 -0700, Jeff Davis wrote: > On Sun, 2023-04-02 at 14:35 -0700, Andres Freund wrote: > > Why not offer a function to add a CV to a WES? It seems somehow odd > > to require > > going through condition_variable.c to create a WES. > > I agree that it's a bit odd, but remember that after waiting on a CV's > latch, it needs to re-insert itself into the CV's wait list. > > A WaitEventSetWait() can't do that, unless we move the details of re- > adding to the wait list into latch.c. I considered that, but latch.c > already implements the APIs for WaitEventSet and Latch, so it felt > complex to also make it responsible for ConditionVariable. I agree that the *wait* has to go through condition_variable.c, but it doesn't seem right that creation of the WES needs to go through condition_variable.c. The only thing that ConditionVariableEventSleep() seems to require is that the WES is waiting for MyLatch. You don't even need a separate WES for that, the already existing WES should suffice. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 41+ messages in thread
* [PATCH v23 4/8] Row pattern recognition patch (planner). @ 2024-10-25 03:56 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 41+ messages in thread From: Tatsuo Ishii @ 2024-10-25 03:56 UTC (permalink / raw) --- src/backend/optimizer/plan/createplan.c | 24 +++++++++++++++----- src/backend/optimizer/plan/planner.c | 3 +++ src/backend/optimizer/plan/setrefs.c | 27 ++++++++++++++++++++++- src/backend/optimizer/prep/prepjointree.c | 9 ++++++++ src/include/nodes/plannodes.h | 19 ++++++++++++++++ 5 files changed, 76 insertions(+), 6 deletions(-) diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index f2ed0d81f6..1490ded185 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -290,9 +290,11 @@ static WindowAgg *make_windowagg(List *tlist, Index winref, int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations, int frameOptions, Node *startOffset, Node *endOffset, Oid startInRangeFunc, Oid endInRangeFunc, - Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, - List *runCondition, List *qual, bool topWindow, - Plan *lefttree); + Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, List *runCondition, + RPSkipTo rpSkipTo, List *patternVariable, List *patternRegexp, + List *defineClause, + List *defineInitial, + List *qual, bool topWindow, Plan *lefttree); static Group *make_group(List *tlist, List *qual, int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations, Plan *lefttree); @@ -2700,6 +2702,11 @@ create_windowagg_plan(PlannerInfo *root, WindowAggPath *best_path) wc->inRangeAsc, wc->inRangeNullsFirst, best_path->runCondition, + wc->rpSkipTo, + wc->patternVariable, + wc->patternRegexp, + wc->defineClause, + wc->defineInitial, best_path->qual, best_path->topwindow, subplan); @@ -6706,8 +6713,10 @@ make_windowagg(List *tlist, Index winref, int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations, int frameOptions, Node *startOffset, Node *endOffset, Oid startInRangeFunc, Oid endInRangeFunc, - Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, - List *runCondition, List *qual, bool topWindow, Plan *lefttree) + Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, List *runCondition, + RPSkipTo rpSkipTo, List *patternVariable, List *patternRegexp, List *defineClause, + List *defineInitial, + List *qual, bool topWindow, Plan *lefttree) { WindowAgg *node = makeNode(WindowAgg); Plan *plan = &node->plan; @@ -6733,6 +6742,11 @@ make_windowagg(List *tlist, Index winref, node->inRangeAsc = inRangeAsc; node->inRangeNullsFirst = inRangeNullsFirst; node->topWindow = topWindow; + node->rpSkipTo = rpSkipTo, + node->patternVariable = patternVariable; + node->patternRegexp = patternRegexp; + node->defineClause = defineClause; + node->defineInitial = defineInitial; plan->targetlist = tlist; plan->lefttree = lefttree; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 0f423e9684..24f0744c7c 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -879,6 +879,9 @@ subquery_planner(PlannerGlobal *glob, Query *parse, PlannerInfo *parent_root, EXPRKIND_LIMIT); wc->endOffset = preprocess_expression(root, wc->endOffset, EXPRKIND_LIMIT); + wc->defineClause = (List *) preprocess_expression(root, + (Node *) wc->defineClause, + EXPRKIND_TARGET); } parse->limitOffset = preprocess_expression(root, parse->limitOffset, diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 91c7c4fe2f..b5310e0d72 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -211,7 +211,6 @@ static List *set_windowagg_runcondition_references(PlannerInfo *root, List *runcondition, Plan *plan); - /***************************************************************************** * * SUBPLAN REFERENCES @@ -2495,6 +2494,32 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset) NRM_EQUAL, NUM_EXEC_QUAL(plan)); + /* + * Modifies an expression tree in each DEFINE clause so that all Var + * nodes's varno refers to OUTER_VAR. + */ + if (IsA(plan, WindowAgg)) + { + WindowAgg *wplan = (WindowAgg *) plan; + + if (wplan->defineClause != NIL) + { + foreach(l, wplan->defineClause) + { + TargetEntry *tle = (TargetEntry *) lfirst(l); + + tle->expr = (Expr *) + fix_upper_expr(root, + (Node *) tle->expr, + subplan_itlist, + OUTER_VAR, + rtoffset, + NRM_EQUAL, + NUM_EXEC_QUAL(plan)); + } + } + } + pfree(subplan_itlist); } diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c index 4d7f972caf..1dcfcf338d 100644 --- a/src/backend/optimizer/prep/prepjointree.c +++ b/src/backend/optimizer/prep/prepjointree.c @@ -2271,6 +2271,15 @@ perform_pullup_replace_vars(PlannerInfo *root, parse->returningList = (List *) pullup_replace_vars((Node *) parse->returningList, rvcontext); + foreach(lc, parse->windowClause) + { + WindowClause *wc = lfirst_node(WindowClause, lc); + + if (wc->defineClause != NIL) + wc->defineClause = (List *) + pullup_replace_vars((Node *) wc->defineClause, rvcontext); + } + if (parse->onConflict) { parse->onConflict->onConflictSet = (List *) diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 52f29bcdb6..294597461b 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -20,6 +20,7 @@ #include "lib/stringinfo.h" #include "nodes/bitmapset.h" #include "nodes/lockoptions.h" +#include "nodes/parsenodes.h" #include "nodes/primnodes.h" @@ -1099,6 +1100,24 @@ typedef struct WindowAgg /* nulls sort first for in_range tests? */ bool inRangeNullsFirst; + /* Row Pattern Recognition AFTER MACH SKIP clause */ + RPSkipTo rpSkipTo; /* Row Pattern Skip To type */ + + /* Row Pattern PATTERN variable name (list of String) */ + List *patternVariable; + + /* + * Row Pattern RPATTERN regular expression quantifier ('+' or ''. list of + * String) + */ + List *patternRegexp; + + /* Row Pattern DEFINE clause (list of TargetEntry) */ + List *defineClause; + + /* Row Pattern DEFINE variable initial names (list of String) */ + List *defineInitial; + /* * false for all apart from the WindowAgg that's closest to the root of * the plan -- 2.25.1 ----Next_Part(Fri_Oct_25_13_04_53_2024_648)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v23-0005-Row-pattern-recognition-patch-executor.patch" ^ permalink raw reply [nested|flat] 41+ messages in thread
end of thread, other threads:[~2024-10-25 03:56 UTC | newest] Thread overview: 41+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-01-05 21:15 Re: Minimal logical decoding on standbys Robert Haas <[email protected]> 2023-01-06 03:40 ` Andres Freund <[email protected]> 2023-01-10 08:33 ` Drouvot, Bertrand <[email protected]> 2023-01-11 07:32 ` Bharath Rupireddy <[email protected]> 2023-01-11 15:23 ` Drouvot, Bertrand <[email protected]> 2023-02-07 10:36 ` Drouvot, Bertrand <[email protected]> 2023-01-11 16:52 ` Andres Freund <[email protected]> 2023-01-27 12:09 ` Drouvot, Bertrand <[email protected]> 2023-01-18 10:24 ` Drouvot, Bertrand <[email protected]> 2023-01-19 02:46 ` Andres Freund <[email protected]> 2023-01-19 09:43 ` Drouvot, Bertrand <[email protected]> 2023-01-23 11:03 ` Drouvot, Bertrand <[email protected]> 2023-01-23 23:21 ` Melanie Plageman <[email protected]> 2023-01-24 14:59 ` Drouvot, Bertrand <[email protected]> 2023-01-30 16:18 ` Drouvot, Bertrand <[email protected]> 2023-01-24 00:46 ` Andres Freund <[email protected]> 2023-01-24 05:20 ` Drouvot, Bertrand <[email protected]> 2023-01-24 14:31 ` Drouvot, Bertrand <[email protected]> 2023-02-07 15:29 ` Drouvot, Bertrand <[email protected]> 2023-02-13 15:27 ` Drouvot, Bertrand <[email protected]> 2023-02-27 08:40 ` Drouvot, Bertrand <[email protected]> 2023-03-01 00:48 ` Jeff Davis <[email protected]> 2023-03-01 10:51 ` Drouvot, Bertrand <[email protected]> 2023-03-02 00:40 ` Jeff Davis <[email protected]> 2023-03-02 09:20 ` Drouvot, Bertrand <[email protected]> 2023-03-02 19:45 ` Jeff Davis <[email protected]> 2023-03-03 07:58 ` Jeff Davis <[email protected]> 2023-03-03 16:26 ` Drouvot, Bertrand <[email protected]> 2023-03-04 11:19 ` Drouvot, Bertrand <[email protected]> 2023-03-08 10:25 ` Drouvot, Bertrand <[email protected]> 2023-03-10 11:33 ` Drouvot, Bertrand <[email protected]> 2023-03-31 09:44 ` Jeff Davis <[email protected]> 2023-04-02 21:25 ` Jeff Davis <[email protected]> 2023-04-02 21:35 ` Andres Freund <[email protected]> 2023-04-02 22:15 ` Jeff Davis <[email protected]> 2023-04-02 22:29 ` Andres Freund <[email protected]> 2023-03-30 05:45 ` Jeff Davis <[email protected]> 2023-03-30 16:31 ` Masahiko Sawada <[email protected]> 2023-03-03 16:23 ` Drouvot, Bertrand <[email protected]> 2023-01-31 11:50 ` Drouvot, Bertrand <[email protected]> 2024-10-25 03:56 [PATCH v23 4/8] Row pattern recognition patch (planner). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox