($INBOX_DIR/description missing)help / color / mirror / Atom feed
[PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks 22+ messages / 2 participants [nested] [flat]
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks @ 2021-01-13 11:53 Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Bharath Rupireddy @ 2021-01-13 11:53 UTC (permalink / raw) Currently, in logical replication, relation map cache in the subscriber is not getting invalidated when anything changes in pg_subscription_rel or pg_subscription catalogues. So we end up not reading the latest system catalogues always in logicalrep_rel_open using GetSubscriptionRelState. To fix this, invalidate the relation map cache entries in invalidate_syncing_table_states and subscription_change_cb which are invalidation callbacks for pg_subscription_rel and pg_subscription catalogues respectively. --- src/backend/replication/logical/relation.c | 26 +++++++++++++++++++++ src/backend/replication/logical/tablesync.c | 3 +++ src/backend/replication/logical/worker.c | 3 +++ src/include/replication/logicalrelation.h | 1 + 4 files changed, 33 insertions(+) diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c index e861c0ff80..88ac772444 100644 --- a/src/backend/replication/logical/relation.c +++ b/src/backend/replication/logical/relation.c @@ -96,6 +96,32 @@ logicalrep_relmap_invalidate_cb(Datum arg, Oid reloid) } } +/* + * Invalidate relation map cache whenever syscache of pg_subscription_rel or + * pg_subscription gets changed. + */ +void +logicalrep_relmap_invalidate(void) +{ + LogicalRepRelMapEntry *entry; + HASH_SEQ_STATUS status; + + /* Just to be sure. */ + if (LogicalRepRelMap == NULL) + return; + + /* + * There is no way to find the cache entry for which the syscache has been + * changed, so we mark all the cache entries state as unknown. Because of + * this, in logicalrep_rel_open the cache entry state will be read from + * the system catalogue using GetSubscriptionRelState. + */ + hash_seq_init(&status, LogicalRepRelMap); + + while ((entry = (LogicalRepRelMapEntry *) hash_seq_search(&status)) != NULL) + entry->state = SUBREL_STATE_UNKNOWN; +} + /* * Initialize the relation map cache. */ diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c index 863d196fd7..6b29173a5c 100644 --- a/src/backend/replication/logical/tablesync.c +++ b/src/backend/replication/logical/tablesync.c @@ -257,6 +257,9 @@ void invalidate_syncing_table_states(Datum arg, int cacheid, uint32 hashvalue) { table_states_valid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c index f2b2549a51..347264e4a8 100644 --- a/src/backend/replication/logical/worker.c +++ b/src/backend/replication/logical/worker.c @@ -2487,6 +2487,9 @@ static void subscription_change_cb(Datum arg, int cacheid, uint32 hashvalue) { MySubscriptionValid = false; + + /* Invalidate relation map cache. */ + logicalrep_relmap_invalidate(); } /* diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h index 3f0b3deefb..f5e70b75b0 100644 --- a/src/include/replication/logicalrelation.h +++ b/src/include/replication/logicalrelation.h @@ -49,4 +49,5 @@ extern void logicalrep_rel_close(LogicalRepRelMapEntry *rel, extern void logicalrep_typmap_update(LogicalRepTyp *remotetyp); extern char *logicalrep_typmap_gettypname(Oid remoteid); +extern void logicalrep_relmap_invalidate(void); #endif /* LOGICALRELATION_H */ -- 2.30.0 --=-=-=-- ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Improve WALRead() to suck data directly from WAL buffers when possible @ 2023-02-07 10:42 Dilip Kumar <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Dilip Kumar @ 2023-02-07 10:42 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected] On Mon, Dec 26, 2022 at 2:20 PM Bharath Rupireddy <[email protected]> wrote: > I have gone through this patch, I have some comments (mostly cosmetic and comments) 1. + /* + * We have found the WAL buffer page holding the given LSN. Read from a + * pointer to the right offset within the page. + */ + memcpy(page, (XLogCtl->pages + idx * (Size) XLOG_BLCKSZ), + (Size) XLOG_BLCKSZ); From the above comments, it appears that we are reading from the exact pointer we are interested to read, but actually, we are reading the complete page. I think this comment needs to be fixed and we can also explain why we read the complete page here. 2. +static char * +GetXLogBufferForRead(XLogRecPtr ptr, TimeLineID tli, char *page) +{ + XLogRecPtr expectedEndPtr; + XLogRecPtr endptr; + int idx; + char *recptr = NULL; Generally, we use the name 'recptr' to represent XLogRecPtr type of variable, but in your case, it is actually data at that recptr, so better use some other name like 'buf' or 'buffer'. 3. + if ((recptr + nbytes) <= (page + XLOG_BLCKSZ)) + { + /* All the bytes are in one page. */ + memcpy(dst, recptr, nbytes); + dst += nbytes; + *read_bytes += nbytes; + ptr += nbytes; + nbytes = 0; + } + else if ((recptr + nbytes) > (page + XLOG_BLCKSZ)) + { + /* All the bytes are not in one page. */ + Size bytes_remaining; Why do you have this 'else if ((recptr + nbytes) > (page + XLOG_BLCKSZ))' check in the else part? why it is not directly else without a condition in 'if'? 4. +XLogReadFromBuffers(XLogRecPtr startptr, + TimeLineID tli, + Size count, + char *buf, + Size *read_bytes) I think we do not need 2 separate variables 'count' and '*read_bytes', just one variable for input/output is sufficient. The original value can always be stored in some temp variable instead of the function argument. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 22+ messages in thread
end of thread, other threads:[~2023-02-07 10:42 UTC | newest] Thread overview: 22+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2021-01-13 11:53 [PATCH v3 2/2] Invalidate relation map cache in subscriber syscache invalidation callbacks Bharath Rupireddy <[email protected]> 2023-02-07 10:42 Re: Improve WALRead() to suck data directly from WAL buffers when possible Dilip Kumar <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox