($INBOX_DIR/description missing)
help / color / mirror / Atom feed[PATCH v3 7/7] Allow to print raw parse tree.
27+ messages / 4 participants
[nested] [flat]
* [PATCH v3 7/7] Allow to print raw parse tree.
@ 2023-07-26 10:49 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-07-26 10:49 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..c01e90f735 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Wed_Jul_26_21_21_34_2023_317)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v4 7/7] Allow to print raw parse tree.
@ 2023-08-09 07:56 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-08-09 07:56 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..c01e90f735 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Wed_Aug__9_17_41_12_2023_134)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v5 7/7] Allow to print raw parse tree.
@ 2023-09-02 06:32 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-09-02 06:32 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index e4756f8be2..fc8efa915b 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Sat_Sep__2_15_52_35_2023_273)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v6 7/7] Allow to print raw parse tree.
@ 2023-09-12 05:22 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-09-12 05:22 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 21b9763183..3e3653816e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -651,6 +651,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Tue_Sep_12_15_18_43_2023_359)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v7 7/7] Allow to print raw parse tree.
@ 2023-09-22 04:53 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-09-22 04:53 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 21b9763183..3e3653816e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -651,6 +651,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Fri_Sep_22_14_16_40_2023_530)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v8 7/7] Allow to print raw parse tree.
@ 2023-09-25 05:01 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-09-25 05:01 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 21b9763183..3e3653816e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -651,6 +651,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Mon_Sep_25_14_26_30_2023_752)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v9 7/7] Allow to print raw parse tree.
@ 2023-10-04 05:51 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-10-04 05:51 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 21b9763183..3e3653816e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -651,6 +651,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Wed_Oct__4_15_03_28_2023_821)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v10 7/7] Allow to print raw parse tree.
@ 2023-10-22 02:22 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-10-22 02:22 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c900427ecf..fa5d862604 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -652,6 +652,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Sun_Oct_22_11_39_20_2023_140)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v11 7/7] Allow to print raw parse tree.
@ 2023-11-08 06:57 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-11-08 06:57 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 6a070b5d8c..beb528f526 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -652,6 +652,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Wed_Nov__8_16_37_05_2023_872)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v12 7/7] Allow to print raw parse tree.
@ 2023-12-04 11:23 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2023-12-04 11:23 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7298a187d1..b43afad0be 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -652,6 +652,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Fri_Dec__8_10_16_13_2023_489)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v13 8/8] Allow to print raw parse tree.
@ 2024-01-22 09:45 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-01-22 09:45 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1a34bd3715..68f5cf3667 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -652,6 +652,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Mon_Jan_22_19_26_18_2024_011)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v14 8/8] Allow to print raw parse tree.
@ 2024-02-28 13:59 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-02-28 13:59 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 59ab812d2e..e433ddafe0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -652,6 +652,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Thu_Feb_29_09_19_54_2024_640)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v15 8/8] Allow to print raw parse tree.
@ 2024-03-28 10:30 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-03-28 10:30 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 76f48b13d2..b93000adc4 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Thu_Mar_28_19_59_25_2024_076)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v16 8/8] Allow to print raw parse tree.
@ 2024-04-12 06:49 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-04-12 06:49 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 76f48b13d2..b93000adc4 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Fri_Apr_12_16_09_08_2024_262)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v17 8/8] Allow to print raw parse tree.
@ 2024-04-28 11:00 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-04-28 11:00 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2dff28afce..db6e91f5d6 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Sun_Apr_28_20_28_26_2024_444)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v18 8/8] Allow to print raw parse tree.
@ 2024-05-11 07:11 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-05-11 07:11 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2dff28afce..db6e91f5d6 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Sat_May_11_16_23_07_2024_789)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v19 8/8] Allow to print raw parse tree.
@ 2024-05-14 23:26 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-05-14 23:26 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2dff28afce..db6e91f5d6 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Wed_May_15_09_02_03_2024_008)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v20 8/8] Allow to print raw parse tree.
@ 2024-05-24 02:26 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-05-24 02:26 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 45a3794b8e..ecbf8e7999 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -653,6 +653,10 @@ pg_parse_query(const char *query_string)
}
#endif
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Fri_May_24_11_39_19_2024_763)----
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
@ 2024-07-09 09:39 shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: shveta malik @ 2024-07-09 09:39 UTC (permalink / raw)
To: Nisha Moond <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Dilip Kumar <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>; shveta malik <[email protected]>
On Fri, Jul 5, 2024 at 5:12 PM Nisha Moond <[email protected]> wrote:
>
> Thank you Ajin for reporting the issue, This is now fixed with the
> v4-0003 patch.
Please find v5 patch-set. Changes are:
1) patch003:
Added test cases for all resolvers (034_conflict_resolver.pl).
2) Patch004:
a) Emit error while resolving conflict if conflict resolver is default
'last_update_wins' but track_commit_timetsamp is not enabled.
b) Emit Warning during create and alter subscription when
'detect_conflict' is ON but 'track_commit_timetsamp' is not enabled.
c) Restrict start of pa worker if either max-clock-skew is configured
or conflict detection and resolution is enabled for a subscription.
d) Implement clock-skew delay/error when changes are applied from a
file (apply_spooled_messages).
e) Implement clock-skew delay while applying prepared changes (two
phase txns). The prepare-timestamp to be considered as base for
clock-skew handling as well as for last_update_win resolver.
<TODO: This needs to be analyzed and tested further to see if there is
any side effect of taking prepare-timestamp as base.>
Thanks Ajin fo working on 1.
Thanks Nisha for working on 2a,2b.
thanks
Shveta
Attachments:
[application/octet-stream] v5-0002-DDL-command-to-configure-Global-Conflict-Resolver.patch (26.5K, ../../CAJpy0uDbDzO4=FrvbtjweU+7P1fDRqU3oON1qaRTWfSjMDnmQg@mail.gmail.com/2-v5-0002-DDL-command-to-configure-Global-Conflict-Resolver.patch)
download | inline diff:
From 19c532794028e833ed2b210f5ddf6379cd5574e7 Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Wed, 3 Jul 2024 10:23:58 +0530
Subject: [PATCH v5 2/5] DDL command to configure Global Conflict Resolvers
This patch adds new DDL commands to configure resolvers
at global level:
a) To set global resolver for a given conflict_type:
SET CONFLICT RESOLVER 'conflict_resolver' FOR 'conflict_type'
b) To reset to default:
RESET CONFLICT RESOLVER FOR 'conflict_type'
A new catalog table pg_conflict has been created to store
global conflict_type and conflict_resolver configurations given by
above DDL command. Initially, this catalog table holds default
configuration for resolvers which is overwritten by the ones given
by the user in 'SET CONFLICT RESOLVER' command.
'RESET CONFLICT RESOLVER' command will reset the resolver to default.
This patch provides support to configure resolvers for below conflict
types:
- insert_exists
- update_differ
- update_missing
- delete_missing
- delete_differ
---
src/backend/parser/gram.y | 42 +++-
src/backend/replication/logical/conflict.c | 218 +++++++++++++++++-
src/backend/tcop/utility.c | 15 ++
src/include/catalog/Makefile | 4 +-
src/include/catalog/meson.build | 2 +
src/include/catalog/pg_conflict.dat | 21 ++
src/include/catalog/pg_conflict.h | 43 ++++
src/include/nodes/parsenodes.h | 12 +
src/include/parser/kwlist.h | 1 +
src/include/replication/conflict.h | 38 ++-
src/include/tcop/cmdtaglist.h | 1 +
.../regress/expected/conflict_resolver.out | 54 +++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/conflict_resolver.sql | 28 +++
src/tools/pgindent/typedefs.list | 2 +
15 files changed, 476 insertions(+), 7 deletions(-)
create mode 100644 src/include/catalog/pg_conflict.dat
create mode 100644 src/include/catalog/pg_conflict.h
create mode 100644 src/test/regress/expected/conflict_resolver.out
create mode 100644 src/test/regress/sql/conflict_resolver.sql
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a043fd4c66..42726fe3a6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -292,7 +292,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
AlterRoleStmt AlterRoleSetStmt AlterPolicyStmt AlterStatsStmt
AlterDefaultPrivilegesStmt DefACLAction
AnalyzeStmt CallStmt ClosePortalStmt ClusterStmt CommentStmt
- ConstraintsSetStmt CopyStmt CreateAsStmt CreateCastStmt
+ ConflictResolverStmt ConstraintsSetStmt CopyStmt CreateAsStmt CreateCastStmt
CreateDomainStmt CreateExtensionStmt CreateGroupStmt CreateOpClassStmt
CreateOpFamilyStmt AlterOpFamilyStmt CreatePLangStmt
CreateSchemaStmt CreateSeqStmt CreateStmt CreateStatsStmt CreateTableSpaceStmt
@@ -613,6 +613,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_check_option
%type <str> opt_provider security_label
+%type <str> conflict_type conflict_resolver
%type <target> xml_attribute_el
%type <list> xml_attribute_list xml_attributes
@@ -772,7 +773,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
RANGE READ REAL REASSIGN RECHECK RECURSIVE REF_P REFERENCES REFERENCING
REFRESH REINDEX RELATIVE_P RELEASE RENAME REPEATABLE REPLACE REPLICA
- RESET RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
+ RESET RESOLVER RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
ROUTINE ROUTINES ROW ROWS RULE
SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT
@@ -1039,6 +1040,7 @@ stmt:
| ClosePortalStmt
| ClusterStmt
| CommentStmt
+ | ConflictResolverStmt
| ConstraintsSetStmt
| CopyStmt
| CreateAmStmt
@@ -7266,7 +7268,41 @@ comment_text:
| NULL_P { $$ = NULL; }
;
+/*****************************************************************************
+ *
+ * SET CONFLICT RESOLVER <conflict_resolver> FOR <conflict_type>
+ * RESET CONFLICT RESOLVER FOR <conflict_type>
+ *
+ *****************************************************************************/
+
+ConflictResolverStmt:
+ SET CONFLICT RESOLVER conflict_resolver FOR conflict_type
+ {
+ ConflictResolverStmt *n = makeNode(ConflictResolverStmt);
+
+ n->conflict_resolver = $4;
+ n->conflict_type = $6;
+ n->isReset = false;
+ $$ = (Node *) n;
+ }
+ | RESET CONFLICT RESOLVER FOR conflict_type
+ {
+ ConflictResolverStmt *n = makeNode(ConflictResolverStmt);
+ n->conflict_type = $5;
+ n->isReset = true;
+ $$ = (Node *) n;
+ }
+ ;
+conflict_type:
+ Sconst { $$ = $1; }
+ | NULL_P { $$ = NULL; }
+ ;
+
+conflict_resolver:
+ Sconst { $$ = $1; }
+ | NULL_P { $$ = NULL; }
+ ;
/*****************************************************************************
*
* SECURITY LABEL [FOR <provider>] ON <object> IS <label>
@@ -17797,6 +17833,7 @@ unreserved_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
@@ -18428,6 +18465,7 @@ bare_label_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index b90e64b05b..93d9659752 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -1,13 +1,13 @@
/*-------------------------------------------------------------------------
* conflict.c
- * Functionality for detecting and logging conflicts.
+ * Functionality for detecting, logging and configuring conflicts.
*
* Copyright (c) 2024, PostgreSQL Global Development Group
*
* IDENTIFICATION
* src/backend/replication/logical/conflict.c
*
- * This file contains the code for detecting and logging conflicts on
+ * This file contains the code for detecting and handling conflicts on
* the subscriber during logical replication.
*-------------------------------------------------------------------------
*/
@@ -15,10 +15,19 @@
#include "postgres.h"
#include "access/commit_ts.h"
+#include "access/genam.h"
+#include "access/htup_details.h"
+#include "access/skey.h"
+#include "access/table.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_conflict.h"
#include "replication/conflict.h"
#include "replication/origin.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/timestamp.h"
const char *const ConflictTypeNames[] = {
[CT_INSERT_EXISTS] = "insert_exists",
@@ -28,6 +37,54 @@ const char *const ConflictTypeNames[] = {
[CT_DELETE_DIFFER] = "delete_differ"
};
+const char *const ConflictResolverNames[] = {
+ [CR_REMOTE_APPLY] = "remote_apply",
+ [CR_KEEP_LOCAL] = "keep_local",
+ [CR_APPLY_OR_SKIP] = "apply_or_skip",
+ [CR_APPLY_OR_ERROR] = "apply_or_error",
+ [CR_SKIP] = "skip",
+ [CR_ERROR] = "error"
+};
+
+#define CONFLICT_TYPE_MAX_RESOLVERS 4
+
+/*
+ * Valid conflict resolvers for each conflict type.
+ *
+ * XXX: If we do not want to maintain different resolvers such as
+ * apply_or_skip and apply_or_error for update_missing conflict,
+ * then we can retain remote_apply and keep_local only. Then these
+ * resolvers in context of update_missing will mean:
+ *
+ * keep_local: do not apply the update as INSERT.
+ * remote_apply: apply the update as INSERT. If we could not apply,
+ * then log and skip.
+ *
+ * Similarly SKIP can be replaced with KEEP_LOCAL for both update_missing
+ * and delete_missing conflicts. For missing rows, 'SKIP' sounds more user
+ * friendly name for a resolver and thus has been added here.
+ */
+const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
+ [CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+};
+
+/*
+ * Default conflict resolver for each conflict type.
+ * If this changes, change it in pg_conflict.dat as well.
+ */
+const int ConflictTypeDefaultResolvers[] = {
+ [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
+ [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
+ [CT_DELETE_MISSING] = CR_SKIP,
+ [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+
+};
+
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
TransactionId localxmin,
@@ -35,6 +92,7 @@ static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
TimestampTz localts,
TupleTableSlot *conflictslot);
+
/*
* Get the xmin and commit timestamp data (origin and timestamp) associated
* with the provided local tuple.
@@ -189,3 +247,159 @@ build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
return conflict_row;
}
+
+/*
+ * Validate the conflict type and resolver.
+ *
+ * Return ConflictType enum value corresponding to given conflict_type.
+ */
+static ConflictType
+validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver,
+ bool isReset)
+{
+ ConflictType type;
+ ConflictResolver resolver;
+ bool valid = false;
+ int i;
+
+ /* Check conflict type validity */
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ if (strcmp(ConflictTypeNames[type], conflict_type) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict type", conflict_type));
+
+ /* If it is RESET statement, we are done */
+ if (isReset)
+ return type;
+
+ /* Reset */
+ valid = false;
+
+ /* Check conflict resolver validity. */
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_resolver) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ /*
+ * XXX: Shall we give supported resolvers list for given type in error
+ * hint?
+ */
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver", conflict_resolver));
+
+ /* Reset */
+ valid = false;
+
+ /* Check if conflict resolver is a valid one for the given conflict type */
+ for (i = 0; i < CONFLICT_TYPE_MAX_RESOLVERS; i++)
+ {
+ if (ConflictTypeResolverMap[type][i] == resolver)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ /*
+ * XXX: Shall we give supported resolvers list for given type in error
+ * hint?
+ */
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver for conflict type %s",
+ conflict_resolver,
+ conflict_type));
+ return type;
+}
+
+/*
+ * Execute Conflict Resolver Stmt
+ *
+ * Validate conflict_type and conflict_resolver given by the user
+ * and save the info in pg_conflict system catalog.
+ */
+void
+ExecConflictResolverStmt(ConflictResolverStmt *stmt)
+{
+ Relation pg_conflict;
+ Datum values[Natts_pg_conflict];
+ bool nulls[Natts_pg_conflict];
+ bool replaces[Natts_pg_conflict];
+ ScanKeyData keys[1];
+ SysScanDesc scan;
+ HeapTuple oldtup;
+ HeapTuple newtup = NULL;
+ ConflictType type;
+
+ type = validate_conflict_type_and_resolver(stmt->conflict_type,
+ stmt->conflict_resolver,
+ stmt->isReset);
+
+
+ /* Prepare to update a tuple in pg_conflict system catalog */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_conflict_conftype - 1] =
+ CStringGetTextDatum(stmt->conflict_type);
+
+ if (stmt->isReset)
+ {
+ /* Set resolver to default */
+ ConflictResolver resolver = ConflictTypeDefaultResolvers[type];
+
+ values[Anum_pg_conflict_confres - 1] =
+ CStringGetTextDatum(ConflictResolverNames[resolver]);
+ }
+ else
+ values[Anum_pg_conflict_confres - 1] =
+ CStringGetTextDatum(stmt->conflict_resolver);
+
+ /* Use the index to search for a matching old tuple */
+ ScanKeyInit(&keys[0],
+ Anum_pg_conflict_conftype,
+ BTEqualStrategyNumber, F_TEXTEQ,
+ values[Anum_pg_conflict_conftype - 1]);
+
+ pg_conflict = table_open(ConflictResRelationId, RowExclusiveLock);
+
+ scan = systable_beginscan(pg_conflict, ConflictTypeIndexId, true,
+ NULL, 1, keys);
+
+ oldtup = systable_getnext(scan);
+ if (HeapTupleIsValid(oldtup))
+ {
+ replaces[Anum_pg_conflict_confres - 1] = true;
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(pg_conflict),
+ values, nulls, replaces);
+ CatalogTupleUpdate(pg_conflict, &oldtup->t_self, newtup);
+ }
+ else
+ Assert(oldtup); /* We must have a tuple for given
+ * conflict_type */
+
+ systable_endscan(scan);
+
+ if (newtup != NULL)
+ heap_freetuple(newtup);
+
+ table_close(pg_conflict, RowExclusiveLock);
+}
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index fa66b8017e..4ddf4aaff0 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -59,6 +59,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/conflict.h"
#include "rewrite/rewriteDefine.h"
#include "storage/fd.h"
#include "tcop/utility.h"
@@ -163,6 +164,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_AlterUserMappingStmt:
case T_CommentStmt:
case T_CompositeTypeStmt:
+ case T_ConflictResolverStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
case T_CreateConversionStmt:
@@ -1062,6 +1064,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
break;
}
+ case T_ConflictResolverStmt:
+ /* no event triggers for conflict types and resolvers */
+ ExecConflictResolverStmt((ConflictResolverStmt *) parsetree);
+ break;
+
default:
/* All other statement types have event trigger support */
ProcessUtilitySlow(pstate, pstmt, queryString,
@@ -2659,6 +2666,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_COMMENT;
break;
+ case T_ConflictResolverStmt:
+ tag = CMDTAG_CONFLICT_RESOLVER;
+ break;
+
case T_SecLabelStmt:
tag = CMDTAG_SECURITY_LABEL;
break;
@@ -3334,6 +3345,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_ConflictResolverStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_SecLabelStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile
index 167f91a6e3..8619d73e5a 100644
--- a/src/include/catalog/Makefile
+++ b/src/include/catalog/Makefile
@@ -81,7 +81,8 @@ CATALOG_HEADERS := \
pg_publication_namespace.h \
pg_publication_rel.h \
pg_subscription.h \
- pg_subscription_rel.h
+ pg_subscription_rel.h \
+ pg_conflict.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h)
@@ -97,6 +98,7 @@ POSTGRES_BKI_DATA = \
pg_cast.dat \
pg_class.dat \
pg_collation.dat \
+ pg_conflict.dat \
pg_conversion.dat \
pg_database.dat \
pg_language.dat \
diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index f70d1daba5..4d6732a303 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -69,6 +69,7 @@ catalog_headers = [
'pg_publication_rel.h',
'pg_subscription.h',
'pg_subscription_rel.h',
+ 'pg_conflict.h',
]
# The .dat files we need can just be listed alphabetically.
@@ -81,6 +82,7 @@ bki_data = [
'pg_cast.dat',
'pg_class.dat',
'pg_collation.dat',
+ 'pg_conflict.dat',
'pg_conversion.dat',
'pg_database.dat',
'pg_language.dat',
diff --git a/src/include/catalog/pg_conflict.dat b/src/include/catalog/pg_conflict.dat
new file mode 100644
index 0000000000..8d6183e3f2
--- /dev/null
+++ b/src/include/catalog/pg_conflict.dat
@@ -0,0 +1,21 @@
+#----------------------------------------------------------------------
+#
+# pg_conflict.dat
+# Initial contents of the pg_conflict system catalog.
+#
+# Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_conflict.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ conftype => 'insert_exists', confres => 'remote_apply' },
+{ conftype => 'update_differ', confres => 'remote_apply' },
+{ conftype => 'update_missing', confres => 'apply_or_skip' },
+{ conftype => 'delete_missing', confres => 'skip' },
+{ conftype => 'delete_differ', confres => 'remote_apply' }
+
+]
diff --git a/src/include/catalog/pg_conflict.h b/src/include/catalog/pg_conflict.h
new file mode 100644
index 0000000000..e3fe3e6d30
--- /dev/null
+++ b/src/include/catalog/pg_conflict.h
@@ -0,0 +1,43 @@
+/* -------------------------------------------------------------------------
+ *
+ * pg_conflict.h
+ * definition of the "global conflict detection" system catalog
+ * (pg_conflict)
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_conflict.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ * -------------------------------------------------------------------------
+ */
+#ifndef PG_CONFLICT_H
+#define PG_CONFLICT_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_conflict_d.h"
+
+/* ----------------
+ * pg_conflict definition. cpp turns this into
+ * typedef struct FormData_pg_conflict
+ * ----------------
+ */
+CATALOG(pg_conflict,8688,ConflictResRelationId)
+{
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text conftype BKI_FORCE_NOT_NULL; /* conflict type */
+ text confres BKI_FORCE_NOT_NULL; /* conflict resolver */
+#endif
+} FormData_pg_conflict;
+
+DECLARE_TOAST(pg_conflict, 8689, 8690);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_conflict_type_index, 8691, ConflictTypeIndexId, pg_conflict, btree(conftype text_ops));
+
+MAKE_SYSCACHE(CONFLICTTYPE, pg_conflict_type_index, 16);
+
+#endif /* PG_CONFLICT_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 85a62b538e..1d41169d5e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3262,6 +3262,18 @@ typedef struct CommentStmt
char *comment; /* Comment to insert, or NULL to remove */
} CommentStmt;
+/* ----------------------
+ * CONFLICT RESOLVER Statement
+ * ----------------------
+ */
+typedef struct ConflictResolverStmt
+{
+ NodeTag type;
+ char *conflict_type;
+ char *conflict_resolver;
+ bool isReset;
+} ConflictResolverStmt;
+
/* ----------------------
* SECURITY LABEL Statement
* ----------------------
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f7fe834cf4..21e5cb162d 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -377,6 +377,7 @@ PG_KEYWORD("repeatable", REPEATABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replace", REPLACE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replica", REPLICA, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("reset", RESET, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("resolver", RESOLVER, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restart", RESTART, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restrict", RESTRICT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("return", RETURN, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index a9f521aaca..c6023e0914 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -10,6 +10,7 @@
#define CONFLICT_H
#include "access/xlogdefs.h"
+#include "catalog/objectaddress.h"
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
#include "utils/relcache.h"
@@ -36,6 +37,41 @@ typedef enum
CT_DELETE_DIFFER,
} ConflictType;
+/* Min and max conflict type */
+#define CT_MIN CT_INSERT_EXISTS
+#define CT_MAX CT_DELETE_DIFFER
+
+/*
+ * Conflict resolvers that can be used to resolve various conflicts.
+ *
+ * See ConflictTypeResolverMap in conflcit.c to find out which all
+ * resolvers are supported for each conflict type.
+ */
+typedef enum ConflictResolver
+{
+ /* Apply the remote change */
+ CR_REMOTE_APPLY = 1,
+
+ /* Keep the local change */
+ CR_KEEP_LOCAL,
+
+ /* Apply the remote change; skip if it can not be applied */
+ CR_APPLY_OR_SKIP,
+
+ /* Apply the remote change; emit error if it can not be applied */
+ CR_APPLY_OR_ERROR,
+
+ /* Skip applying the change */
+ CR_SKIP,
+
+ /* Error out */
+ CR_ERROR,
+} ConflictResolver;
+
+/* Min and max conflict resolver */
+#define CR_MIN CR_REMOTE_APPLY
+#define CR_MAX CR_ERROR
+
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
extern void ReportApplyConflict(int elevel, ConflictType type,
@@ -43,5 +79,5 @@ extern void ReportApplyConflict(int elevel, ConflictType type,
TransactionId localxmin, RepOriginId localorigin,
TimestampTz localts, TupleTableSlot *conflictslot);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
-
+extern void ExecConflictResolverStmt(ConflictResolverStmt *stmt);
#endif
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index 7fdcec6dd9..562478e232 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -80,6 +80,7 @@ PG_CMDTAG(CMDTAG_CLUSTER, "CLUSTER", false, false, false)
PG_CMDTAG(CMDTAG_COMMENT, "COMMENT", true, false, false)
PG_CMDTAG(CMDTAG_COMMIT, "COMMIT", false, false, false)
PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
+PG_CMDTAG(CMDTAG_CONFLICT_RESOLVER, "CONFLICT RESOLVER", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
diff --git a/src/test/regress/expected/conflict_resolver.out b/src/test/regress/expected/conflict_resolver.out
new file mode 100644
index 0000000000..53ecb430fa
--- /dev/null
+++ b/src/test/regress/expected/conflict_resolver.out
@@ -0,0 +1,54 @@
+--check default global resolvers in system catalog
+select * from pg_conflict order by conftype;
+ conftype | confres
+----------------+---------------
+ delete_differ | remote_apply
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | remote_apply
+ update_missing | apply_or_skip
+(5 rows)
+
+--
+-- Test of SET/RESET CONFLICT RESOLVER with invalid names
+--
+SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
+ERROR: aaaa is not a valid conflict type
+SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
+ERROR: bbbbb is not a valid conflict resolver
+SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
+ERROR: remote_apply is not a valid conflict resolver for conflict type delete_missing
+RESET CONFLICT RESOLVER for 'ct'; -- fail
+ERROR: ct is not a valid conflict type
+--
+-- Test of SET/RESET CONFLICT RESOLVER with valid names
+--
+SET CONFLICT RESOLVER 'error' for 'delete_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
+SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
+SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
+--check new resolvers are saved
+select * from pg_conflict order by conftype;
+ conftype | confres
+----------------+----------------
+ delete_differ | keep_local
+ delete_missing | error
+ insert_exists | keep_local
+ update_differ | keep_local
+ update_missing | apply_or_error
+(5 rows)
+
+RESET CONFLICT RESOLVER for 'delete_missing';
+RESET CONFLICT RESOLVER for 'insert_exists';
+--check resolvers are reset to default for delete_missing and insert_exists
+select * from pg_conflict order by conftype;
+ conftype | confres
+----------------+----------------
+ delete_differ | keep_local
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | keep_local
+ update_missing | apply_or_error
+(5 rows)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 2429ec2bba..76ea2d359b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -48,7 +48,7 @@ test: create_index create_index_spgist create_view index_including index_includi
# ----------
# Another group of parallel tests
# ----------
-test: create_aggregate create_function_sql create_cast constraints triggers select inherit typed_table vacuum drop_if_exists updatable_views roleattributes create_am hash_func errors infinite_recurse
+test: create_aggregate create_function_sql create_cast conflict_resolver constraints triggers select inherit typed_table vacuum drop_if_exists updatable_views roleattributes create_am hash_func errors infinite_recurse
# ----------
# sanity_check does a vacuum, affecting the sort order of SELECT *
diff --git a/src/test/regress/sql/conflict_resolver.sql b/src/test/regress/sql/conflict_resolver.sql
new file mode 100644
index 0000000000..f83d14b229
--- /dev/null
+++ b/src/test/regress/sql/conflict_resolver.sql
@@ -0,0 +1,28 @@
+--check default global resolvers in system catalog
+select * from pg_conflict order by conftype;
+
+--
+-- Test of SET/RESET CONFLICT RESOLVER with invalid names
+--
+SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
+SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
+SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
+RESET CONFLICT RESOLVER for 'ct'; -- fail
+
+--
+-- Test of SET/RESET CONFLICT RESOLVER with valid names
+--
+SET CONFLICT RESOLVER 'error' for 'delete_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
+SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
+SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
+
+--check new resolvers are saved
+select * from pg_conflict order by conftype;
+
+RESET CONFLICT RESOLVER for 'delete_missing';
+RESET CONFLICT RESOLVER for 'insert_exists';
+
+--check resolvers are reset to default for delete_missing and insert_exists
+select * from pg_conflict order by conftype;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3c79446ebb..c920a59525 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -467,6 +467,8 @@ ConditionVariableMinimallyPadded
ConditionalStack
ConfigData
ConfigVariable
+ConflictResolver
+ConflictResolverStmt
ConflictType
ConnCacheEntry
ConnCacheKey
--
2.34.1
[application/octet-stream] v5-0003-Conflict-resolvers-for-insert-update-and-delete.patch (52.8K, ../../CAJpy0uDbDzO4=FrvbtjweU+7P1fDRqU3oON1qaRTWfSjMDnmQg@mail.gmail.com/3-v5-0003-Conflict-resolvers-for-insert-update-and-delete.patch)
download | inline diff:
From 51ca8c0c0568510807270f41d2ebde95943bf384 Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Wed, 3 Jul 2024 15:46:58 +0530
Subject: [PATCH v5 3/5] Conflict resolvers for insert, update, and delete
This patch introduces support for handling conflicts with the following resolutions:
- For INSERT conflicts:
- insert_exists: remote_apply, keep_local, error
- For UPDATE conflicts:
- update_differ: remote_apply, keep_local, error
- update_missing: apply_or_skip, apply_or_error, skip, error
- For DELETE conflicts:
- delete_missing: skip, error
- delete_differ: remote_apply, keep_local, error
Each of the conflict type now has defined actions to manage data synchronization
between Publisher and subscriber databases.
---
src/backend/executor/execReplication.c | 80 ++-
src/backend/replication/logical/conflict.c | 191 ++++++-
src/backend/replication/logical/worker.c | 361 +++++++++----
src/include/executor/executor.h | 5 +-
src/include/replication/conflict.h | 19 +-
src/test/subscription/meson.build | 1 +
src/test/subscription/t/029_on_error.pl | 7 +
.../subscription/t/034_conflict_resolver.pl | 475 ++++++++++++++++++
8 files changed, 1013 insertions(+), 126 deletions(-)
create mode 100755 src/test/subscription/t/034_conflict_resolver.pl
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index f01927a933..3e8d768214 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -566,7 +566,8 @@ retry:
*/
void
ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot)
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot)
{
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
@@ -602,21 +603,82 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
+ /*
+ * If caller has passed non null conflictslot, check all the unique
+ * indexes for potential conflicts. If the configured resolver is in
+ * favour of apply, give the conflicted tuple information in
+ * conflictslot.
+ */
+ if (conflictslot)
+ {
+ foreach_oid(uniqueidx, conflictindexes)
+ {
+ /*
+ * Return if any conflict is found other than one with 'ERROR'
+ * resolver configured. In case of 'ERROR' resolver, emit
+ * error here; otherwise return to caller for resolutions.
+ */
+ if (FindConflictTuple(resultRelInfo, estate, uniqueidx,
+ slot, &(*conflictslot)))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+ ConflictResolver resolver;
+ bool apply_remote = false;
+
+ GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
+ resolver = GetConflictResolver(*conflictslot, rel,
+ CT_INSERT_EXISTS,
+ &apply_remote, NULL, uniqueidx,
+ xmin, origin, committs);
+
+ ReportApplyConflict(LOG, CT_INSERT_EXISTS, resolver, rel,
+ uniqueidx, xmin, origin, committs,
+ *conflictslot);
+
+ /* Nothing to apply, free the resources */
+ if (!apply_remote)
+ {
+ ExecDropSingleTupleTableSlot(*conflictslot);
+ *conflictslot = NULL;
+ }
+
+ return;
+ }
+ }
+ }
+
+
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
- conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
-
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, false,
conflictindexes, &conflict,
conflictindexes, false);
- /* Re-check all the unique indexes for potential conflicts */
+ /*
+ * Re-check all the unique indexes for potential conflicts.
+ *
+ * The Reason for doing the check again is:
+ *
+ * 1. If the remote change violated multiple unique constraints, the
+ * current resolution cannot solve it, so we still need to report this
+ * conflict after resolution.
+ *
+ * 2. If the local data is changed immediately after converting the
+ * insert to update but before updating the data, then the conflict
+ * can still happen, and we may need to report it again.
+ *
+ * XXX: Needs further review and discussion on usefulness of this
+ * repeated call.
+ */
foreach_oid(uniqueidx, (conflict ? conflictindexes : NIL))
{
- TupleTableSlot *conflictslot;
/*
* Reports the conflict if any.
@@ -631,15 +693,15 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
* cost of finding the tuple should be acceptable in this case.
*/
if (list_member_oid(recheckIndexes, uniqueidx) &&
- FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &conflictslot))
+ FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &(*conflictslot)))
{
RepOriginId origin;
TimestampTz committs;
TransactionId xmin;
- GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
- ReportApplyConflict(ERROR, CT_INSERT_EXISTS, rel, uniqueidx,
- xmin, origin, committs, conflictslot);
+ GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
+ ReportApplyConflict(ERROR, CT_INSERT_EXISTS, 0, rel, uniqueidx,
+ xmin, origin, committs, *conflictslot);
}
}
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 93d9659752..5a7f0e0b53 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -21,13 +21,17 @@
#include "access/table.h"
#include "catalog/indexing.h"
#include "catalog/pg_conflict.h"
+#include "executor/executor.h"
#include "replication/conflict.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "replication/origin.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/timestamp.h"
+#include "utils/syscache.h"
const char *const ConflictTypeNames[] = {
[CT_INSERT_EXISTS] = "insert_exists",
@@ -86,8 +90,9 @@ const int ConflictTypeDefaultResolvers[] = {
};
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
-static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin,
+static int errdetail_apply_conflict(ConflictType type,
+ ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
RepOriginId localorigin,
TimestampTz localts,
TupleTableSlot *conflictslot);
@@ -126,13 +131,13 @@ GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
}
/*
- * Report a conflict when applying remote changes.
+ * Report conflict and resolution applied while applying remote changes.
*/
void
-ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
- Oid conflictidx, TransactionId localxmin,
- RepOriginId localorigin, TimestampTz localts,
- TupleTableSlot *conflictslot)
+ReportApplyConflict(int elevel, ConflictType type, ConflictResolver resolver,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot)
{
ereport(elevel,
errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
@@ -140,7 +145,7 @@ ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
ConflictTypeNames[type],
get_namespace_name(RelationGetNamespace(localrel)),
RelationGetRelationName(localrel)),
- errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
+ errdetail_apply_conflict(type, resolver, conflictidx, localxmin, localorigin,
localts, conflictslot));
}
@@ -176,12 +181,13 @@ InitConflictIndexes(ResultRelInfo *relInfo)
}
/*
- * Add an errdetail() line showing conflict detail.
+ * Add an errdetail() line showing conflict and resolution details.
*/
static int
-errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot)
+errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot)
{
switch (type)
{
@@ -195,27 +201,35 @@ errdetail_apply_conflict(ConflictType type, Oid conflictidx,
char *index_value = build_index_value_desc(conflictidx,
conflictslot);
- if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
- else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ if (!resolver || (resolver == CR_ERROR))
+ {
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ return errdetail("Key already exists. Applying resolution method \"%s\"", ConflictResolverNames[resolver]);
}
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\"",
+ localorigin, localxmin, timestamptz_to_str(localts), ConflictResolverNames[resolver]);
case CT_UPDATE_MISSING:
- return errdetail("Did not find the row to be updated.");
+ return errdetail("Did not find the row to be updated. Applying resolution method \"%s\"",
+ ConflictResolverNames[resolver]);
case CT_DELETE_MISSING:
- return errdetail("Did not find the row to be deleted.");
+ return errdetail("Did not find the row to be deleted. Applying resolution method \"%s\"",
+ ConflictResolverNames[resolver]);
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\"",
+ localorigin, localxmin, timestamptz_to_str(localts),
+ ConflictResolverNames[resolver]);
}
return 0; /* silence compiler warning */
@@ -329,6 +343,67 @@ validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver
return type;
}
+/*
+ * Get the global conflict resolver configured for given conflict type
+ */
+static ConflictResolver
+get_conflict_resolver_internal(ConflictType type)
+{
+
+ ConflictResolver resolver;
+ HeapTuple tuple;
+ Datum datum;
+ char *conflict_res;
+
+ /*
+ * XXX: Currently, we fetch the conflict resolver from cache for each
+ * conflict detection. If needed, we can keep the info in global variable
+ * and fetch from cache only once after cache invalidation.
+ */
+ tuple = SearchSysCache1(CONFLICTTYPE,
+ CStringGetTextDatum(ConflictTypeNames[type]));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for conflict type %u", type);
+
+ datum = SysCacheGetAttrNotNull(CONFLICTTYPE,
+ tuple, Anum_pg_conflict_confres);
+
+ conflict_res = TextDatumGetCString(datum);
+
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_res) == 0)
+ break;
+ }
+
+ ReleaseSysCache(tuple);
+ return resolver;
+}
+
+/*
+ * Check if a full tuple can be created from the new tuple.
+ * Return true if yes, false otherwise.
+ */
+static bool
+can_create_full_tuple(Relation localrel,
+ LogicalRepTupleData *newtup)
+{
+ int i;
+ int local_att = RelationGetNumberOfAttributes(localrel);
+
+ if (newtup->ncols != local_att)
+ return false;
+
+ for (i = 0; i < newtup->ncols; i++)
+ {
+ if (newtup->colstatus[i] == LOGICALREP_COLUMN_UNCHANGED)
+ return false;
+ }
+
+ return true;
+}
+
/*
* Execute Conflict Resolver Stmt
*
@@ -403,3 +478,65 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
table_close(pg_conflict, RowExclusiveLock);
}
+
+/*
+ * Find the global resolver for the given conflict type.
+ * It emits error in case the resolver is set to 'ERROR'.
+ *
+ * Set 'apply_remote' to true if remote tuple should be applied,
+ * false otherwise.
+ */
+ConflictResolver
+GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
+ ConflictType type, bool *apply_remote,
+ LogicalRepTupleData *newtup,
+ Oid conflictidx, TransactionId xmin,
+ RepOriginId origin, TimestampTz committs)
+{
+ ConflictResolver resolver;
+
+ resolver = get_conflict_resolver_internal(type);
+
+ switch (resolver)
+ {
+ case CR_REMOTE_APPLY:
+ *apply_remote = true;
+ break;
+ case CR_APPLY_OR_SKIP:
+ if (can_create_full_tuple(localrel, newtup))
+ {
+ *apply_remote = true;
+ break;
+ }
+ else
+ {
+ *apply_remote = false;
+ elog(LOG, "UPDATE can not be converted to INSERT, hence SKIP the update!");
+ break;
+ }
+ case CR_APPLY_OR_ERROR:
+ if (can_create_full_tuple(localrel, newtup))
+ {
+ *apply_remote = true;
+ break;
+ }
+ else
+ {
+ ReportApplyConflict(ERROR, type, resolver, localrel, conflictidx,
+ xmin, origin, committs, localslot);
+ break;
+ }
+ case CR_KEEP_LOCAL:
+ case CR_SKIP:
+ *apply_remote = false;
+ break;
+ case CR_ERROR:
+ ReportApplyConflict(ERROR, type, resolver, localrel, conflictidx,
+ xmin, origin, committs, localslot);
+ break;
+ default:
+ elog(ERROR, "Conflict %s is detected! Unrecogonized conflict resolution method", ConflictTypeNames[type]);
+ }
+
+ return resolver;
+}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index b4c834ed3f..239d5c7cd1 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -382,7 +382,9 @@ static void send_feedback(XLogRecPtr recvpos, bool force, bool requestReply);
static void apply_handle_commit_internal(LogicalRepCommitData *commit_data);
static void apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot);
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry);
static void apply_handle_update_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
TupleTableSlot *remoteslot,
@@ -2431,10 +2433,10 @@ apply_handle_insert(StringInfo s)
/* For a partitioned table, insert the tuple into a partition. */
if (rel->localrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
apply_handle_tuple_routing(edata,
- remoteslot, NULL, CMD_INSERT);
+ remoteslot, &newtup, CMD_INSERT);
else
apply_handle_insert_internal(edata, edata->targetRelInfo,
- remoteslot);
+ remoteslot, &newtup, rel);
finish_edata(edata);
@@ -2457,9 +2459,13 @@ apply_handle_insert(StringInfo s)
static void
apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot)
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry)
{
EState *estate = edata->estate;
+ MemoryContext oldctx;
+ TupleTableSlot *conflictslot = NULL;
/* We must open indexes here. */
ExecOpenIndices(relinfo, MySubscription->detectconflict);
@@ -2469,7 +2475,35 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
- ExecSimpleRelationInsert(relinfo, estate, remoteslot);
+
+ if (MySubscription->detectconflict)
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, &conflictslot);
+ else
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, NULL);
+
+
+ /* Apply remote tuple by converting INSERT to UPDATE */
+ if (conflictslot)
+ {
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, conflictslot, rel_entry, newtup);
+ MemoryContextSwitchTo(oldctx);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
+
+ /* Do the update */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, conflictslot,
+ remoteslot);
+
+ EvalPlanQualEnd(&epqstate);
+ ExecDropSingleTupleTableSlot(conflictslot);
+ }
/* Cleanup. */
ExecCloseIndices(relinfo);
@@ -2651,6 +2685,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
TupleTableSlot *localslot;
bool found;
MemoryContext oldctx;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2674,35 +2710,73 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* If conflict detection is enabled, check whether the local tuple was
- * modified by a different origin. If detected, report the conflict.
+ * modified by a different origin. If detected, report the conflict
+ * and configured resolver.
*/
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL, InvalidOid,
+ localxmin, localorigin, localts);
- /* Process and store remote tuple in the slot */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
- MemoryContextSwitchTo(oldctx);
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL);
+ }
- EvalPlanQualSetSlot(&epqstate, remoteslot);
+ /*
+ * Apply the change if configured resolver is in favor of that, else
+ * ignore the remote update.
+ */
+ if (apply_remote)
+ {
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ MemoryContextSwitchTo(oldctx);
- /* Do the actual update. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
- ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
- remoteslot);
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
+
+ /* Do the actual update. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
+ remoteslot);
+ }
}
else
{
/*
- * The tuple to be updated could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be updated could not be found. Report the conflict. If
+ * the configured resolver is in favor of applying the change, convert
+ * UPDATE to INSERT and apply the change.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0);
+
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot, relmapentry, newtup);
+ slot_fill_defaults(relmapentry, estate, remoteslot);
+ MemoryContextSwitchTo(oldctx);
+
+ if (apply_remote)
+ {
+ elog(LOG, "Converting UPDATE to INSERT and applying!");
+ apply_handle_insert_internal(edata, relinfo, remoteslot,
+ newtup, relmapentry);
+ }
+
+ }
}
/* Cleanup. */
@@ -2815,6 +2889,8 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
EPQState epqstate;
TupleTableSlot *localslot;
bool found;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2836,24 +2912,45 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
+ {
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_DIFFER,
+ &apply_remote, NULL, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0);
+
+ ReportApplyConflict(LOG, CT_DELETE_DIFFER, resolver, localrel, InvalidOid,
localxmin, localorigin, localts, NULL);
+ }
- EvalPlanQualSetSlot(&epqstate, localslot);
+ if (apply_remote)
+ {
+ EvalPlanQualSetSlot(&epqstate, localslot);
+
+ /* Do the actual delete. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
+ ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
+ }
- /* Do the actual delete. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
- ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
}
else
{
/*
- * The tuple to be deleted could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be deleted could not be found. Based on resolver
+ * configured, either skip and log a message or emit an error.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_MISSING,
+ &apply_remote, NULL, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0);
+
+ /* Resolver is set to skip, thus report the conflict and skip */
+ if (!apply_remote)
+ ReportApplyConflict(LOG, CT_DELETE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL);
+ }
}
/* Cleanup. */
@@ -2986,19 +3083,21 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
}
MemoryContextSwitchTo(oldctx);
- /* Check if we can do the update or delete on the leaf partition. */
- if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ if (operation == CMD_INSERT || operation == CMD_UPDATE ||
+ operation == CMD_DELETE)
{
- part_entry = logicalrep_partition_open(relmapentry, partrel,
- attrmap);
- check_relation_updatable(part_entry);
+ part_entry = logicalrep_partition_open(relmapentry, partrel, attrmap);
+
+ /* Check if we can do the update or delete on the leaf partition */
+ if (operation != CMD_INSERT)
+ check_relation_updatable(part_entry);
}
switch (operation)
{
case CMD_INSERT:
apply_handle_insert_internal(edata, partrelinfo,
- remoteslot_part);
+ remoteslot_part, newtup, part_entry);
break;
case CMD_DELETE:
@@ -3023,6 +3122,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
RepOriginId localorigin;
TransactionId localxmin;
TimestampTz localts;
+ LogicalRepRelMapEntry *part_entry_new = NULL;
+ ConflictResolver resolver;
+ bool apply_remote = true;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3032,38 +3134,81 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
if (!found)
{
/*
- * The tuple to be updated could not be found. Do nothing
- * except for emitting a log message.
+ * The tuple to be updated could not be found. Report the
+ * conflict and resolver. And take action based on the
+ * configured resolver.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING,
- partrel, InvalidOid,
- InvalidTransactionId,
+ {
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0);
+
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING, resolver, partrel,
+ InvalidOid, InvalidTransactionId,
InvalidRepOriginId, 0, NULL);
- return;
+ if (apply_remote)
+ {
+ /*
+ * Resolver is in favour of applying the remote
+ * changes. Prepare the slot for the INSERT.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot_part, part_entry, newtup);
+ slot_fill_defaults(part_entry, estate, remoteslot_part);
+ MemoryContextSwitchTo(oldctx);
+ }
+ }
}
+ else
+ {
+ /*
+ * The tuple to be updated is found. If conflict detection
+ * is enabled, check whether the local tuple was modified
+ * by a different origin. If detected, report and resolve
+ * the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ {
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL, InvalidOid,
+ localxmin, localorigin, localts);
- /*
- * If conflict detection is enabled, check whether the local
- * tuple was modified by a different origin. If detected,
- * report the conflict.
- */
- if (MySubscription->detectconflict &&
- GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
- localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
- InvalidOid, localxmin, localorigin,
- localts, NULL);
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, resolver, partrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL);
+ }
- /*
- * Apply the update to the local tuple, putting the result in
- * remoteslot_part.
- */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
- MemoryContextSwitchTo(oldctx);
+ if (apply_remote)
+ {
+ /*
+ * We can reach here in two cases:
+ *
+ * 1. If we found a tuple but conflict detection is
+ * OFF
+ *
+ * 2. If we found a tuple and resolver is in favor of
+ * applying the change when conflict detection is ON
+ *
+ * Putting the result in remoteslot_part.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
+ MemoryContextSwitchTo(oldctx);
+ }
+ else
+
+ /*
+ * apply_remote can be toggled if resolver for
+ * update_differ is set to skip. Ignore remote update.
+ */
+ return;
+
+ }
/*
* Does the updated tuple still satisfy the current
@@ -3073,27 +3218,59 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ExecPartitionCheck(partrelinfo, remoteslot_part, estate,
false))
{
- /*
- * Yes, so simply UPDATE the partition. We don't call
- * apply_handle_update_internal() here, which would
- * normally do the following work, to avoid repeating some
- * work already done above to find the local tuple in the
- * partition.
- */
- EPQState epqstate;
-
- EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(partrelinfo, false);
-
- EvalPlanQualSetSlot(&epqstate, remoteslot_part);
- TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
- ACL_UPDATE);
- ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
- localslot, remoteslot_part);
- ExecCloseIndices(partrelinfo);
- EvalPlanQualEnd(&epqstate);
+ if (found && apply_remote)
+ {
+ /*
+ * Yes, so simply UPDATE the partition. We don't call
+ * apply_handle_update_internal() here, which would
+ * normally do the following work, to avoid repeating
+ * some work already done above to find the local
+ * tuple in the partition.
+ *
+ * Do the update in cases - 1. conflict detection is
+ * OFF and found a tuple 2. conflict detection is ON,
+ * update_differ conflict is detected for the found
+ * tuple and the resolver is in favour of applying the
+ * update.
+ */
+
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+ ExecOpenIndices(partrelinfo, false);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot_part);
+ TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
+ ACL_UPDATE);
+ ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
+ localslot, remoteslot_part);
+ ExecCloseIndices(partrelinfo);
+ EvalPlanQualEnd(&epqstate);
+
+ }
+ else if (apply_remote)
+ {
+ /*
+ * Tuple is not found but update_missing resolver is
+ * in favour of applying the change as INSERT.
+ */
+ apply_handle_insert_internal(edata, partrelinfo,
+ remoteslot_part, newtup,
+ part_entry);
+ }
}
- else
+
+ /*
+ * Updated tuple doesn't satisfy the current partition's
+ * constraint.
+ *
+ * If conflict detection is OFF, proceed by always applying
+ * the update (as 'apply_remote' is by default true). If
+ * conflict detection is ON, 'apply_remote' can be OFF as well
+ * if the resolver for update_missing conflict conveys to skip
+ * the update.
+ */
+ else if (apply_remote)
{
/* Move the tuple into the new partition. */
@@ -3134,10 +3311,16 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
get_namespace_name(RelationGetNamespace(partrel_new)),
RelationGetRelationName(partrel_new));
- /* DELETE old tuple found in the old partition. */
- apply_handle_delete_internal(edata, partrelinfo,
- localslot,
- part_entry->localindexoid);
+ /*
+ * If tuple is found, delete it from old partition. We can
+ * reach this flow even for the case when the 'found' flag
+ * is false for 'update_missing' conflict and resolver is
+ * in favor of inserting the tuple.
+ */
+ if (found)
+ apply_handle_delete_internal(edata, partrelinfo,
+ localslot,
+ part_entry->localindexoid);
/* INSERT new tuple into the new partition. */
@@ -3153,19 +3336,25 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
map = ExecGetRootToChildMap(partrelinfo_new, estate);
if (map != NULL)
{
+ attrmap = map->attrMap;
remoteslot_part = execute_attr_map_slot(map->attrMap,
remoteslot,
remoteslot_part);
}
else
{
+ attrmap = NULL;
remoteslot_part = ExecCopySlot(remoteslot_part,
remoteslot);
slot_getallattrs(remoteslot);
}
MemoryContextSwitchTo(oldctx);
+ part_entry_new = logicalrep_partition_open(part_entry,
+ partrel_new,
+ attrmap);
apply_handle_insert_internal(edata, partrelinfo_new,
- remoteslot_part);
+ remoteslot_part, newtup,
+ part_entry_new);
}
}
break;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 9770752ea3..bc72b1f4fd 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,8 @@
#include "fmgr.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/memutils.h"
@@ -655,7 +657,8 @@ extern bool RelationFindReplTupleSeq(Relation rel, LockTupleMode lockmode,
TupleTableSlot *searchslot, TupleTableSlot *outslot);
extern void ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot);
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictsloty);
extern void ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
EState *estate, EPQState *epqstate,
TupleTableSlot *searchslot, TupleTableSlot *slot);
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index c6023e0914..dcc89f222f 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -13,6 +13,8 @@
#include "catalog/objectaddress.h"
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/relcache.h"
#include "utils/timestamp.h"
@@ -75,9 +77,20 @@ typedef enum ConflictResolver
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
extern void ReportApplyConflict(int elevel, ConflictType type,
- Relation localrel, Oid conflictidx,
- TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot);
+ ConflictResolver resolver, Relation localrel,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
extern void ExecConflictResolverStmt(ConflictResolverStmt *stmt);
+extern ConflictResolver GetConflictResolver(TupleTableSlot *localslot,
+ Relation localrel,
+ ConflictType type,
+ bool *apply_remote,
+ LogicalRepTupleData *newtup,
+ Oid conflictidx, TransactionId xmin,
+ RepOriginId origin,
+ TimestampTz committs);
+extern bool CanCreateFullTuple(Relation localrel,
+ LogicalRepTupleData *newtup);
#endif
diff --git a/src/test/subscription/meson.build b/src/test/subscription/meson.build
index c591cd7d61..00ade29b02 100644
--- a/src/test/subscription/meson.build
+++ b/src/test/subscription/meson.build
@@ -40,6 +40,7 @@ tests += {
't/031_column_list.pl',
't/032_subscribe_use_index.pl',
't/033_run_as_table_owner.pl',
+ 't/034_conflict_resolver.pl',
't/100_bugs.pl',
],
},
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 496a3c6cd9..c746191601 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -113,6 +113,10 @@ $node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
+# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
+$node_subscriber->safe_psql('postgres',"set conflict resolver 'error' for 'insert_exists'");
+
+
# Initial synchronization failure causes the subscription to be disabled.
$node_subscriber->poll_query_until('postgres',
"SELECT subenabled = false FROM pg_catalog.pg_subscription WHERE subname = 'sub'"
@@ -177,6 +181,9 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
+# Reset conflict resolver for 'insert_exist' conflict type to default.
+$node_subscriber->safe_psql('postgres',"reset conflict resolver for 'insert_exists'");
+
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
new file mode 100755
index 0000000000..ae8030aac6
--- /dev/null
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -0,0 +1,475 @@
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+###############################
+# Setup
+###############################
+
+# Initialize publisher node
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf',
+ qq(max_prepared_transactions = 10));
+$node_publisher->start;
+
+# Create subscriber node with track_commit_timestamp_set
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ qq(track_commit_timestamp = on));
+$node_subscriber->start;
+
+# Create table on publisher
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Create similar table on subscriber
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Setup logical replication
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE conf_tab");
+
+# Create the subscription
+my $appname = 'tap_sub';
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on);");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT conftype, confres FROM pg_conflict ORDER BY conftype"
+);
+is( $result, qq(delete_differ|remote_apply
+delete_missing|skip
+insert_exists|remote_apply
+update_differ|remote_apply
+update_missing|apply_or_skip),
+ "confirm that the default conflict resolvers are in place"
+);
+
+############################################
+# Test 'remote_apply' for 'insert_exists'
+############################################
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
+
+# Create conflicting data on the publisher
+my $log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=1);"
+);
+
+is($result, 'frompub', "remote data is kept");
+
+
+########################################
+# Test 'keep_local' for 'insert_exists'
+########################################
+
+# Change CONFLICT RESOLVER of insert_exists to keep_local
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'fromsub')");
+
+# Confirm that row is updated
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "data 2 from local is inserted");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that remote insert is ignored and the local row is kept
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "data from local is kept");
+
+###################################
+# Test 'error' for 'insert_exists'
+###################################
+
+# Change CONFLICT RESOLVER of insert_exists to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'insert_exists';"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'fromsub')");
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'frompub')");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Truncate table on subscriber to get rid of the error
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+###################################
+# Test 'skip' for 'delete_missing'
+###################################
+
+# Delete row on publisher that is not present on the subscriber and confirm that it is skipped
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=3);");
+
+# XXX: Fixme. It intermittently fails to capture this log.
+# Confirm that the missing row is skipped because 'delete_missing' is set to 'skip'
+#$node_subscriber->wait_for_log(
+# qr/LOG: conflict delete_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+####################################
+# Test 'error' for 'delete_missing'
+####################################
+
+# Change CONFLICT RESOLVER of delete_missing to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'delete_missing';"
+);
+
+# Capture the log offset before performing the delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+
+# Perform the delete on the publisher
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=2);");
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);"
+);
+
+is($result, 'frompubnew', "update from remote is kept");
+
+#########################################
+# Test 'keep_local' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'keep_local' for 'update_differ';"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'update_differ';"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###########################################
+# Test 'apply_or_skip' for 'update_missing'
+###########################################
+
+#test the apply part
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=3);"
+);
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the skip part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(4,'frompub',repeat('abcdefghij', 200));");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=4);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: UPDATE can not be converted to INSERT, hence SKIP the update!/, $log_offset);
+
+###########################################
+# Test 'apply_or_error' for 'update_missing'
+###########################################
+
+# test the apply part
+
+#Change CONFLICT RESOLVER of update_missing to apply_or_error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';"
+);
+
+# Create new row on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (5,'frompub');");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=5);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=5);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=5);"
+);
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the error part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(6,'frompub',repeat('abcdefghij', 200));");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=6);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=6);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###################################
+# Test 'skip' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to skip
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'skip' for 'update_missing';"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the update does not change anything on the subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);"
+);
+
+is($result, '', "update from remote is skipped on the subscriber");
+
+###################################
+# Test 'error' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'update_missing';"
+);
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnewer' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+done_testing();
--
2.34.1
[application/octet-stream] v5-0004-Manage-Clock-skew-and-implement-last_update_wins.patch (31.5K, ../../CAJpy0uDbDzO4=FrvbtjweU+7P1fDRqU3oON1qaRTWfSjMDnmQg@mail.gmail.com/4-v5-0004-Manage-Clock-skew-and-implement-last_update_wins.patch)
download | inline diff:
From 36acc99dbfe4c6ed9b846dee067839954ce6778d Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Thu, 4 Jul 2024 16:16:46 +0530
Subject: [PATCH v5 4/5] Manage Clock skew and implement last_update_wins
This patch attempts to manage clock skew between nodes by
introducing two new GUCs:
a) max_logical_rep_clock_skew
b) max_logical_rep_clock_skew_action
This patch also implements last_update_wins resolver.
---
src/backend/commands/subscriptioncmds.c | 18 +++
.../replication/logical/applyparallelworker.c | 30 +++-
src/backend/replication/logical/conflict.c | 107 ++++++++++++--
src/backend/replication/logical/origin.c | 1 +
src/backend/replication/logical/worker.c | 130 +++++++++++++++++-
.../utils/activity/wait_event_names.txt | 1 +
src/backend/utils/misc/guc_tables.c | 26 ++++
src/backend/utils/misc/postgresql.conf.sample | 6 +-
src/include/catalog/pg_conflict.dat | 6 +-
src/include/replication/conflict.h | 3 +
src/include/replication/logicalworker.h | 17 +++
src/include/replication/origin.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/include/utils/timestamp.h | 1 +
.../regress/expected/conflict_resolver.out | 16 +--
src/test/regress/expected/subscription.out | 2 +
.../subscription/t/034_conflict_resolver.pl | 16 ++-
src/tools/pgindent/typedefs.list | 1 +
18 files changed, 349 insertions(+), 35 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index e670d72708..9126255ff3 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -14,6 +14,7 @@
#include "postgres.h"
+#include "access/commit_ts.h"
#include "access/htup_details.h"
#include "access/table.h"
#include "access/xact.h"
@@ -671,6 +672,13 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
elog(WARNING, "subscriptions created by regression test cases should have names starting with \"regress_\"");
#endif
+ /* Warn if detect_conflict is enabled and track_commit_timestamp is off */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
+
rel = table_open(SubscriptionRelationId, RowExclusiveLock);
/* Check if name is used */
@@ -1277,6 +1285,16 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
values[Anum_pg_subscription_subdetectconflict - 1] =
BoolGetDatum(opts.detectconflict);
replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+
+ /*
+ * Warn if detect_conflict is enabled and
+ * track_commit_timestamp is off.
+ */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
}
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
diff --git a/src/backend/replication/logical/applyparallelworker.c b/src/backend/replication/logical/applyparallelworker.c
index e7f7d4c5e4..ab4ede3fa9 100644
--- a/src/backend/replication/logical/applyparallelworker.c
+++ b/src/backend/replication/logical/applyparallelworker.c
@@ -312,6 +312,24 @@ pa_can_start(void)
if (!AllTablesyncsReady())
return false;
+ /*
+ * Don't start a new parallel worker if user has set max clock skew
+ * tolerance as the commit timestamp will be needed during 'begin' itself
+ * to manage clock skew.
+ *
+ * Also don't start parallel apply worker if conflict detection and
+ * resolution is ON as commit timesamp will be needed for time based
+ * resolution methods while applying concerned changes.
+ *
+ * XXX: For second case, see if we can reduce the scope of this
+ * restriction to only such cases where time-based resolvers are actually
+ * being used.
+ */
+ if ((max_logical_rep_clock_skew > LR_CLOCK_SKEW_DEFAULT) ||
+ MySubscription->detectconflict)
+ return false;
+
+
return true;
}
@@ -696,9 +714,19 @@ pa_process_spooled_messages_if_required(void)
}
else if (fileset_state == FS_READY)
{
+ /*
+ * Currently we do not support starting parallel apply worker when
+ * either clock skew is configured or conflict resolution is
+ * configured, thus it is okay to pass 0 as origin-timestamp here.
+ *
+ * XXX: If in future, we support starting pa worker even with conflict
+ * detection enabled, then here we need to pass remote's
+ * commit/prepare/abort timestamp; we can get that info from leader
+ * worker in shared memory.
+ */
apply_spooled_messages(&MyParallelShared->fileset,
MyParallelShared->xid,
- InvalidXLogRecPtr);
+ InvalidXLogRecPtr, 0);
pa_set_fileset_state(MyParallelShared, FS_EMPTY);
}
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 5a7f0e0b53..c330b21217 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -44,6 +44,7 @@ const char *const ConflictTypeNames[] = {
const char *const ConflictResolverNames[] = {
[CR_REMOTE_APPLY] = "remote_apply",
[CR_KEEP_LOCAL] = "keep_local",
+ [CR_LAST_UPDATE_WINS] = "last_update_wins",
[CR_APPLY_OR_SKIP] = "apply_or_skip",
[CR_APPLY_OR_ERROR] = "apply_or_error",
[CR_SKIP] = "skip",
@@ -69,11 +70,11 @@ const char *const ConflictResolverNames[] = {
* friendly name for a resolver and thus has been added here.
*/
const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
- [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
- [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
[CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
[CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
- [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR}
};
/*
@@ -81,11 +82,11 @@ const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
* If this changes, change it in pg_conflict.dat as well.
*/
const int ConflictTypeDefaultResolvers[] = {
- [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
- [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_INSERT_EXISTS] = CR_LAST_UPDATE_WINS,
+ [CT_UPDATE_DIFFER] = CR_LAST_UPDATE_WINS,
[CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
[CT_DELETE_MISSING] = CR_SKIP,
- [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+ [CT_DELETE_DIFFER] = CR_LAST_UPDATE_WINS
};
@@ -215,11 +216,30 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
get_rel_name(conflictidx));
}
else
- return errdetail("Key already exists. Applying resolution method \"%s\"", ConflictResolverNames[resolver]);
+ {
+ if (resolver == CR_LAST_UPDATE_WINS)
+ return errdetail("Key already exists. Applying resolution method \"%s\". The local tuple : origin=%u, timestamp=%s; The remote tuple : origin=%u, timestamp=%s.",
+ ConflictResolverNames[resolver],
+ localorigin, timestamptz_to_str(localts),
+ replorigin_session_origin,
+ timestamptz_to_str(replorigin_session_origin_timestamp));
+ else
+ return errdetail("Key already exists. Applying resolution method \"%s\"",
+ ConflictResolverNames[resolver]);
+ }
+
}
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\"",
- localorigin, localxmin, timestamptz_to_str(localts), ConflictResolverNames[resolver]);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\". The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, timestamptz_to_str(localts),
+ ConflictResolverNames[resolver],
+ replorigin_session_origin,
+ timestamptz_to_str(replorigin_session_origin_timestamp));
+ else
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\"",
+ localorigin, localxmin, timestamptz_to_str(localts),
+ ConflictResolverNames[resolver]);
case CT_UPDATE_MISSING:
return errdetail("Did not find the row to be updated. Applying resolution method \"%s\"",
ConflictResolverNames[resolver]);
@@ -227,9 +247,16 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
return errdetail("Did not find the row to be deleted. Applying resolution method \"%s\"",
ConflictResolverNames[resolver]);
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\"",
- localorigin, localxmin, timestamptz_to_str(localts),
- ConflictResolverNames[resolver]);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\". The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, timestamptz_to_str(localts),
+ ConflictResolverNames[resolver],
+ replorigin_session_origin,
+ timestamptz_to_str(replorigin_session_origin_timestamp));
+ else
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s. Applying resolution method \"%s\"",
+ localorigin, localxmin, timestamptz_to_str(localts),
+ ConflictResolverNames[resolver]);
}
return 0; /* silence compiler warning */
@@ -340,6 +367,15 @@ validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver
errmsg("%s is not a valid conflict resolver for conflict type %s",
conflict_resolver,
conflict_type));
+
+ if ((resolver == CR_LAST_UPDATE_WINS) && !track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ conflict_resolver, "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+
return type;
}
@@ -381,6 +417,42 @@ get_conflict_resolver_internal(ConflictType type)
return resolver;
}
+/*
+ * Compare the timestamps of given local tuple and the remote tuple to
+ * resolve the conflict.
+ *
+ * Returns true if remote tuple has the latest timestamp, false otherwise.
+ */
+static bool
+resolve_by_timestamp(TupleTableSlot *localslot)
+{
+ TransactionId local_xmin;
+ TimestampTz local_ts;
+ RepOriginId local_origin;
+ int ts_cmp;
+ uint64 local_system_identifier;
+
+ /* Get origin and timestamp info of the local tuple */
+ GetTupleCommitTs(localslot, &local_xmin, &local_origin, &local_ts);
+
+ /* Compare the timestamps of remote & local tuple to decide the winner */
+ ts_cmp = timestamptz_cmp_internal(replorigin_session_origin_timestamp,
+ local_ts);
+
+ if (ts_cmp == 0)
+ {
+ elog(LOG, "Timestamps of remote and local tuple are equal, comparing remote and local system identifiers");
+
+ /* Get current system's identifier */
+ local_system_identifier = GetSystemIdentifier();
+
+ return local_system_identifier <= replorigin_session_origin_sysid;
+ }
+ else
+ return (ts_cmp > 0);
+
+}
+
/*
* Check if a full tuple can be created from the new tuple.
* Return true if yes, false otherwise.
@@ -499,6 +571,17 @@ GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
switch (resolver)
{
+ case CR_LAST_UPDATE_WINS:
+ if (!track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ ConflictResolverNames[resolver], "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+ else
+ *apply_remote = resolve_by_timestamp(localslot);
+ break;
case CR_REMOTE_APPLY:
*apply_remote = true;
break;
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 419e4814f0..bd8e6f0024 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -155,6 +155,7 @@ typedef struct ReplicationStateCtl
RepOriginId replorigin_session_origin = InvalidRepOriginId; /* assumed identity */
XLogRecPtr replorigin_session_origin_lsn = InvalidXLogRecPtr;
TimestampTz replorigin_session_origin_timestamp = 0;
+uint64 replorigin_session_origin_sysid = 0;
/*
* Base address into a shared memory array of replication states of size
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 239d5c7cd1..3156b0773f 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -318,6 +318,19 @@ static uint32 parallel_stream_nchanges = 0;
/* Are we initializing an apply worker? */
bool InitializingApplyWorker = false;
+/*
+ * GUC support
+ */
+const struct config_enum_entry logical_rep_clock_skew_action_options[] = {
+ {"error", LR_CLOCK_SKEW_ACTION_ERROR, false},
+ {"wait", LR_CLOCK_SKEW_ACTION_WAIT, false},
+ {NULL, 0, false}
+};
+
+/* GUCs */
+int max_logical_rep_clock_skew = LR_CLOCK_SKEW_DEFAULT;
+int max_logical_rep_clock_skew_action = LR_CLOCK_SKEW_ACTION_ERROR;
+
/*
* We enable skipping all data modification changes (INSERT, UPDATE, etc.) for
* the subscription if the remote transaction's finish LSN matches the subskiplsn.
@@ -987,6 +1000,86 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
ExecStoreVirtualTuple(slot);
}
+/*
+ * Manage clock skew between nodes.
+ *
+ * It checks if the remote commit timestamp is ahead of the local clock
+ * and if the difference exceeds max_logical_rep_clock_skew, it performs
+ * the action specified by the max_logical_rep_clock_skew_action.
+ */
+static void
+manage_clock_skew(TimestampTz origin_timestamp)
+{
+ TimestampTz current;
+ TimestampTz delayUntil;
+ long msecs;
+ int rc;
+
+ /* nothing to do if no max clock skew configured */
+ if (max_logical_rep_clock_skew == LR_CLOCK_SKEW_DEFAULT)
+ return;
+
+ current = GetCurrentTimestamp();
+
+ /*
+ * If the timestamp of the currently replayed transaction is in the future
+ * compared to the current time on the subscriber and the difference is
+ * larger than max_logical_rep_clock_skew, then perform the action
+ * specified by the max_logical_rep_clock_skew_action setting.
+ */
+ if (origin_timestamp > current &&
+ TimestampDifferenceExceeds(current, origin_timestamp,
+ max_logical_rep_clock_skew * 1000))
+ {
+ if (max_logical_rep_clock_skew_action == LR_CLOCK_SKEW_ACTION_ERROR)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg_internal("logical replication clock skew exceeded max tolerated value of %d seconds",
+ max_logical_rep_clock_skew)));
+
+ /* Perform the wait */
+ while (true)
+ {
+ delayUntil =
+ TimestampTzMinusSeconds(origin_timestamp,
+ max_logical_rep_clock_skew);
+
+ /* Exit without waiting if it's already past 'delayUntil' time */
+ msecs = TimestampDifferenceMilliseconds(GetCurrentTimestamp(),
+ delayUntil);
+ if (msecs <= 0)
+ break;
+
+ elog(LOG, "delaying apply for %ld milliseconds to bring clock skew "
+ "within permissible value of %d seconds",
+ msecs, max_logical_rep_clock_skew);
+
+ /* Sleep until we are signaled or msecs have elapsed */
+ rc = WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ msecs,
+ WAIT_EVENT_LOGICAL_CLOCK_SKEW);
+
+ /* Exit the loop if msecs have elapsed */
+ if (rc & WL_TIMEOUT)
+ break;
+
+ if (rc & WL_LATCH_SET)
+ {
+ ResetLatch(MyLatch);
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ /* This might change max_logical_rep_clock_skew. */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+ }
+ }
+}
+
/*
* Handle BEGIN message.
*/
@@ -1008,6 +1101,12 @@ apply_handle_begin(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.committime);
+
+ /* Capture the commit timestamp of the remote transaction */
+ replorigin_session_origin_timestamp = begin_data.committime;
}
/*
@@ -1065,6 +1164,12 @@ apply_handle_begin_prepare(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.prepare_time);
+
+ /* Capture the prepare timestamp of the remote transaction */
+ replorigin_session_origin_timestamp = begin_data.prepare_time;
}
/*
@@ -1305,7 +1410,8 @@ apply_handle_stream_prepare(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset,
- prepare_data.xid, prepare_data.prepare_lsn);
+ prepare_data.xid, prepare_data.prepare_lsn,
+ prepare_data.prepare_time);
/* Mark the transaction as prepared. */
apply_handle_prepare_internal(&prepare_data);
@@ -2002,7 +2108,8 @@ ensure_last_message(FileSet *stream_fileset, TransactionId xid, int fileno,
*/
void
apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn)
+ XLogRecPtr lsn,
+ TimestampTz origin_timestamp)
{
int nchanges;
char path[MAXPGPATH];
@@ -2055,6 +2162,13 @@ apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
end_replication_step();
+ /* Check if there is any clock skew and take configured action */
+ if (origin_timestamp)
+ manage_clock_skew(origin_timestamp);
+
+ /* Capture the timestamp (prepare or commit) of the remote transaction */
+ replorigin_session_origin_timestamp = origin_timestamp;
+
/*
* Read the entries one by one and pass them through the same logic as in
* apply_dispatch.
@@ -2160,7 +2274,8 @@ apply_handle_stream_commit(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset, xid,
- commit_data.commit_lsn);
+ commit_data.commit_lsn,
+ commit_data.committime);
apply_handle_commit_internal(&commit_data);
@@ -4694,6 +4809,7 @@ run_apply_worker()
TimeLineID startpointTLI;
char *err;
bool must_use_password;
+ char *replorigin_sysid;
slotname = MySubscription->slotname;
@@ -4733,10 +4849,12 @@ run_apply_worker()
errmsg("could not connect to the publisher: %s", err)));
/*
- * We don't really use the output identify_system for anything but it does
- * some initializations on the upstream so let's still call it.
+ * Call identify_system to do some initializations on the upstream and
+ * store the output as system identifier of the replication origin node.
*/
- (void) walrcv_identify_system(LogRepWorkerWalRcvConn, &startpointTLI);
+ replorigin_sysid = walrcv_identify_system(LogRepWorkerWalRcvConn,
+ &startpointTLI);
+ replorigin_session_origin_sysid = strtoul(replorigin_sysid, NULL, 10);
set_apply_error_context_origin(originname);
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index db37beeaae..a51f82169e 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -59,6 +59,7 @@ CHECKPOINTER_MAIN "Waiting in main loop of checkpointer process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_CLOCK_SKEW "Waiting in apply-begin of logical replication apply process to bring clock skew in permissible range."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot sync worker."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 630ed0f162..f6911ac4a9 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -67,6 +67,7 @@
#include "postmaster/walsummarizer.h"
#include "postmaster/walwriter.h"
#include "replication/logicallauncher.h"
+#include "replication/logicalworker.h"
#include "replication/slot.h"
#include "replication/slotsync.h"
#include "replication/syncrep.h"
@@ -492,6 +493,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry wal_sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry logical_rep_clock_skew_action_options[];
/*
* GUC option variables that are exported from this module
@@ -3649,6 +3651,19 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets maximum clock skew tolerance between logical "
+ "replication nodes beyond which action configured "
+ "in max_logical_rep_clock_skew_action is triggered."),
+ NULL,
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew,
+ LR_CLOCK_SKEW_DEFAULT, LR_CLOCK_SKEW_DEFAULT, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
@@ -4915,6 +4930,17 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew_action", PGC_POSTMASTER, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the action to perform if a clock skew higher "
+ "than max_logical_rep_clock_skew is detected."),
+ NULL
+ },
+ &max_logical_rep_clock_skew_action,
+ LR_CLOCK_SKEW_ACTION_ERROR, logical_rep_clock_skew_action_options,
+ NULL, NULL, NULL
+ },
+
{
{"track_functions", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects function-level statistics on database activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 9ec9f97e92..3a7fd70506 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -383,7 +383,11 @@
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
-
+#max_logical_rep_clock_skew = -1 # maximum clock skew tolerance between logical
+ # replication nodes beyond which action configured in
+ # 'max_logical_rep_clock_skew_action' is triggered.
+#max_logical_rep_clock_skew_action = error # error or wait
+ # (change requires restart)
#------------------------------------------------------------------------------
# QUERY TUNING
diff --git a/src/include/catalog/pg_conflict.dat b/src/include/catalog/pg_conflict.dat
index 8d6183e3f2..1905151840 100644
--- a/src/include/catalog/pg_conflict.dat
+++ b/src/include/catalog/pg_conflict.dat
@@ -12,10 +12,10 @@
[
-{ conftype => 'insert_exists', confres => 'remote_apply' },
-{ conftype => 'update_differ', confres => 'remote_apply' },
+{ conftype => 'insert_exists', confres => 'last_update_wins' },
+{ conftype => 'update_differ', confres => 'last_update_wins' },
{ conftype => 'update_missing', confres => 'apply_or_skip' },
{ conftype => 'delete_missing', confres => 'skip' },
-{ conftype => 'delete_differ', confres => 'remote_apply' }
+{ conftype => 'delete_differ', confres => 'last_update_wins' }
]
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index dcc89f222f..d38a22ddde 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -57,6 +57,9 @@ typedef enum ConflictResolver
/* Keep the local change */
CR_KEEP_LOCAL,
+ /* Apply the change with latest timestamp */
+ CR_LAST_UPDATE_WINS,
+
/* Apply the remote change; skip if it can not be applied */
CR_APPLY_OR_SKIP,
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index a18d79d1b2..2b922f9c62 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,7 +14,24 @@
#include <signal.h>
+/*
+ * The default for max_logical_rep_clock_skew is -1, which means ignore clock
+ * skew (the check is turned off).
+ */
+#define LR_CLOCK_SKEW_DEFAULT -1
+
+/*
+ * Worker Clock Skew Action.
+ */
+typedef enum
+{
+ LR_CLOCK_SKEW_ACTION_ERROR,
+ LR_CLOCK_SKEW_ACTION_WAIT,
+} LogicalRepClockSkewAction;
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
+extern PGDLLIMPORT int max_logical_rep_clock_skew;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_action;
extern void ApplyWorkerMain(Datum main_arg);
extern void ParallelApplyWorkerMain(Datum main_arg);
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 7189ba9e76..dcbbbdf6ea 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -36,6 +36,7 @@ typedef struct xl_replorigin_drop
extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+extern PGDLLIMPORT uint64 replorigin_session_origin_sysid;
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 515aefd519..dc9e067fac 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -267,7 +267,7 @@ extern void stream_stop_internal(TransactionId xid);
/* Common streaming function to apply all the spooled messages */
extern void apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn);
+ XLogRecPtr lsn, TimestampTz origin_timestamp);
extern void apply_dispatch(StringInfo s);
diff --git a/src/include/utils/timestamp.h b/src/include/utils/timestamp.h
index a6ce03ed46..53b828d89d 100644
--- a/src/include/utils/timestamp.h
+++ b/src/include/utils/timestamp.h
@@ -84,6 +84,7 @@ IntervalPGetDatum(const Interval *X)
/* Macros for doing timestamp arithmetic without assuming timestamp's units */
#define TimestampTzPlusMilliseconds(tz,ms) ((tz) + ((ms) * (int64) 1000))
#define TimestampTzPlusSeconds(tz,s) ((tz) + ((s) * (int64) 1000000))
+#define TimestampTzMinusSeconds(tz,s) ((tz) - ((s) * (int64) 1000000))
/* Set at postmaster start */
diff --git a/src/test/regress/expected/conflict_resolver.out b/src/test/regress/expected/conflict_resolver.out
index 53ecb430fa..c21486dbb4 100644
--- a/src/test/regress/expected/conflict_resolver.out
+++ b/src/test/regress/expected/conflict_resolver.out
@@ -1,11 +1,11 @@
--check default global resolvers in system catalog
select * from pg_conflict order by conftype;
- conftype | confres
-----------------+---------------
- delete_differ | remote_apply
+ conftype | confres
+----------------+------------------
+ delete_differ | last_update_wins
delete_missing | skip
- insert_exists | remote_apply
- update_differ | remote_apply
+ insert_exists | last_update_wins
+ update_differ | last_update_wins
update_missing | apply_or_skip
(5 rows)
@@ -43,11 +43,11 @@ RESET CONFLICT RESOLVER for 'delete_missing';
RESET CONFLICT RESOLVER for 'insert_exists';
--check resolvers are reset to default for delete_missing and insert_exists
select * from pg_conflict order by conftype;
- conftype | confres
-----------------+----------------
+ conftype | confres
+----------------+------------------
delete_differ | keep_local
delete_missing | skip
- insert_exists | remote_apply
+ insert_exists | last_update_wins
update_differ | keep_local
update_missing | apply_or_error
(5 rows)
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index a8b0086dd9..47c4dccc3e 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -433,6 +433,8 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
ERROR: detect_conflict requires a Boolean value
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
index ae8030aac6..6f1479dff7 100755
--- a/src/test/subscription/t/034_conflict_resolver.pl
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -51,10 +51,10 @@ $node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
my $result = $node_subscriber->safe_psql('postgres',
"SELECT conftype, confres FROM pg_conflict ORDER BY conftype"
);
-is( $result, qq(delete_differ|remote_apply
+is( $result, qq(delete_differ|last_update_wins
delete_missing|skip
-insert_exists|remote_apply
-update_differ|remote_apply
+insert_exists|last_update_wins
+update_differ|last_update_wins
update_missing|apply_or_skip),
"confirm that the default conflict resolvers are in place"
);
@@ -63,6 +63,11 @@ update_missing|apply_or_skip),
# Test 'remote_apply' for 'insert_exists'
############################################
+# Change CONFLICT RESOLVER of insert_exists to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'remote_apply' for 'insert_exists';"
+);
+
# Create local data on the subscriber
$node_subscriber->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
@@ -202,6 +207,11 @@ $node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
# Test 'remote_apply' for 'update_differ'
#########################################
+# Change CONFLICT RESOLVER of update_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'remote_apply' for 'update_differ';"
+);
+
# Insert data in the publisher
$node_publisher->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c920a59525..13cc7a33fb 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1566,6 +1566,7 @@ LogicalOutputPluginWriterPrepareWrite
LogicalOutputPluginWriterUpdateProgress
LogicalOutputPluginWriterWrite
LogicalRepBeginData
+LogicalRepClockSkewAction
LogicalRepCommitData
LogicalRepCommitPreparedTxnData
LogicalRepCtxStruct
--
2.34.1
[application/octet-stream] v5-0001-Detect-and-log-conflicts-in-logical-replication.patch (95.5K, ../../CAJpy0uDbDzO4=FrvbtjweU+7P1fDRqU3oON1qaRTWfSjMDnmQg@mail.gmail.com/5-v5-0001-Detect-and-log-conflicts-in-logical-replication.patch)
download | inline diff:
From 3fd759ad7ce87a573730c27ec61e7d392d47aa4d Mon Sep 17 00:00:00 2001
From: Hou Zhijie <[email protected]>
Date: Fri, 31 May 2024 14:20:45 +0530
Subject: [PATCH v5 1/5] Detect and log conflicts in logical replication
This patch adds a new parameter detect_conflict for CREATE and ALTER
subscription commands. This new parameter will decide if subscription will
go for confict detection. By default, conflict detection will be off for a
subscription.
When conflict detection is enabled, additional logging is triggered in the
following conflict scenarios:
insert_exists: Inserting a row that violates a NOT DEFERRABLE unique constraint.
update_differ: Updating a row that was previously modified by another origin.
update_missing: The tuple to be updated is missing.
delete_missing: The tuple to be deleted is missing.
delete_differ: Deleting a row that was previously modified by another origin.
For insert_exists conflict, the log can include origin and commit
timestamp details of the conflicting key with track_commit_timestamp
enabled.
update_differ and delete_differ conflicts can only be detected when
track_commit_timestamp is enabled.
---
doc/src/sgml/catalogs.sgml | 9 +
doc/src/sgml/logical-replication.sgml | 23 ++-
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 66 +++++++
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 3 +-
src/backend/commands/subscriptioncmds.c | 31 +++-
src/backend/executor/execIndexing.c | 14 +-
src/backend/executor/execReplication.c | 117 +++++++++++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/conflict.c | 191 ++++++++++++++++++++
src/backend/replication/logical/meson.build | 1 +
src/backend/replication/logical/worker.c | 79 ++++++--
src/bin/pg_dump/pg_dump.c | 17 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 6 +-
src/bin/psql/tab-complete.c | 14 +-
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/conflict.h | 47 +++++
src/test/regress/expected/subscription.out | 176 ++++++++++--------
src/test/regress/sql/subscription.sql | 15 ++
src/test/subscription/t/001_rep_changes.pl | 15 +-
src/test/subscription/t/013_partition.pl | 68 ++++---
src/test/subscription/t/029_on_error.pl | 5 +-
src/test/subscription/t/030_origin.pl | 43 +++++
src/tools/pgindent/typedefs.list | 1 +
26 files changed, 801 insertions(+), 152 deletions(-)
create mode 100644 src/backend/replication/logical/conflict.c
create mode 100644 src/include/replication/conflict.h
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b654fae1b2..b042a5a94a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8035,6 +8035,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subdetectconflict</structfield> <type>bool</type>
+ </para>
+ <para>
+ If true, the subscription is enabled for conflict detection.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subconninfo</structfield> <type>text</type>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index ccdd24312b..f078d6364b 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1565,7 +1565,9 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
stop. This is referred to as a <firstterm>conflict</firstterm>. When
replicating <command>UPDATE</command> or <command>DELETE</command>
operations, missing data will not produce a conflict and such operations
- will simply be skipped.
+ will simply be skipped, but note that this scenario can be reported in the server log
+ if <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ is enabled.
</para>
<para>
@@ -1634,6 +1636,25 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
linkend="sql-altersubscription"><command>ALTER SUBSCRIPTION ...
SKIP</command></link>.
</para>
+
+ <para>
+ Enabling both <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ and <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ on the subscriber can provide additional details regarding conflicting
+ rows, such as their origin and commit timestamp, in case of a unique
+ constraint violation conflict:
+<screen>
+ERROR: conflict insert_exists detected on relation "public.t"
+DETAIL: Key (a)=(1) already exists in unique index "t_pkey", which was modified by origin 0 in transaction 740 at 2024-06-26 10:47:04.727375+08.
+CONTEXT: processing remote data for replication origin "pg_16389" during message type "INSERT" for replication target relation "public.t" in transaction 740, finished at 0/14F7EC0
+</screen>
+ Users can use these information to make decisions on whether to retain
+ the local change or adopt the remote alteration. For instance, the
+ origin in above log indicates that the existing row was modified by a
+ local change, users can manually perform a remote-change-win resolution
+ by deleting the local row. Refer to <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ for other conflicts that will be logged when enabling <literal>detect_conflict</literal>.
+ </para>
</sect1>
<sect1 id="logical-replication-restrictions">
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 476f195622..5f6b83e415 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -228,8 +228,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<link linkend="sql-createsubscription-params-with-disable-on-error"><literal>disable_on_error</literal></link>,
<link linkend="sql-createsubscription-params-with-password-required"><literal>password_required</literal></link>,
<link linkend="sql-createsubscription-params-with-run-as-owner"><literal>run_as_owner</literal></link>,
- <link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>, and
- <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>.
+ <link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>,
+ <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>, and
+ <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>.
Only a superuser can set <literal>password_required = false</literal>.
</para>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 740b7d9421..caa523b9bd 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -428,6 +428,72 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createsubscription-params-with-detect-conflict">
+ <term><literal>detect_conflict</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the subscription is enabled for conflict detection.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ When conflict detection is enabled, additional logging is triggered
+ in the following scenarios:
+ <variablelist>
+ <varlistentry>
+ <term><literal>insert_exists</literal></term>
+ <listitem>
+ <para>
+ Inserting a row that violates a <literal>NOT DEFERRABLE</literal>
+ unique constraint. Note that to obtain the origin and commit
+ timestamp details of the conflicting key in the log, ensure that
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_differ</literal></term>
+ <listitem>
+ <para>
+ Updating a row that was previously modified by another origin. Note that this
+ conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be updated is not found.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be deleted is not found.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_differ</literal></term>
+ <listitem>
+ <para>
+ Deleting a row that was previously modified by another origin. Note that this
+ conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist></para>
</listitem>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..5a423f4fb0 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -72,6 +72,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->passwordrequired = subform->subpasswordrequired;
sub->runasowner = subform->subrunasowner;
sub->failover = subform->subfailover;
+ sub->detectconflict = subform->subdetectconflict;
/* Get conninfo */
datum = SysCacheGetAttrNotNull(SUBSCRIPTIONOID,
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 19cabc9a47..d084bfc48a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1356,7 +1356,8 @@ REVOKE ALL ON pg_subscription FROM public;
GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
subbinary, substream, subtwophasestate, subdisableonerr,
subpasswordrequired, subrunasowner, subfailover,
- subslotname, subsynccommit, subpublications, suborigin)
+ subdetectconflict, subslotname, subsynccommit,
+ subpublications, suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index e407428dbc..e670d72708 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -70,8 +70,9 @@
#define SUBOPT_PASSWORD_REQUIRED 0x00000800
#define SUBOPT_RUN_AS_OWNER 0x00001000
#define SUBOPT_FAILOVER 0x00002000
-#define SUBOPT_LSN 0x00004000
-#define SUBOPT_ORIGIN 0x00008000
+#define SUBOPT_DETECT_CONFLICT 0x00004000
+#define SUBOPT_LSN 0x00008000
+#define SUBOPT_ORIGIN 0x00010000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -97,6 +98,7 @@ typedef struct SubOpts
bool passwordrequired;
bool runasowner;
bool failover;
+ bool detectconflict;
char *origin;
XLogRecPtr lsn;
} SubOpts;
@@ -159,6 +161,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->runasowner = false;
if (IsSet(supported_opts, SUBOPT_FAILOVER))
opts->failover = false;
+ if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT))
+ opts->detectconflict = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
@@ -316,6 +320,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_FAILOVER;
opts->failover = defGetBoolean(defel);
}
+ else if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT) &&
+ strcmp(defel->defname, "detect_conflict") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_DETECT_CONFLICT;
+ opts->detectconflict = defGetBoolean(defel);
+ }
else if (IsSet(supported_opts, SUBOPT_ORIGIN) &&
strcmp(defel->defname, "origin") == 0)
{
@@ -603,7 +616,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
SUBOPT_DISABLE_ON_ERR | SUBOPT_PASSWORD_REQUIRED |
- SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER | SUBOPT_ORIGIN);
+ SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -710,6 +724,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subpasswordrequired - 1] = BoolGetDatum(opts.passwordrequired);
values[Anum_pg_subscription_subrunasowner - 1] = BoolGetDatum(opts.runasowner);
values[Anum_pg_subscription_subfailover - 1] = BoolGetDatum(opts.failover);
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
values[Anum_pg_subscription_subconninfo - 1] =
CStringGetTextDatum(conninfo);
if (opts.slot_name)
@@ -1146,7 +1162,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
SUBOPT_PASSWORD_REQUIRED |
SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
- SUBOPT_ORIGIN);
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1256,6 +1272,13 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
replaces[Anum_pg_subscription_subfailover - 1] = true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
+ replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9f05b3654c..ef522778a2 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -207,8 +207,9 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
ii = BuildIndexInfo(indexDesc);
/*
- * If the indexes are to be used for speculative insertion, add extra
- * information required by unique index entries.
+ * If the indexes are to be used for speculative insertion or conflict
+ * detection in logical replication, add extra information required by
+ * unique index entries.
*/
if (speculative && ii->ii_Unique)
BuildSpeculativeIndexInfo(indexDesc, ii);
@@ -521,6 +522,11 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* possible that a conflicting tuple is inserted immediately
* after this returns. But this can be used for a pre-check
* before insertion.
+ *
+ * If the 'slot' holds a tuple with valid tid, this tuple will
+ * be ignored when checking conflict. This can help in scenarios
+ * where we want to re-check for conflicts after inserting a
+ * tuple.
* ----------------------------------------------------------------
*/
bool
@@ -536,11 +542,9 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
ExprContext *econtext;
Datum values[INDEX_MAX_KEYS];
bool isnull[INDEX_MAX_KEYS];
- ItemPointerData invalidItemPtr;
bool checkedIndex = false;
ItemPointerSetInvalid(conflictTid);
- ItemPointerSetInvalid(&invalidItemPtr);
/*
* Get information from the result relation info structure.
@@ -629,7 +633,7 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
satisfiesConstraint =
check_exclusion_or_unique_constraint(heapRelation, indexRelation,
- indexInfo, &invalidItemPtr,
+ indexInfo, &slot->tts_tid,
values, isnull, estate, false,
CEOUC_WAIT, true,
conflictTid);
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index d0a89cd577..f01927a933 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -23,6 +23,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "replication/conflict.h"
#include "replication/logicalrelation.h"
#include "storage/lmgr.h"
#include "utils/builtins.h"
@@ -480,6 +481,83 @@ retry:
return found;
}
+/*
+ * Find the tuple that violates the passed in unique index constraint
+ * (conflictindex).
+ *
+ * Return false if there is no conflict and *conflictslot is set to NULL.
+ * Otherwise return true, and the conflicting tuple is locked and returned
+ * in *conflictslot.
+ */
+static bool
+FindConflictTuple(ResultRelInfo *resultRelInfo, EState *estate,
+ Oid conflictindex, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot)
+{
+ Relation rel = resultRelInfo->ri_RelationDesc;
+ ItemPointerData conflictTid;
+ TM_FailureData tmfd;
+ TM_Result res;
+
+ *conflictslot = NULL;
+
+retry:
+ if (ExecCheckIndexConstraints(resultRelInfo, slot, estate,
+ &conflictTid, list_make1_oid(conflictindex)))
+ {
+ if (*conflictslot)
+ ExecDropSingleTupleTableSlot(*conflictslot);
+
+ *conflictslot = NULL;
+ return false;
+ }
+
+ *conflictslot = table_slot_create(rel, NULL);
+
+ PushActiveSnapshot(GetLatestSnapshot());
+
+ res = table_tuple_lock(rel, &conflictTid, GetLatestSnapshot(),
+ *conflictslot,
+ GetCurrentCommandId(false),
+ LockTupleShare,
+ LockWaitBlock,
+ 0 /* don't follow updates */ ,
+ &tmfd);
+
+ PopActiveSnapshot();
+
+ switch (res)
+ {
+ case TM_Ok:
+ break;
+ case TM_Updated:
+ /* XXX: Improve handling here */
+ if (ItemPointerIndicatesMovedPartitions(&tmfd.ctid))
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("tuple to be locked was already moved to another partition due to concurrent update, retrying")));
+ else
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent update, retrying")));
+ goto retry;
+ case TM_Deleted:
+ /* XXX: Improve handling here */
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent delete, retrying")));
+ goto retry;
+ case TM_Invisible:
+ elog(ERROR, "attempted to lock invisible tuple");
+ break;
+ default:
+ elog(ERROR, "unexpected table_tuple_lock status: %u", res);
+ break;
+ }
+
+ return true;
+}
+
/*
* Insert tuple represented in the slot to the relation, update the indexes,
* and execute any constraints and per-row triggers.
@@ -509,6 +587,8 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
+ List *conflictindexes;
+ bool conflict = false;
/* Compute stored generated columns */
if (rel->rd_att->constr &&
@@ -525,10 +605,43 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
- slot, estate, false, false,
- NULL, NIL, false);
+ slot, estate, false,
+ conflictindexes, &conflict,
+ conflictindexes, false);
+
+ /* Re-check all the unique indexes for potential conflicts */
+ foreach_oid(uniqueidx, (conflict ? conflictindexes : NIL))
+ {
+ TupleTableSlot *conflictslot;
+
+ /*
+ * Reports the conflict if any.
+ *
+ * Here, we attempt to find the conflict tuple. This operation may
+ * seem redundant with the unique violation check of indexam, but
+ * since we perform this only when we are detecting conflict in
+ * logical replication and encountering potential conflicts with
+ * any unique index constraints (which should not be frequent), so
+ * it's ok. Moreover, upon detecting a conflict, we will report an
+ * ERROR and restart the logical replication, so the additional
+ * cost of finding the tuple should be acceptable in this case.
+ */
+ if (list_member_oid(recheckIndexes, uniqueidx) &&
+ FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &conflictslot))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+
+ GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
+ ReportApplyConflict(ERROR, CT_INSERT_EXISTS, rel, uniqueidx,
+ xmin, origin, committs, conflictslot);
+ }
+ }
/* AFTER ROW INSERT Triggers */
ExecARInsertTriggers(estate, resultRelInfo, slot,
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index ba03eeff1c..1e08bbbd4e 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
applyparallelworker.o \
+ conflict.o \
decode.o \
launcher.o \
logical.o \
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
new file mode 100644
index 0000000000..b90e64b05b
--- /dev/null
+++ b/src/backend/replication/logical/conflict.c
@@ -0,0 +1,191 @@
+/*-------------------------------------------------------------------------
+ * conflict.c
+ * Functionality for detecting and logging conflicts.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/conflict.c
+ *
+ * This file contains the code for detecting and logging conflicts on
+ * the subscriber during logical replication.
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/commit_ts.h"
+#include "replication/conflict.h"
+#include "replication/origin.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+
+const char *const ConflictTypeNames[] = {
+ [CT_INSERT_EXISTS] = "insert_exists",
+ [CT_UPDATE_DIFFER] = "update_differ",
+ [CT_UPDATE_MISSING] = "update_missing",
+ [CT_DELETE_MISSING] = "delete_missing",
+ [CT_DELETE_DIFFER] = "delete_differ"
+};
+
+static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
+static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin,
+ RepOriginId localorigin,
+ TimestampTz localts,
+ TupleTableSlot *conflictslot);
+
+/*
+ * Get the xmin and commit timestamp data (origin and timestamp) associated
+ * with the provided local tuple.
+ *
+ * Return true if the commit timestamp data was found, false otherwise.
+ */
+bool
+GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts)
+{
+ Datum xminDatum;
+ bool isnull;
+
+ xminDatum = slot_getsysattr(localslot, MinTransactionIdAttributeNumber,
+ &isnull);
+ *xmin = DatumGetTransactionId(xminDatum);
+ Assert(!isnull);
+
+ /*
+ * The commit timestamp data is not available if track_commit_timestamp is
+ * disabled.
+ */
+ if (!track_commit_timestamp)
+ {
+ *localorigin = InvalidRepOriginId;
+ *localts = 0;
+ return false;
+ }
+
+ return TransactionIdGetCommitTsData(*xmin, localts, localorigin);
+}
+
+/*
+ * Report a conflict when applying remote changes.
+ */
+void
+ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot)
+{
+ ereport(elevel,
+ errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
+ errmsg("conflict %s detected on relation \"%s.%s\"",
+ ConflictTypeNames[type],
+ get_namespace_name(RelationGetNamespace(localrel)),
+ RelationGetRelationName(localrel)),
+ errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
+ localts, conflictslot));
+}
+
+/*
+ * Find all unique indexes to check for a conflict and store them into
+ * ResultRelInfo.
+ */
+void
+InitConflictIndexes(ResultRelInfo *relInfo)
+{
+ List *uniqueIndexes = NIL;
+
+ for (int i = 0; i < relInfo->ri_NumIndices; i++)
+ {
+ Relation indexRelation = relInfo->ri_IndexRelationDescs[i];
+
+ if (indexRelation == NULL)
+ continue;
+
+ /* Detect conflict only for unique indexes */
+ if (!relInfo->ri_IndexRelationInfo[i]->ii_Unique)
+ continue;
+
+ /* Don't support conflict detection for deferrable index */
+ if (!indexRelation->rd_index->indimmediate)
+ continue;
+
+ uniqueIndexes = lappend_oid(uniqueIndexes,
+ RelationGetRelid(indexRelation));
+ }
+
+ relInfo->ri_onConflictArbiterIndexes = uniqueIndexes;
+}
+
+/*
+ * Add an errdetail() line showing conflict detail.
+ */
+static int
+errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot)
+{
+ switch (type)
+ {
+ case CT_INSERT_EXISTS:
+ {
+ /*
+ * Bulid the index value string. If the return value is NULL,
+ * it indicates that the current user lacks permissions to
+ * view all the columns involved.
+ */
+ char *index_value = build_index_value_desc(conflictidx,
+ conflictslot);
+
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
+ case CT_UPDATE_DIFFER:
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ case CT_UPDATE_MISSING:
+ return errdetail("Did not find the row to be updated.");
+ case CT_DELETE_MISSING:
+ return errdetail("Did not find the row to be deleted.");
+ case CT_DELETE_DIFFER:
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ }
+
+ return 0; /* silence compiler warning */
+}
+
+/*
+ * Helper functions to construct a string describing the contents of an index
+ * entry. See BuildIndexValueDescription for details.
+ */
+static char *
+build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
+{
+ char *conflict_row;
+ Relation indexDesc;
+
+ if (!conflictslot)
+ return NULL;
+
+ /* Assume the index has been locked */
+ indexDesc = index_open(indexoid, NoLock);
+
+ slot_getallattrs(conflictslot);
+
+ conflict_row = BuildIndexValueDescription(indexDesc,
+ conflictslot->tts_values,
+ conflictslot->tts_isnull);
+
+ index_close(indexDesc, NoLock);
+
+ return conflict_row;
+}
diff --git a/src/backend/replication/logical/meson.build b/src/backend/replication/logical/meson.build
index 3dec36a6de..3d36249d8a 100644
--- a/src/backend/replication/logical/meson.build
+++ b/src/backend/replication/logical/meson.build
@@ -2,6 +2,7 @@
backend_sources += files(
'applyparallelworker.c',
+ 'conflict.c',
'decode.c',
'launcher.c',
'logical.c',
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 3b285894db..b4c834ed3f 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -167,6 +167,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
+#include "replication/conflict.h"
#include "replication/logicallauncher.h"
#include "replication/logicalproto.h"
#include "replication/logicalrelation.h"
@@ -2461,7 +2462,10 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
EState *estate = edata->estate;
/* We must open indexes here. */
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, MySubscription->detectconflict);
+
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
@@ -2664,6 +2668,20 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
slot_modify_data(remoteslot, localslot, relmapentry, newtup);
@@ -2681,13 +2699,10 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2810,6 +2825,20 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/* If found delete it. */
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
EvalPlanQualSetSlot(&epqstate, localslot);
/* Do the actual delete. */
@@ -2821,13 +2850,10 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/*
* The tuple to be deleted could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be deleted "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2994,6 +3020,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ResultRelInfo *partrelinfo_new;
Relation partrel_new;
bool found;
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3005,16 +3034,28 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing
* except for emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation's partition \"%s\"",
- RelationGetRelationName(partrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING,
+ partrel, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL);
+
return;
}
+ /*
+ * If conflict detection is enabled, check whether the local
+ * tuple was modified by a different origin. If detected,
+ * report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
+ InvalidOid, localxmin, localorigin,
+ localts, NULL);
+
/*
* Apply the update to the local tuple, putting the result in
* remoteslot_part.
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b8b1888bd3..829f9b3e88 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4760,6 +4760,7 @@ getSubscriptions(Archive *fout)
int i_suboriginremotelsn;
int i_subenabled;
int i_subfailover;
+ int i_subdetectconflict;
int i,
ntups;
@@ -4832,11 +4833,17 @@ getSubscriptions(Archive *fout)
if (fout->remoteVersion >= 170000)
appendPQExpBufferStr(query,
- " s.subfailover\n");
+ " s.subfailover,\n");
else
appendPQExpBuffer(query,
- " false AS subfailover\n");
+ " false AS subfailover,\n");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ " s.subdetectconflict\n");
+ else
+ appendPQExpBuffer(query,
+ " false AS subdetectconflict\n");
appendPQExpBufferStr(query,
"FROM pg_subscription s\n");
@@ -4875,6 +4882,7 @@ getSubscriptions(Archive *fout)
i_suboriginremotelsn = PQfnumber(res, "suboriginremotelsn");
i_subenabled = PQfnumber(res, "subenabled");
i_subfailover = PQfnumber(res, "subfailover");
+ i_subdetectconflict = PQfnumber(res, "subdetectconflict");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4921,6 +4929,8 @@ getSubscriptions(Archive *fout)
pg_strdup(PQgetvalue(res, i, i_subenabled));
subinfo[i].subfailover =
pg_strdup(PQgetvalue(res, i, i_subfailover));
+ subinfo[i].subdetectconflict =
+ pg_strdup(PQgetvalue(res, i, i_subdetectconflict));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -5161,6 +5171,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subfailover, "t") == 0)
appendPQExpBufferStr(query, ", failover = true");
+ if (strcmp(subinfo->subdetectconflict, "t") == 0)
+ appendPQExpBufferStr(query, ", detect_conflict = true");
+
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 4b2e5870a9..bbd7cbeff6 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -671,6 +671,7 @@ typedef struct _SubscriptionInfo
char *suborigin;
char *suboriginremotelsn;
char *subfailover;
+ char *subdetectconflict;
} SubscriptionInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 7c9a1f234c..fef1ad0d70 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6539,7 +6539,7 @@ describeSubscriptions(const char *pattern, bool verbose)
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
false, false, false, false, false, false, false, false, false, false,
- false};
+ false, false};
if (pset.sversion < 100000)
{
@@ -6607,6 +6607,10 @@ describeSubscriptions(const char *pattern, bool verbose)
appendPQExpBuffer(&buf,
", subfailover AS \"%s\"\n",
gettext_noop("Failover"));
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ ", subdetectconflict AS \"%s\"\n",
+ gettext_noop("Detect conflict"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index d453e224d9..219fac7e71 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1946,9 +1946,10 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit");
+ COMPLETE_WITH("binary", "detect_conflict", "disable_on_error",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
COMPLETE_WITH("lsn");
@@ -3363,9 +3364,10 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit", "two_phase");
+ "detect_conflict", "disable_on_error", "enabled",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..17daf11dc7 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -98,6 +98,9 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool subdetectconflict; /* True if replication should perform
+ * conflict detection */
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* Connection string to the publisher */
text subconninfo BKI_FORCE_NOT_NULL;
@@ -151,6 +154,7 @@ typedef struct Subscription
* (i.e. the main slot and the table sync
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool detectconflict; /* True if conflict detection is enabled */
char *conninfo; /* Connection string to the publisher */
char *slotname; /* Name of the replication slot */
char *synccommit; /* Synchronous commit setting for worker */
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
new file mode 100644
index 0000000000..a9f521aaca
--- /dev/null
+++ b/src/include/replication/conflict.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * conflict.h
+ * Exports for conflict detection and log
+ *
+ * Copyright (c) 2012-2024, PostgreSQL Global Development Group
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef CONFLICT_H
+#define CONFLICT_H
+
+#include "access/xlogdefs.h"
+#include "executor/tuptable.h"
+#include "nodes/execnodes.h"
+#include "utils/relcache.h"
+#include "utils/timestamp.h"
+
+/*
+ * Conflict types that could be encountered when applying remote changes.
+ */
+typedef enum
+{
+ /* The row to be inserted violates unique constraint */
+ CT_INSERT_EXISTS,
+
+ /* The row to be updated was modified by a different origin */
+ CT_UPDATE_DIFFER,
+
+ /* The row to be updated is missing */
+ CT_UPDATE_MISSING,
+
+ /* The row to be deleted is missing */
+ CT_DELETE_MISSING,
+
+ /* The row to be deleted was modified by a different origin */
+ CT_DELETE_DIFFER,
+} ConflictType;
+
+extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts);
+extern void ReportApplyConflict(int elevel, ConflictType type,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot);
+extern void InitConflictIndexes(ResultRelInfo *relInfo);
+
+#endif
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 0f2a25cdc1..a8b0086dd9 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -116,18 +116,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -145,10 +145,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -157,10 +157,10 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = 'newname');
ALTER SUBSCRIPTION regress_testsub SET (password_required = false);
ALTER SUBSCRIPTION regress_testsub SET (run_as_owner = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (password_required = true);
@@ -176,10 +176,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -188,10 +188,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -223,10 +223,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -255,19 +255,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -279,27 +279,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -314,10 +314,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -332,10 +332,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -371,10 +371,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -383,10 +383,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -396,10 +396,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -412,18 +412,42 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+ERROR: detect_conflict requires a Boolean value
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | t | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 3e5ba4cb8c..a77b196704 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -290,6 +290,21 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
-- let's do some tests with pg_create_subscription rather than superuser
SET SESSION AUTHORIZATION regress_subscription_user3;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 471e981962..78c0307165 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -331,11 +331,12 @@ is( $result, qq(1|bar
2|baz),
'update works with REPLICA IDENTITY FULL and a primary key');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber->append_conf('postgresql.conf', "log_min_messages = debug1");
-$node_subscriber->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub SET (detect_conflict = true)"
+);
$node_subscriber->safe_psql('postgres', "DELETE FROM tab_full_pk");
@@ -352,10 +353,10 @@ $node_publisher->wait_for_catchup('tap_sub');
my $logfile = slurp_file($node_subscriber->logfile, $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation "tab_full_pk"/,
+ qr/conflict update_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be updated./m,
'update target row is missing');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab_full_pk"/,
+ qr/conflict delete_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be deleted./m,
'delete target row is missing');
$node_subscriber->append_conf('postgresql.conf',
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 29580525a9..7a66a06b51 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -343,12 +343,12 @@ $result =
$node_subscriber2->safe_psql('postgres', "SELECT a FROM tab1 ORDER BY 1");
is($result, qq(), 'truncate of tab1 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = true)"
+);
$node_publisher->safe_psql('postgres',
"INSERT INTO tab1 VALUES (1, 'foo'), (4, 'bar'), (10, 'baz')");
@@ -372,21 +372,21 @@ $node_publisher->wait_for_catchup('sub2');
my $logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab1_2_2"/,
+ qr/conflict update_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_1"/,
+ qr/conflict delete_missing detected on relation "public.tab1_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_2_2"/,
+ qr/conflict delete_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_def"/,
+ qr/conflict delete_missing detected on relation "public.tab1_def".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_def');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = false)"
+);
# Tests for replication using root table identity and schema
@@ -773,12 +773,12 @@ pub_tab2|3|yyy
pub_tab2|5|zzz
xxx_c|6|aaa), 'inserts into tab2 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = true)"
+);
$node_subscriber1->safe_psql('postgres', "DELETE FROM tab2");
@@ -796,15 +796,35 @@ $node_publisher->wait_for_catchup('sub2');
$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab2_1"/,
+ qr/conflict update_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab2_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab2_1"/,
+ qr/conflict delete_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab2_1');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+# Enable the track_commit_timestamp to detect the conflict when attempting
+# to update a row that was previously modified by a different origin.
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = on');
+$node_subscriber1->restart;
+
+$node_subscriber1->safe_psql('postgres', "INSERT INTO tab2 VALUES (3, 'yyy')");
+$node_publisher->safe_psql('postgres',
+ "UPDATE tab2 SET b = 'quux' WHERE a = 3");
+
+$node_publisher->wait_for_catchup('sub_viaroot');
+
+$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
+ok( $logfile =~
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/,
+ 'updating a tuple that was modified by a different origin');
+
+# The remaining tests no longer test conflict detection.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = false)"
+);
+
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_subscriber1->restart;
# Test that replication continues to work correctly after altering the
# partition of a partitioned target table.
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 0ab57a4b5b..496a3c6cd9 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -30,7 +30,7 @@ sub test_skip_lsn
# ERROR with its CONTEXT when retrieving this information.
my $contents = slurp_file($node_subscriber->logfile, $offset);
$contents =~
- qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
or die "could not get error-LSN";
my $lsn = $1;
@@ -83,6 +83,7 @@ $node_subscriber->append_conf(
'postgresql.conf',
qq[
max_prepared_transactions = 10
+track_commit_timestamp = on
]);
$node_subscriber->start;
@@ -109,7 +110,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
# Initial synchronization failure causes the subscription to be disabled.
diff --git a/src/test/subscription/t/030_origin.pl b/src/test/subscription/t/030_origin.pl
index 056561f008..8c929c07c7 100644
--- a/src/test/subscription/t/030_origin.pl
+++ b/src/test/subscription/t/030_origin.pl
@@ -27,9 +27,14 @@ my $stderr;
my $node_A = PostgreSQL::Test::Cluster->new('node_A');
$node_A->init(allows_streaming => 'logical');
$node_A->start;
+
# node_B
my $node_B = PostgreSQL::Test::Cluster->new('node_B');
$node_B->init(allows_streaming => 'logical');
+
+# Enable the track_commit_timestamp to detect the conflict when attempting to
+# update a row that was previously modified by a different origin.
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = on');
$node_B->start;
# Create table on node_A
@@ -139,6 +144,44 @@ is($result, qq(),
'Remote data originating from another node (not the publisher) is not replicated when origin parameter is none'
);
+###############################################################################
+# Check that the conflict can be detected when attempting to update or
+# delete a row that was previously modified by a different source.
+###############################################################################
+
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = true);
+ DELETE FROM tab;");
+
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (32);");
+
+# The update should update the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "UPDATE tab SET a = 33 WHERE a = 32;");
+
+$node_B->wait_for_log(
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_B->safe_psql('postgres', "DELETE FROM tab;");
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (33);");
+
+# The delete should remove the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "DELETE FROM tab WHERE a = 33;");
+
+$node_B->wait_for_log(
+ qr/Deleting a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_A->wait_for_catchup($subname_BA);
+$node_B->wait_for_catchup($subname_AB);
+
+# The remaining tests no longer test conflict detection.
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = false);");
+
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_B->restart;
+
###############################################################################
# Specifying origin = NONE indicates that the publisher should only replicate the
# changes that are generated locally from node_B, but in this case since the
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 635e6d6e21..3c79446ebb 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -467,6 +467,7 @@ ConditionVariableMinimallyPadded
ConditionalStack
ConfigData
ConfigVariable
+ConflictType
ConnCacheEntry
ConnCacheKey
ConnParams
--
2.34.1
[application/octet-stream] v5-0005-Configure-table-level-conflict-resolvers.patch (48.9K, ../../CAJpy0uDbDzO4=FrvbtjweU+7P1fDRqU3oON1qaRTWfSjMDnmQg@mail.gmail.com/6-v5-0005-Configure-table-level-conflict-resolvers.patch)
download | inline diff:
From b7bddbe815109f7fda24d8d1ca3fc2b097a55424 Mon Sep 17 00:00:00 2001
From: Shveta Malik <[email protected]>
Date: Tue, 9 Jul 2024 14:37:08 +0530
Subject: [PATCH v5 5/5] Configure table level conflict resolvers
This patch provides support for configuring table level conflict
resolvers using ALTER TABLE cmd.
Syntax to SET resolvers:
ALTER TABLE <name> SET CONFLICT RESOLVER <resolver1> on <conflict_type1>,
SET CONFLICT RESOLVER <resolver2> on <conflict_type2>, ...;
A new catalog table pg_conflict_rel has been created to store
table-level conflict_type and conflict_resolver configurations given by
above DDL command.
Syntax to RESET resolvers:
ALTER TABLE <name> RESET CONFLICT RESOLVER on <conflict_type1>,
RESET CONFLICT RESOLVER on <conflict_type2>, ...;
Above RESET command will remove entry for that particular conflict_type for
the given table from pg_conflict_rel catalog table.
---
src/backend/catalog/dependency.c | 6 +
src/backend/catalog/objectaddress.c | 30 ++
src/backend/commands/tablecmds.c | 70 ++++
src/backend/parser/gram.y | 30 ++
src/backend/replication/logical/conflict.c | 330 +++++++++++++++++-
src/include/catalog/Makefile | 4 +-
src/include/catalog/meson.build | 1 +
src/include/catalog/pg_conflict.h | 2 +-
src/include/catalog/pg_conflict_rel.h | 58 +++
src/include/nodes/parsenodes.h | 2 +
src/include/replication/conflict.h | 13 +
.../regress/expected/conflict_resolver.out | 245 ++++++++++++-
src/test/regress/expected/oidjoins.out | 1 +
src/test/regress/sql/conflict_resolver.sql | 161 ++++++++-
14 files changed, 929 insertions(+), 24 deletions(-)
create mode 100644 src/include/catalog/pg_conflict_rel.h
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0489cbabcb..9dd78d9094 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -31,6 +31,7 @@
#include "catalog/pg_auth_members.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_conflict_rel.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -79,6 +80,7 @@
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
#include "parser/parsetree.h"
+#include "replication/conflict.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/fmgroids.h"
@@ -1444,6 +1446,10 @@ doDeletion(const ObjectAddress *object, int flags)
RemovePublicationById(object->objectId);
break;
+ case ConflictRelRelationId:
+ RemoveTableConflictById(object->objectId);
+ break;
+
case CastRelationId:
case CollationRelationId:
case ConversionRelationId:
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 2983b9180f..1ba0f74a2d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -30,6 +30,7 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
+#include "catalog/pg_conflict_rel.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
#include "catalog/pg_default_acl.h"
@@ -3005,6 +3006,35 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case ConflictRelRelationId:
+ {
+ HeapTuple confTup;
+ Datum typeDatum;
+ Datum resDatum;
+ char *contype;
+ char *conres;
+
+ confTup = SearchSysCache1(CONFLICTRELOID,
+ ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(confTup))
+ {
+ if (!missing_ok)
+ elog(ERROR, "cache lookup failed for table conflict %u",
+ object->objectId);
+ break;
+ }
+
+ typeDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, confTup,
+ Anum_pg_conflict_rel_confrtype);
+ resDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, confTup,
+ Anum_pg_conflict_rel_confrres);
+ contype = TextDatumGetCString(typeDatum);
+ conres = TextDatumGetCString(resDatum);
+ ReleaseSysCache(confTup);
+ appendStringInfo(&buffer, _("conflict_resolver %s on conflict_type %s"),
+ conres, contype);
+ break;
+ }
case ConstraintRelationId:
{
HeapTuple conTup;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dbfe0d6b1c..2d75297340 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -83,6 +83,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/conflict.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -662,6 +663,11 @@ static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab,
AlterTableUtilityContext *context);
static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation rel,
PartitionCmd *cmd, AlterTableUtilityContext *context);
+static void ATExecSetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode);
+static void
+ ATExecResetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode);
/* ----------------------------------------------------------------
* DefineRelation
@@ -1245,6 +1251,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
CloneForeignKeyConstraints(NULL, parent, rel);
+ /* Inherit conflict resolvers configuration from parent. */
+ InheritTableConflictResolvers(rel, parent);
+
table_close(parent, NoLock);
}
@@ -4543,6 +4552,8 @@ AlterTableGetLockLevel(List *cmds)
case AT_SetExpression:
case AT_DropExpression:
case AT_SetCompression:
+ case AT_SetConflictResolver:
+ case AT_ResetConflictResolver:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -5116,6 +5127,15 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_ResetConflictResolver: /* RESET CONFLICT RESOLVER */
+ case AT_SetConflictResolver: /* SET CONFLICT RESOLVER */
+ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE);
+ /* Recursion occurs during execution phase */
+ /* No command-specific prep needed except saving recurse flag */
+ if (recurse)
+ cmd->recurse = true;
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5528,6 +5548,14 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
ATExecMergePartitions(wqueue, tab, rel, (PartitionCmd *) cmd->def,
context);
break;
+ case AT_SetConflictResolver:
+ ATExecSetConflictResolver(rel, (ConflictResolverStmt *) cmd->def,
+ cmd->recurse, false, lockmode);
+ break;
+ case AT_ResetConflictResolver:
+ ATExecResetConflictResolver(rel, (ConflictResolverStmt *) cmd->def,
+ cmd->recurse, false, lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -6528,6 +6556,10 @@ alter_table_type_to_string(AlterTableType cmdtype)
return "ALTER COLUMN ... DROP IDENTITY";
case AT_ReAddStatistics:
return NULL; /* not real grammar */
+ case AT_SetConflictResolver:
+ return "SET CONFLICT RESOLVER";
+ case AT_ResetConflictResolver:
+ return "RESET CONFLICT RESOLVER";
}
return NULL;
@@ -15650,6 +15682,13 @@ CreateInheritance(Relation child_rel, Relation parent_rel, bool ispartition)
/* Match up the constraints and bump coninhcount as needed */
MergeConstraintsIntoExisting(child_rel, parent_rel);
+ /*
+ * Inherit resolvers configuration from parent if not explicitly set for
+ * child partition.
+ */
+ if (ispartition)
+ InheritTableConflictResolvers(child_rel, parent_rel);
+
/*
* OK, it looks valid. Make the catalog entries that show inheritance.
*/
@@ -16238,6 +16277,10 @@ RemoveInheritance(Relation child_rel, Relation parent_rel, bool expect_detached)
systable_endscan(scan);
table_close(catalogRelation, RowExclusiveLock);
+ /* Find inherited conflict resolvers and disinherit them */
+ if (is_partitioning)
+ ResetResolversInheritance(child_rel);
+
drop_parent_dependency(RelationGetRelid(child_rel),
RelationRelationId,
RelationGetRelid(parent_rel),
@@ -20766,3 +20809,30 @@ ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Keep the lock until commit. */
table_close(newPartRel, NoLock);
}
+
+/*
+ * ALTER TABLE <name> SET CONFLICT RESOLVER ...
+ */
+static void
+ATExecSetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ ValidateConflictTypeAndResolver(stmt->conflict_type, stmt->conflict_resolver, false);
+
+ SetTableConflictResolver(NULL, rel, stmt->conflict_type,
+ stmt->conflict_resolver,
+ recurse, recursing, lockmode);
+}
+
+/*
+ * ALTER TABLE <name> RESET CONFLICT RESOLVER ...
+ */
+static void
+ATExecResetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ ValidateConflictTypeAndResolver(stmt->conflict_type, stmt->conflict_resolver, true);
+
+ ResetTableConflictResolver(rel, stmt->conflict_type,
+ recurse, recursing, lockmode);
+}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 42726fe3a6..628b9d10e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -3015,6 +3015,36 @@ alter_table_cmd:
n->subtype = AT_GenericOptions;
n->def = (Node *) $1;
+ $$ = (Node *) n;
+ }
+ /*
+ * ALTER TABLE <name> SET CONFLICT RESOLVER <conflict_resolver>
+ * on <conflict_type>
+ */
+ | SET CONFLICT RESOLVER conflict_resolver ON conflict_type
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ ConflictResolverStmt *c = makeNode(ConflictResolverStmt);
+
+ c->conflict_resolver = $4;
+ c->conflict_type = $6;
+
+ n->subtype = AT_SetConflictResolver;
+ n->def = (Node *) c;
+
+ $$ = (Node *) n;
+ }
+ /* ALTER TABLE <name> RESET CONFLICT RESOLVER on <conflict_type> */
+ | RESET CONFLICT RESOLVER ON conflict_type
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ ConflictResolverStmt *c = makeNode(ConflictResolverStmt);
+
+ c->conflict_type = $5;
+
+ n->subtype = AT_ResetConflictResolver;
+ n->def = (Node *) c;
+
$$ = (Node *) n;
}
;
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index c330b21217..37615b6e85 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -19,8 +19,11 @@
#include "access/htup_details.h"
#include "access/skey.h"
#include "access/table.h"
+#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_conflict.h"
+#include "catalog/pg_conflict_rel.h"
+#include "catalog/pg_inherits.h"
#include "executor/executor.h"
#include "replication/conflict.h"
#include "replication/logicalproto.h"
@@ -28,8 +31,10 @@
#include "replication/origin.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/syscache.h"
#include "utils/timestamp.h"
#include "utils/syscache.h"
@@ -294,9 +299,9 @@ build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
*
* Return ConflictType enum value corresponding to given conflict_type.
*/
-static ConflictType
-validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver,
- bool isReset)
+ConflictType
+ValidateConflictTypeAndResolver(char *conflict_type, char *conflict_resolver,
+ bool isReset)
{
ConflictType type;
ConflictResolver resolver;
@@ -495,10 +500,9 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
HeapTuple newtup = NULL;
ConflictType type;
- type = validate_conflict_type_and_resolver(stmt->conflict_type,
- stmt->conflict_resolver,
- stmt->isReset);
-
+ type = ValidateConflictTypeAndResolver(stmt->conflict_type,
+ stmt->conflict_resolver,
+ stmt->isReset);
/* Prepare to update a tuple in pg_conflict system catalog */
memset(values, 0, sizeof(values));
@@ -526,7 +530,7 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
BTEqualStrategyNumber, F_TEXTEQ,
values[Anum_pg_conflict_conftype - 1]);
- pg_conflict = table_open(ConflictResRelationId, RowExclusiveLock);
+ pg_conflict = table_open(ConflictRelationId, RowExclusiveLock);
scan = systable_beginscan(pg_conflict, ConflictTypeIndexId, true,
NULL, 1, keys);
@@ -623,3 +627,313 @@ GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
return resolver;
}
+
+/*
+ * Update the table level conflict resolver for a conflict_type in
+ * pg_conflict_rel system catalog.
+ */
+void
+SetTableConflictResolver(Relation pg_conf_rel, Relation rel, char *conflict_type, char *conflict_resolver,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ Relation pg_conflict_rel;
+ Datum values[Natts_pg_conflict_rel];
+ bool nulls[Natts_pg_conflict_rel];
+ bool replaces[Natts_pg_conflict_rel];
+ HeapTuple oldtup;
+ HeapTuple newtup = NULL;
+ Oid relid = rel->rd_id;
+
+ /* Prepare to update a tuple. */
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ if (!pg_conf_rel)
+ pg_conflict_rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+ else
+ pg_conflict_rel = pg_conf_rel;
+
+ values[Anum_pg_conflict_rel_confrrelid - 1] = ObjectIdGetDatum(relid);
+ values[Anum_pg_conflict_rel_confrtype - 1] = CStringGetTextDatum(conflict_type);
+ values[Anum_pg_conflict_rel_confrres - 1] = CStringGetTextDatum(conflict_resolver);
+ values[Anum_pg_conflict_rel_confrinherited - 1] = BoolGetDatum(recursing);
+
+ oldtup = SearchSysCache2(CONFLICTRELTYPE,
+ values[Anum_pg_conflict_rel_confrrelid - 1],
+ values[Anum_pg_conflict_rel_confrtype - 1]);
+ if (HeapTupleIsValid(oldtup))
+ {
+ Form_pg_conflict_rel confForm = (Form_pg_conflict_rel) GETSTRUCT(oldtup);
+
+ /*
+ * Update resolver for recursing=false cases.
+ *
+ * recursing=true indicates it is a child parittion table and if it
+ * already has a resolver set then overwrite it only if it is an
+ * inherited resolver. We should not overwrite non-inherited resolvers
+ * for child partition tables during recursion.
+ */
+ if (!recursing || (recursing && confForm->confrinherited))
+ {
+ replaces[Anum_pg_conflict_rel_confrres - 1] = true;
+ replaces[Anum_pg_conflict_rel_confrinherited - 1] = !recursing;
+
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(pg_conflict_rel),
+ values, nulls, replaces);
+ CatalogTupleUpdate(pg_conflict_rel, &oldtup->t_self, newtup);
+ }
+ else
+ {
+ /*
+ * If we did not update resolver for this child table, do not
+ * update for child's child tables as well.
+ */
+ recurse = false;
+ }
+
+ ReleaseSysCache(oldtup);
+ }
+ /* If we didn't find an old tuple, insert a new one */
+ else
+ {
+ ObjectAddress myself,
+ referenced;
+ Oid conflict_oid;
+
+ conflict_oid = GetNewOidWithIndex(pg_conflict_rel, ConflictRelOidIndexId,
+ Anum_pg_conflict_rel_oid);
+ values[Anum_pg_conflict_rel_oid - 1] = ObjectIdGetDatum(conflict_oid);
+
+ newtup = heap_form_tuple(RelationGetDescr(pg_conflict_rel),
+ values, nulls);
+ CatalogTupleInsert(pg_conflict_rel, newtup);
+
+ /* Add dependency on the relation */
+ ObjectAddressSet(myself, ConflictRelRelationId, conflict_oid);
+ ObjectAddressSet(referenced, RelationRelationId, relid);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
+ }
+
+ if (HeapTupleIsValid(newtup))
+ heap_freetuple(newtup);
+
+ if (!pg_conf_rel)
+ table_close(pg_conflict_rel, RowExclusiveLock);
+
+ if (recurse && (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ List *children;
+ ListCell *lc;
+
+ children =
+ find_inheritance_children(RelationGetRelid(rel), lockmode);
+
+ foreach(lc, children)
+ {
+ Relation childrel;
+
+ /* find_inheritance_children already got lock */
+ childrel = table_open(lfirst_oid(lc), NoLock);
+ SetTableConflictResolver(NULL, childrel, conflict_type, conflict_resolver, recurse, true, lockmode);
+ table_close(childrel, NoLock);
+ }
+ }
+}
+
+/*
+ * Reset table level conflict_type configuration.
+ *
+ * Removes table's resolver configuration for given conflict_type from
+ * pg_conflict_rel system catalog.
+ */
+void
+ResetTableConflictResolver(Relation rel, char *conflict_type,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ Oid relid = rel->rd_id;
+ HeapTuple tup;
+ ObjectAddress confobj;
+ Form_pg_conflict_rel confForm;
+
+ tup = SearchSysCache2(CONFLICTRELTYPE,
+ ObjectIdGetDatum(relid),
+ CStringGetTextDatum(conflict_type));
+
+ /* Nothing to reset */
+ if (!HeapTupleIsValid(tup))
+ return;
+
+ confForm = (Form_pg_conflict_rel) GETSTRUCT(tup);
+
+ if (confForm->confrinherited && !recursing)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TABLE_DEFINITION),
+ errmsg("cannot reset inherited resolver for conflict type \"%s\" of relation \"%s\"",
+ conflict_type, RelationGetRelationName(rel))));
+
+ /*
+ * While resetting resolver on parent table, reset resolver for child
+ * partition table only if it is inherited one.
+ *
+ * OTOH, if user has invoked RESET directly on child partition table,
+ * ensure we reset only non-inherited resolvers. Inherited resolvers can
+ * not be reset alone on child table, RESET has to come through parent
+ * table.
+ */
+ if ((recursing && confForm->confrinherited) ||
+ (!recursing && !confForm->confrinherited))
+ {
+ ObjectAddressSet(confobj, ConflictRelRelationId, confForm->oid);
+ performDeletion(&confobj, DROP_CASCADE, 0);
+ }
+ else
+ {
+ /*
+ * If we did not reset resolver for this child table, do not reset for
+ * child's child tables as well.
+ */
+ recurse = false;
+ }
+
+ ReleaseSysCache(tup);
+
+ if (recurse && (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ List *children;
+ ListCell *lc;
+
+ children =
+ find_inheritance_children(RelationGetRelid(rel), lockmode);
+
+ foreach(lc, children)
+ {
+ Relation childrel;
+
+ /* find_inheritance_children already got lock */
+ childrel = table_open(lfirst_oid(lc), NoLock);
+ ResetTableConflictResolver(childrel, conflict_type, recurse, true, lockmode);
+ table_close(childrel, NoLock);
+ }
+ }
+}
+
+/*
+ * Inherit conflict resolvers from parent paritioned table.
+ *
+ * If child partition table has resolvers set already for some conflict
+ * types, do not overwrite those, inherit rest from parent.
+ *
+ * Used during attach partition operation.
+ */
+void
+InheritTableConflictResolvers(Relation child_rel, Relation parent_rel)
+{
+ Relation pg_conflict_rel;
+ SysScanDesc parent_scan;
+ ScanKeyData parent_key;
+ HeapTuple parent_tuple;
+ Oid parent_relid = RelationGetRelid(parent_rel);
+
+ Assert(parent_rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
+
+ pg_conflict_rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&parent_key,
+ Anum_pg_conflict_rel_confrrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(parent_relid));
+
+ parent_scan = systable_beginscan(pg_conflict_rel, InvalidOid, false, NULL, 1, &parent_key);
+
+ while (HeapTupleIsValid(parent_tuple = systable_getnext(parent_scan)))
+ {
+ Datum typeDatum;
+ Datum resDatum;
+ char *parent_conftype;
+ char *parent_confres;
+
+ typeDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, parent_tuple,
+ Anum_pg_conflict_rel_confrtype);
+ parent_conftype = TextDatumGetCString(typeDatum);
+
+ resDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, parent_tuple,
+ Anum_pg_conflict_rel_confrres);
+ parent_confres = TextDatumGetCString(resDatum);
+
+ SetTableConflictResolver(pg_conflict_rel, child_rel, parent_conftype, parent_confres, true, true, AccessExclusiveLock);
+ }
+
+ systable_endscan(parent_scan);
+ table_close(pg_conflict_rel, RowExclusiveLock);
+}
+
+/*
+ * Reset inheritance of conflict resolvers.
+ *
+ * Used during detach partition operation. Detach partition
+ * will not remove inherited resolvers but will mark them
+ * as non inherited.
+ */
+void
+ResetResolversInheritance(Relation rel)
+{
+ Relation pg_conflict_rel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+ Oid relid = RelationGetRelid(rel);
+
+ Assert(rel->rd_rel->relispartition);
+
+ pg_conflict_rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_conflict_rel_confrrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ scan = systable_beginscan(pg_conflict_rel, InvalidOid, false, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_conflict_rel conf =
+ (Form_pg_conflict_rel) GETSTRUCT(tuple);
+
+ if (conf->confrinherited)
+ {
+ HeapTuple copy_tuple = heap_copytuple(tuple);
+ Form_pg_conflict_rel copy_conf =
+ (Form_pg_conflict_rel) GETSTRUCT(copy_tuple);
+
+ copy_conf->confrinherited = false;
+ CatalogTupleUpdate(pg_conflict_rel, ©_tuple->t_self, copy_tuple);
+ heap_freetuple(copy_tuple);
+
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(pg_conflict_rel, RowExclusiveLock);
+}
+
+/*
+ * Remove the conflict resolver configuration by table conflict oid.
+ */
+void
+RemoveTableConflictById(Oid confid)
+{
+ Relation rel;
+ HeapTuple tup;
+
+ rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(CONFLICTRELOID, ObjectIdGetDatum(confid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for table conflict %u", confid);
+
+ CatalogTupleDelete(rel, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ table_close(rel, RowExclusiveLock);
+}
diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile
index 8619d73e5a..e192375fa0 100644
--- a/src/include/catalog/Makefile
+++ b/src/include/catalog/Makefile
@@ -82,7 +82,9 @@ CATALOG_HEADERS := \
pg_publication_rel.h \
pg_subscription.h \
pg_subscription_rel.h \
- pg_conflict.h
+ pg_conflict.h \
+ pg_conflict_rel.h
+
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h)
diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index 4d6732a303..28924f5df8 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -70,6 +70,7 @@ catalog_headers = [
'pg_subscription.h',
'pg_subscription_rel.h',
'pg_conflict.h',
+ 'pg_conflict_rel.h',
]
# The .dat files we need can just be listed alphabetically.
diff --git a/src/include/catalog/pg_conflict.h b/src/include/catalog/pg_conflict.h
index e3fe3e6d30..c080a0d0be 100644
--- a/src/include/catalog/pg_conflict.h
+++ b/src/include/catalog/pg_conflict.h
@@ -26,7 +26,7 @@
* typedef struct FormData_pg_conflict
* ----------------
*/
-CATALOG(pg_conflict,8688,ConflictResRelationId)
+CATALOG(pg_conflict,8688,ConflictRelationId)
{
#ifdef CATALOG_VARLEN /* variable-length fields start here */
text conftype BKI_FORCE_NOT_NULL; /* conflict type */
diff --git a/src/include/catalog/pg_conflict_rel.h b/src/include/catalog/pg_conflict_rel.h
new file mode 100644
index 0000000000..636398baf1
--- /dev/null
+++ b/src/include/catalog/pg_conflict_rel.h
@@ -0,0 +1,58 @@
+/* -------------------------------------------------------------------------
+ *
+ * pg_conflict_rel.h
+ * definition of the "table level conflict detection" system
+ * catalog (pg_conflict_rel)
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_conflict_rel.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ * -------------------------------------------------------------------------
+ */
+#ifndef PG_CONFLICT_REL_H
+#define PG_CONFLICT_REL_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_conflict_rel_d.h"
+
+/* ----------------
+ * pg_conflict_rel definition. cpp turns this into
+ * typedef struct FormData_pg_conflict_rel
+ * ----------------
+ */
+CATALOG(pg_conflict_rel,8881,ConflictRelRelationId)
+{
+ Oid oid; /* OID of the object itself */
+ Oid confrrelid BKI_LOOKUP(pg_class); /* Oid of the relation
+ * having resolver */
+ bool confrinherited; /* Is this an inherited configuration */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text confrtype BKI_FORCE_NOT_NULL; /* conflict type */
+ text confrres BKI_FORCE_NOT_NULL; /* conflict resolver */
+#endif
+} FormData_pg_conflict_rel;
+
+/* ----------------
+ * Form_pg_conflict_rel corresponds to a pointer to a row with
+ * the format of pg_conflict_rel relation.
+ * ----------------
+ */
+typedef FormData_pg_conflict_rel * Form_pg_conflict_rel;
+
+DECLARE_TOAST(pg_conflict_rel, 8882, 8883);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_conflict_rel_oid_index, 8884, ConflictRelOidIndexId, pg_conflict_rel, btree(oid oid_ops));
+
+DECLARE_UNIQUE_INDEX(pg_conflict_rel_type_index, 8885, ConflictRelTypeIndexId, pg_conflict_rel, btree(confrrelid oid_ops, confrtype text_ops));
+
+MAKE_SYSCACHE(CONFLICTRELOID, pg_conflict_rel_oid_index, 256);
+MAKE_SYSCACHE(CONFLICTRELTYPE, pg_conflict_rel_type_index, 256);
+
+#endif /* PG_CONFLICT_REL_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 1d41169d5e..7677cf023b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -2419,6 +2419,8 @@ typedef enum AlterTableType
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
AT_ReAddStatistics, /* internal to commands/tablecmds.c */
+ AT_SetConflictResolver, /* SET CONFLICT RESOLVER */
+ AT_ResetConflictResolver, /* RESET CONFLICT RESOLVER */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index d38a22ddde..3ba83a25cd 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -96,4 +96,17 @@ extern ConflictResolver GetConflictResolver(TupleTableSlot *localslot,
TimestampTz committs);
extern bool CanCreateFullTuple(Relation localrel,
LogicalRepTupleData *newtup);
+
+extern ConflictType ValidateConflictTypeAndResolver(char *conflict_type,
+ char *conflict_resolver,
+ bool isReset);
+extern void SetTableConflictResolver(Relation pg_conf_rel, Relation rel, char *conflict_type,
+ char *conflict_resolver, bool recurse,
+ bool recursing, LOCKMODE lockmode);
+extern void ResetTableConflictResolver(Relation rel, char *conflict_type,
+ bool recurse, bool recursing, LOCKMODE lockmode);
+extern void RemoveTableConflictById(Oid confid);
+extern void InheritTableConflictResolvers(Relation child_rel, Relation parent_rel);
+extern void ResetResolversInheritance(Relation rel);
+
#endif
diff --git a/src/test/regress/expected/conflict_resolver.out b/src/test/regress/expected/conflict_resolver.out
index c21486dbb4..802a7583d4 100644
--- a/src/test/regress/expected/conflict_resolver.out
+++ b/src/test/regress/expected/conflict_resolver.out
@@ -1,5 +1,8 @@
+--
+-- Test for configuration of global resolvers
+--
--check default global resolvers in system catalog
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
conftype | confres
----------------+------------------
delete_differ | last_update_wins
@@ -9,9 +12,7 @@ select * from pg_conflict order by conftype;
update_missing | apply_or_skip
(5 rows)
---
-- Test of SET/RESET CONFLICT RESOLVER with invalid names
---
SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
ERROR: aaaa is not a valid conflict type
SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
@@ -20,16 +21,14 @@ SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
ERROR: remote_apply is not a valid conflict resolver for conflict type delete_missing
RESET CONFLICT RESOLVER for 'ct'; -- fail
ERROR: ct is not a valid conflict type
---
-- Test of SET/RESET CONFLICT RESOLVER with valid names
---
SET CONFLICT RESOLVER 'error' for 'delete_missing';
SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
--check new resolvers are saved
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
conftype | confres
----------------+----------------
delete_differ | keep_local
@@ -42,7 +41,7 @@ select * from pg_conflict order by conftype;
RESET CONFLICT RESOLVER for 'delete_missing';
RESET CONFLICT RESOLVER for 'insert_exists';
--check resolvers are reset to default for delete_missing and insert_exists
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
conftype | confres
----------------+------------------
delete_differ | keep_local
@@ -52,3 +51,235 @@ select * from pg_conflict order by conftype;
update_missing | apply_or_error
(5 rows)
+--
+-- Test for configuration of table level resolvers
+--
+-- Test for ALTER TABLE..SET/RESET CONFLICT RESOLVER
+CREATE TABLE ptntable (
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+CREATE TABLE ptntable_1 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-02-01') TO ('2006-03-01');
+CREATE TABLE ptntable_2 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01')
+ PARTITION BY RANGE (logdate);
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'keep_local' ON 'insert_exists';
+--Expect 8 entries. 2 for parent and 2 for each child partition (inherited
+--resolvers)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | keep_local | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | keep_local | t
+ ptntable_2 | delete_missing | error | t
+ ptntable_2 | insert_exists | keep_local | t
+ ptntable_2_1 | delete_missing | error | t
+ ptntable_2_1 | insert_exists | keep_local | t
+(8 rows)
+
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'skip' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ';
+--For both ptntable_2 and ptntable_2_1, expect:
+--The resolver for 'delete_missing' changed from 'error' to 'skip'
+--The 'update_differ'configuration added
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | keep_local | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | keep_local | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | keep_local | t
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | keep_local | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+--Set resolvers on parent table again. It should not overwrite non inherited
+--entries for ptntable_2 and corresponding entries for ptntable_2_1
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'insert_exists';
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | t
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+--Reset of inherited entry ON child alone should result in error
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'insert_exists';
+ERROR: cannot reset inherited resolver for conflict type "insert_exists" of relation "ptntable_2"
+--Reset of non-inherited entry ON child alone should suceed
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'update_differ';
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | t
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+(8 rows)
+
+DROP TABLE ptntable_2;
+--Expect entries removed for both ptntable_2 and its child ptntable_2_1
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+(4 rows)
+
+-- Test for ALTER TABLE..SPLIT PARTITION and ALTER TABLE..MERGE PARTITIONS
+ALTER TABLE ptntable SPLIT PARTITION ptntable_1 INTO
+ (PARTITION ptntable_1_1 FOR VALUES FROM ('2006-02-01') TO ('2006-02-10'),
+ PARTITION ptntable_1_10 FOR VALUES FROM ('2006-02-10') TO ('2006-02-20'),
+ PARTITION ptntable_1_20 FOR VALUES FROM ('2006-02-20') TO ('2006-03-01'));
+--Expect split partitions to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+---------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1_1 | delete_missing | error | t
+ ptntable_1_1 | insert_exists | remote_apply | t
+ ptntable_1_10 | delete_missing | error | t
+ ptntable_1_10 | insert_exists | remote_apply | t
+ ptntable_1_20 | delete_missing | error | t
+ ptntable_1_20 | insert_exists | remote_apply | t
+(8 rows)
+
+ALTER TABLE ptntable MERGE PARTITIONS (ptntable_1_1, ptntable_1_10, ptntable_1_20)
+ INTO ptntable_1;
+--Expect merged partition to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+(4 rows)
+
+-- Test for ALTER TABLE..ATTACH PARTITION and CREATE TABLE..PARTITION OF
+CREATE TABLE ptntable_2(
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ',
+ SET CONFLICT RESOLVER 'skip' ON 'delete_missing';
+--Expect ptntable_2_1 to inherit resolvers of ptntable_2
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+--Expect 4 new entries for ptntable_2 and its child ptntable_2_1
+--Entried for ptntable_2_1 should be marked as inherited.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(8 rows)
+
+--Attach ptntable_2 to ptntable now
+ALTER TABLE ptntable ATTACH PARTITION ptntable_2
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01');
+--For both ptntable_2 and ptntable_2_1, expect:
+--The 'insert_exists' configuration inherited
+--Resolver for 'delete_missing' not overwritten
+--Resolver for 'update_differ' retained.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | t
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+-- Test for ALTER TABLE..DETACH PARTITION
+ALTER TABLE ptntable DETACH PARTITION ptntable_2;
+--All resolvers of ptntable_2 should be marked as non-inherited
+--All resolvers for ptntable_2_1's should still be marked as inherited (as
+--they are still inherited from ptntable_2)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | f
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+DROP TABLE ptntable_2;
+DROP TABLE ptntable;
+--Expect no resolvers
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+---------+-----------+----------+----------------
+(0 rows)
+
diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out
index 215eb899be..8caf852600 100644
--- a/src/test/regress/expected/oidjoins.out
+++ b/src/test/regress/expected/oidjoins.out
@@ -266,3 +266,4 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid}
NOTICE: checking pg_subscription {subowner} => pg_authid {oid}
NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid}
NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid}
+NOTICE: checking pg_conflict_rel {confrrelid} => pg_class {oid}
diff --git a/src/test/regress/sql/conflict_resolver.sql b/src/test/regress/sql/conflict_resolver.sql
index f83d14b229..b4574d1df4 100644
--- a/src/test/regress/sql/conflict_resolver.sql
+++ b/src/test/regress/sql/conflict_resolver.sql
@@ -1,17 +1,19 @@
+--
+-- Test for configuration of global resolvers
+--
+
--check default global resolvers in system catalog
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
---
-- Test of SET/RESET CONFLICT RESOLVER with invalid names
---
+
SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
RESET CONFLICT RESOLVER for 'ct'; -- fail
---
-- Test of SET/RESET CONFLICT RESOLVER with valid names
---
+
SET CONFLICT RESOLVER 'error' for 'delete_missing';
SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
@@ -19,10 +21,155 @@ SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
--check new resolvers are saved
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
RESET CONFLICT RESOLVER for 'delete_missing';
RESET CONFLICT RESOLVER for 'insert_exists';
--check resolvers are reset to default for delete_missing and insert_exists
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
+
+--
+-- Test for configuration of table level resolvers
+--
+
+
+-- Test for ALTER TABLE..SET/RESET CONFLICT RESOLVER
+
+CREATE TABLE ptntable (
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+
+CREATE TABLE ptntable_1 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-02-01') TO ('2006-03-01');
+
+CREATE TABLE ptntable_2 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01')
+ PARTITION BY RANGE (logdate);
+
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'keep_local' ON 'insert_exists';
+
+--Expect 8 entries. 2 for parent and 2 for each child partition (inherited
+--resolvers)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'skip' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ';
+
+--For both ptntable_2 and ptntable_2_1, expect:
+--The resolver for 'delete_missing' changed from 'error' to 'skip'
+--The 'update_differ'configuration added
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+--Set resolvers on parent table again. It should not overwrite non inherited
+--entries for ptntable_2 and corresponding entries for ptntable_2_1
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'insert_exists';
+
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+--Reset of inherited entry ON child alone should result in error
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'insert_exists';
+
+--Reset of non-inherited entry ON child alone should suceed
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'update_differ';
+
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+DROP TABLE ptntable_2;
+
+--Expect entries removed for both ptntable_2 and its child ptntable_2_1
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+
+-- Test for ALTER TABLE..SPLIT PARTITION and ALTER TABLE..MERGE PARTITIONS
+
+ALTER TABLE ptntable SPLIT PARTITION ptntable_1 INTO
+ (PARTITION ptntable_1_1 FOR VALUES FROM ('2006-02-01') TO ('2006-02-10'),
+ PARTITION ptntable_1_10 FOR VALUES FROM ('2006-02-10') TO ('2006-02-20'),
+ PARTITION ptntable_1_20 FOR VALUES FROM ('2006-02-20') TO ('2006-03-01'));
+
+--Expect split partitions to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+ALTER TABLE ptntable MERGE PARTITIONS (ptntable_1_1, ptntable_1_10, ptntable_1_20)
+ INTO ptntable_1;
+
+--Expect merged partition to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+
+-- Test for ALTER TABLE..ATTACH PARTITION and CREATE TABLE..PARTITION OF
+
+CREATE TABLE ptntable_2(
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ',
+ SET CONFLICT RESOLVER 'skip' ON 'delete_missing';
+
+--Expect ptntable_2_1 to inherit resolvers of ptntable_2
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+
+--Expect 4 new entries for ptntable_2 and its child ptntable_2_1
+--Entried for ptntable_2_1 should be marked as inherited.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+--Attach ptntable_2 to ptntable now
+ALTER TABLE ptntable ATTACH PARTITION ptntable_2
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01');
+
+--For both ptntable_2 and ptntable_2_1, expect:
+--The 'insert_exists' configuration inherited
+--Resolver for 'delete_missing' not overwritten
+--Resolver for 'update_differ' retained.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+
+-- Test for ALTER TABLE..DETACH PARTITION
+
+ALTER TABLE ptntable DETACH PARTITION ptntable_2;
+
+--All resolvers of ptntable_2 should be marked as non-inherited
+--All resolvers for ptntable_2_1's should still be marked as inherited (as
+--they are still inherited from ptntable_2)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+DROP TABLE ptntable_2;
+DROP TABLE ptntable;
+
+--Expect no resolvers
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
--
2.34.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
@ 2024-07-17 06:01 ` shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: shveta malik @ 2024-07-17 06:01 UTC (permalink / raw)
To: Nisha Moond <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Dilip Kumar <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>; shveta malik <[email protected]>
On Tue, Jul 9, 2024 at 3:09 PM shveta malik <[email protected]> wrote:
>
> On Fri, Jul 5, 2024 at 5:12 PM Nisha Moond <[email protected]> wrote:
> >
> > Thank you Ajin for reporting the issue, This is now fixed with the
> > v4-0003 patch.
>
> Please find v5 patch-set. Changes are:
>
> 1) patch003:
> Added test cases for all resolvers (034_conflict_resolver.pl).
>
> 2) Patch004:
> a) Emit error while resolving conflict if conflict resolver is default
> 'last_update_wins' but track_commit_timetsamp is not enabled.
> b) Emit Warning during create and alter subscription when
> 'detect_conflict' is ON but 'track_commit_timetsamp' is not enabled.
> c) Restrict start of pa worker if either max-clock-skew is configured
> or conflict detection and resolution is enabled for a subscription.
> d) Implement clock-skew delay/error when changes are applied from a
> file (apply_spooled_messages).
> e) Implement clock-skew delay while applying prepared changes (two
> phase txns). The prepare-timestamp to be considered as base for
> clock-skew handling as well as for last_update_win resolver.
> <TODO: This needs to be analyzed and tested further to see if there is
> any side effect of taking prepare-timestamp as base.>
>
> Thanks Ajin fo working on 1.
> Thanks Nisha for working on 2a,2b.
>
Please find v6 patch-set. Changes are:
1) patch003:
1a) Improved log and restructured code around it.
1b) added test case for delete_differ.
2) patch004:
2a) Local and remote timestamps were logged incorrectly due to a bug,
corrected that.
2b) Added tests for last_update_wins.
2c) Added a cap on wait time; introduced a new GUC for this. Apply
worker will now error out without waiting if the computed wait exceeds
this GUC's value.
2d) Restricted enabling two_phase and detect_conflict together for a
subscription. This is because the time based resolvers may result in
data divergence for two phase commit transactions if prepare-timestamp
is used for comparison.
Thanks Nisha for working on 1a to 2b.
thanks
Shveta
Attachments:
[application/octet-stream] v6-0003-Conflict-resolvers-for-insert-update-and-delete.patch (55.8K, ../../CAJpy0uDAMtrqXWFtxrXSwTNO8awFAoENtbo47Czj-67k3OVMyQ@mail.gmail.com/2-v6-0003-Conflict-resolvers-for-insert-update-and-delete.patch)
download | inline diff:
From a104b11c86d5245a5595b5fb061fa6776d0bc704 Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Wed, 3 Jul 2024 15:46:58 +0530
Subject: [PATCH v6 3/5] Conflict resolvers for insert, update, and delete
This patch introduces support for handling conflicts with the following resolutions:
- For INSERT conflicts:
- insert_exists: remote_apply, keep_local, error
- For UPDATE conflicts:
- update_differ: remote_apply, keep_local, error
- update_missing: apply_or_skip, apply_or_error, skip, error
- For DELETE conflicts:
- delete_missing: skip, error
- delete_differ: remote_apply, keep_local, error
Each of the conflict type now has defined actions to manage data synchronization
between Publisher and subscriber databases.
---
src/backend/executor/execReplication.c | 79 ++-
src/backend/replication/logical/conflict.c | 205 ++++++-
src/backend/replication/logical/worker.c | 355 ++++++++---
src/include/executor/executor.h | 5 +-
src/include/replication/conflict.h | 14 +-
src/test/subscription/meson.build | 1 +
src/test/subscription/t/029_on_error.pl | 7 +
.../subscription/t/034_conflict_resolver.pl | 579 ++++++++++++++++++
8 files changed, 1113 insertions(+), 132 deletions(-)
create mode 100755 src/test/subscription/t/034_conflict_resolver.pl
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index f01927a933..b18723e56b 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -566,7 +566,8 @@ retry:
*/
void
ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot)
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot)
{
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
@@ -602,21 +603,80 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
+ /*
+ * If caller has passed non null conflictslot, check all the unique
+ * indexes for potential conflicts. If the configured resolver is in
+ * favour of apply, give the conflicted tuple information in
+ * conflictslot.
+ */
+ if (conflictslot)
+ {
+ foreach_oid(uniqueidx, conflictindexes)
+ {
+ /*
+ * Return if any conflict is found other than one with 'ERROR'
+ * resolver configured. In case of 'ERROR' resolver, emit
+ * error here; otherwise return to caller for resolutions.
+ */
+ if (FindConflictTuple(resultRelInfo, estate, uniqueidx,
+ slot, &(*conflictslot)))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+ ConflictResolver resolver;
+ bool apply_remote = false;
+
+ GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
+ resolver = GetConflictResolver(rel, CT_INSERT_EXISTS,
+ &apply_remote, NULL);
+
+ ReportApplyConflict(CT_INSERT_EXISTS, resolver, rel,
+ uniqueidx, xmin, origin, committs,
+ *conflictslot, apply_remote);
+
+ /* Nothing to apply, free the resources */
+ if (!apply_remote)
+ {
+ ExecDropSingleTupleTableSlot(*conflictslot);
+ *conflictslot = NULL;
+ }
+
+ return;
+ }
+ }
+ }
+
+
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
- conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
-
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, false,
conflictindexes, &conflict,
conflictindexes, false);
- /* Re-check all the unique indexes for potential conflicts */
+ /*
+ * Re-check all the unique indexes for potential conflicts.
+ *
+ * The Reason for doing the check again is:
+ *
+ * 1. If the remote change violated multiple unique constraints, the
+ * current resolution cannot solve it, so we still need to report this
+ * conflict after resolution.
+ *
+ * 2. If the local data is changed immediately after converting the
+ * insert to update but before updating the data, then the conflict
+ * can still happen, and we may need to report it again.
+ *
+ * XXX: Needs further review and discussion on usefulness of this
+ * repeated call.
+ */
foreach_oid(uniqueidx, (conflict ? conflictindexes : NIL))
{
- TupleTableSlot *conflictslot;
/*
* Reports the conflict if any.
@@ -631,15 +691,16 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
* cost of finding the tuple should be acceptable in this case.
*/
if (list_member_oid(recheckIndexes, uniqueidx) &&
- FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &conflictslot))
+ FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &(*conflictslot)))
{
RepOriginId origin;
TimestampTz committs;
TransactionId xmin;
- GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
- ReportApplyConflict(ERROR, CT_INSERT_EXISTS, rel, uniqueidx,
- xmin, origin, committs, conflictslot);
+ GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
+ ReportApplyConflict(CT_INSERT_EXISTS, CR_ERROR, rel, uniqueidx,
+ xmin, origin, committs, *conflictslot,
+ false);
}
}
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 93d9659752..574afe71f3 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -21,13 +21,17 @@
#include "access/table.h"
#include "catalog/indexing.h"
#include "catalog/pg_conflict.h"
+#include "executor/executor.h"
#include "replication/conflict.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "replication/origin.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/timestamp.h"
+#include "utils/syscache.h"
const char *const ConflictTypeNames[] = {
[CT_INSERT_EXISTS] = "insert_exists",
@@ -86,11 +90,13 @@ const int ConflictTypeDefaultResolvers[] = {
};
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
-static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin,
+static int errdetail_apply_conflict(ConflictType type,
+ ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
RepOriginId localorigin,
TimestampTz localts,
- TupleTableSlot *conflictslot);
+ TupleTableSlot *conflictslot,
+ bool apply_remote);
/*
@@ -126,22 +132,32 @@ GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
}
/*
- * Report a conflict when applying remote changes.
+ * Report conflict and resolution applied while applying remote changes.
*/
void
-ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
- Oid conflictidx, TransactionId localxmin,
- RepOriginId localorigin, TimestampTz localts,
- TupleTableSlot *conflictslot)
+ReportApplyConflict(ConflictType type, ConflictResolver resolver,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot,
+ bool apply_remote)
{
+ int elevel;
+
+ if (resolver == CR_ERROR ||
+ (resolver == CR_APPLY_OR_ERROR && !apply_remote))
+ elevel = ERROR;
+ else
+ elevel = LOG;
+
ereport(elevel,
errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
- errmsg("conflict %s detected on relation \"%s.%s\"",
+ errmsg("conflict %s detected on relation \"%s.%s\". Resolution: %s",
ConflictTypeNames[type],
get_namespace_name(RelationGetNamespace(localrel)),
- RelationGetRelationName(localrel)),
- errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
- localts, conflictslot));
+ RelationGetRelationName(localrel),
+ ConflictResolverNames[resolver]),
+ errdetail_apply_conflict(type, resolver, conflictidx, localxmin, localorigin,
+ localts, conflictslot, apply_remote));
}
/*
@@ -176,13 +192,21 @@ InitConflictIndexes(ResultRelInfo *relInfo)
}
/*
- * Add an errdetail() line showing conflict detail.
+ * Add an errdetail() line showing conflict and resolution details.
*/
static int
-errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot)
+errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot, bool apply_remote)
{
+ char *applymsg;
+
+ if (apply_remote)
+ applymsg = "applying the remote changes.";
+ else
+ applymsg = "ignoring the remote changes.";
+
switch (type)
{
case CT_INSERT_EXISTS:
@@ -195,27 +219,43 @@ errdetail_apply_conflict(ConflictType type, Oid conflictidx,
char *index_value = build_index_value_desc(conflictidx,
conflictslot);
- if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
- else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ if (resolver == CR_ERROR)
+ {
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ return errdetail("Key already exists, %s", applymsg);
}
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin,
+ timestamptz_to_str(localts), applymsg);
case CT_UPDATE_MISSING:
- return errdetail("Did not find the row to be updated.");
+ if (resolver == CR_APPLY_OR_SKIP && !apply_remote)
+ return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
+ else if (resolver == CR_APPLY_OR_ERROR && !apply_remote)
+ return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
+ else if (apply_remote)
+ return errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
+ applymsg);
+ else
+ return errdetail("Did not find the row to be updated, %s",
+ applymsg);
case CT_DELETE_MISSING:
return errdetail("Did not find the row to be deleted.");
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin,
+ timestamptz_to_str(localts), applymsg);
}
return 0; /* silence compiler warning */
@@ -329,6 +369,67 @@ validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver
return type;
}
+/*
+ * Get the global conflict resolver configured for given conflict type
+ */
+static ConflictResolver
+get_conflict_resolver_internal(ConflictType type)
+{
+
+ ConflictResolver resolver;
+ HeapTuple tuple;
+ Datum datum;
+ char *conflict_res;
+
+ /*
+ * XXX: Currently, we fetch the conflict resolver from cache for each
+ * conflict detection. If needed, we can keep the info in global variable
+ * and fetch from cache only once after cache invalidation.
+ */
+ tuple = SearchSysCache1(CONFLICTTYPE,
+ CStringGetTextDatum(ConflictTypeNames[type]));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for conflict type %u", type);
+
+ datum = SysCacheGetAttrNotNull(CONFLICTTYPE,
+ tuple, Anum_pg_conflict_confres);
+
+ conflict_res = TextDatumGetCString(datum);
+
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_res) == 0)
+ break;
+ }
+
+ ReleaseSysCache(tuple);
+ return resolver;
+}
+
+/*
+ * Check if a full tuple can be created from the new tuple.
+ * Return true if yes, false otherwise.
+ */
+static bool
+can_create_full_tuple(Relation localrel,
+ LogicalRepTupleData *newtup)
+{
+ int i;
+ int local_att = RelationGetNumberOfAttributes(localrel);
+
+ if (newtup->ncols != local_att)
+ return false;
+
+ for (i = 0; i < newtup->ncols; i++)
+ {
+ if (newtup->colstatus[i] == LOGICALREP_COLUMN_UNCHANGED)
+ return false;
+ }
+
+ return true;
+}
+
/*
* Execute Conflict Resolver Stmt
*
@@ -403,3 +504,47 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
table_close(pg_conflict, RowExclusiveLock);
}
+
+/*
+ * Find the global resolver for the given conflict type.
+ *
+ * Set 'apply_remote' to true if remote tuple should be applied,
+ * false otherwise.
+ */
+ConflictResolver
+GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
+ LogicalRepTupleData *newtup)
+{
+ ConflictResolver resolver;
+
+ resolver = get_conflict_resolver_internal(type);
+
+ switch (resolver)
+ {
+ case CR_REMOTE_APPLY:
+ *apply_remote = true;
+ break;
+ case CR_APPLY_OR_SKIP:
+ if (can_create_full_tuple(localrel, newtup))
+ *apply_remote = true;
+ else
+ *apply_remote = false;
+ break;
+ case CR_APPLY_OR_ERROR:
+ if (can_create_full_tuple(localrel, newtup))
+ *apply_remote = true;
+ else
+ *apply_remote = false;
+ break;
+ case CR_KEEP_LOCAL:
+ case CR_SKIP:
+ case CR_ERROR:
+ *apply_remote = false;
+ break;
+ default:
+ elog(ERROR, "Conflict %s is detected! Unrecogonized conflict resolution method",
+ ConflictTypeNames[type]);
+ }
+
+ return resolver;
+}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 7e7de195ed..9859f0f2bd 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -382,7 +382,9 @@ static void send_feedback(XLogRecPtr recvpos, bool force, bool requestReply);
static void apply_handle_commit_internal(LogicalRepCommitData *commit_data);
static void apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot);
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry);
static void apply_handle_update_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
TupleTableSlot *remoteslot,
@@ -2431,10 +2433,10 @@ apply_handle_insert(StringInfo s)
/* For a partitioned table, insert the tuple into a partition. */
if (rel->localrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
apply_handle_tuple_routing(edata,
- remoteslot, NULL, CMD_INSERT);
+ remoteslot, &newtup, CMD_INSERT);
else
apply_handle_insert_internal(edata, edata->targetRelInfo,
- remoteslot);
+ remoteslot, &newtup, rel);
finish_edata(edata);
@@ -2457,9 +2459,13 @@ apply_handle_insert(StringInfo s)
static void
apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot)
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry)
{
EState *estate = edata->estate;
+ MemoryContext oldctx;
+ TupleTableSlot *conflictslot = NULL;
/* We must open indexes here. */
ExecOpenIndices(relinfo, MySubscription->detectconflict);
@@ -2469,7 +2475,35 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
- ExecSimpleRelationInsert(relinfo, estate, remoteslot);
+
+ if (MySubscription->detectconflict)
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, &conflictslot);
+ else
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, NULL);
+
+
+ /* Apply remote tuple by converting INSERT to UPDATE */
+ if (conflictslot)
+ {
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, conflictslot, rel_entry, newtup);
+ MemoryContextSwitchTo(oldctx);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
+
+ /* Do the update */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, conflictslot,
+ remoteslot);
+
+ EvalPlanQualEnd(&epqstate);
+ ExecDropSingleTupleTableSlot(conflictslot);
+ }
/* Cleanup. */
ExecCloseIndices(relinfo);
@@ -2651,6 +2685,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
TupleTableSlot *localslot;
bool found;
MemoryContext oldctx;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2674,35 +2710,66 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* If conflict detection is enabled, check whether the local tuple was
- * modified by a different origin. If detected, report the conflict.
+ * modified by a different origin. If detected, report the conflict
+ * and configured resolver.
*/
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL);
- /* Process and store remote tuple in the slot */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
- MemoryContextSwitchTo(oldctx);
+ ReportApplyConflict(CT_UPDATE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- EvalPlanQualSetSlot(&epqstate, remoteslot);
+ /*
+ * Apply the change if configured resolver is in favor of that, else
+ * ignore the remote update.
+ */
+ if (apply_remote)
+ {
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ MemoryContextSwitchTo(oldctx);
- /* Do the actual update. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
- ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
- remoteslot);
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
+
+ /* Do the actual update. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
+ remoteslot);
+ }
}
else
{
/*
- * The tuple to be updated could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be updated could not be found. Report the conflict. If
+ * the configured resolver is in favor of applying the change, convert
+ * UPDATE to INSERT and apply the change.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup);
+
+ ReportApplyConflict(CT_UPDATE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot, relmapentry, newtup);
+ slot_fill_defaults(relmapentry, estate, remoteslot);
+ MemoryContextSwitchTo(oldctx);
+
+ if (apply_remote)
+ apply_handle_insert_internal(edata, relinfo, remoteslot,
+ newtup, relmapentry);
+ }
}
/* Cleanup. */
@@ -2815,6 +2882,8 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
EPQState epqstate;
TupleTableSlot *localslot;
bool found;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2836,24 +2905,42 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_DELETE_DIFFER,
+ &apply_remote, NULL);
- EvalPlanQualSetSlot(&epqstate, localslot);
+ ReportApplyConflict(CT_DELETE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
+
+ if (apply_remote)
+ {
+ EvalPlanQualSetSlot(&epqstate, localslot);
+
+ /* Do the actual delete. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
+ ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
+ }
- /* Do the actual delete. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
- ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
}
else
{
/*
- * The tuple to be deleted could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be deleted could not be found. Based on resolver
+ * configured, either skip and log a message or emit an error.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_DELETE_MISSING,
+ &apply_remote, NULL);
+
+ /* Resolver is set to skip, thus report the conflict and skip */
+ if (!apply_remote)
+ ReportApplyConflict(CT_DELETE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+ }
}
/* Cleanup. */
@@ -2986,19 +3073,21 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
}
MemoryContextSwitchTo(oldctx);
- /* Check if we can do the update or delete on the leaf partition. */
- if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ if (operation == CMD_INSERT || operation == CMD_UPDATE ||
+ operation == CMD_DELETE)
{
- part_entry = logicalrep_partition_open(relmapentry, partrel,
- attrmap);
- check_relation_updatable(part_entry);
+ part_entry = logicalrep_partition_open(relmapentry, partrel, attrmap);
+
+ /* Check if we can do the update or delete on the leaf partition */
+ if (operation != CMD_INSERT)
+ check_relation_updatable(part_entry);
}
switch (operation)
{
case CMD_INSERT:
apply_handle_insert_internal(edata, partrelinfo,
- remoteslot_part);
+ remoteslot_part, newtup, part_entry);
break;
case CMD_DELETE:
@@ -3023,6 +3112,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
RepOriginId localorigin;
TransactionId localxmin;
TimestampTz localts;
+ LogicalRepRelMapEntry *part_entry_new = NULL;
+ ConflictResolver resolver;
+ bool apply_remote = true;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3032,38 +3124,79 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
if (!found)
{
/*
- * The tuple to be updated could not be found. Do nothing
- * except for emitting a log message.
+ * The tuple to be updated could not be found. Report the
+ * conflict and resolver. And take action based on the
+ * configured resolver.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING,
- partrel, InvalidOid,
- InvalidTransactionId,
- InvalidRepOriginId, 0, NULL);
-
- return;
+ {
+ resolver = GetConflictResolver(partrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup);
+
+ ReportApplyConflict(CT_UPDATE_MISSING, resolver, partrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+
+ if (apply_remote)
+ {
+ /*
+ * Resolver is in favour of applying the remote
+ * changes. Prepare the slot for the INSERT.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot_part, part_entry, newtup);
+ slot_fill_defaults(part_entry, estate, remoteslot_part);
+ MemoryContextSwitchTo(oldctx);
+ }
+ }
}
+ else
+ {
+ /*
+ * The tuple to be updated is found. If conflict detection
+ * is enabled, check whether the local tuple was modified
+ * by a different origin. If detected, report and resolve
+ * the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ {
+ resolver = GetConflictResolver(partrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL);
- /*
- * If conflict detection is enabled, check whether the local
- * tuple was modified by a different origin. If detected,
- * report the conflict.
- */
- if (MySubscription->detectconflict &&
- GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
- localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
- InvalidOid, localxmin, localorigin,
- localts, NULL);
+ ReportApplyConflict(CT_UPDATE_DIFFER, resolver, partrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- /*
- * Apply the update to the local tuple, putting the result in
- * remoteslot_part.
- */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
- MemoryContextSwitchTo(oldctx);
+ if (apply_remote)
+ {
+ /*
+ * We can reach here in two cases:
+ *
+ * 1. If we found a tuple but conflict detection is
+ * OFF
+ *
+ * 2. If we found a tuple and resolver is in favor of
+ * applying the change when conflict detection is ON
+ *
+ * Putting the result in remoteslot_part.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
+ MemoryContextSwitchTo(oldctx);
+ }
+ else
+
+ /*
+ * apply_remote can be toggled if resolver for
+ * update_differ is set to skip. Ignore remote update.
+ */
+ return;
+
+ }
/*
* Does the updated tuple still satisfy the current
@@ -3073,27 +3206,59 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ExecPartitionCheck(partrelinfo, remoteslot_part, estate,
false))
{
- /*
- * Yes, so simply UPDATE the partition. We don't call
- * apply_handle_update_internal() here, which would
- * normally do the following work, to avoid repeating some
- * work already done above to find the local tuple in the
- * partition.
- */
- EPQState epqstate;
-
- EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(partrelinfo, false);
-
- EvalPlanQualSetSlot(&epqstate, remoteslot_part);
- TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
- ACL_UPDATE);
- ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
- localslot, remoteslot_part);
- ExecCloseIndices(partrelinfo);
- EvalPlanQualEnd(&epqstate);
+ if (found && apply_remote)
+ {
+ /*
+ * Yes, so simply UPDATE the partition. We don't call
+ * apply_handle_update_internal() here, which would
+ * normally do the following work, to avoid repeating
+ * some work already done above to find the local
+ * tuple in the partition.
+ *
+ * Do the update in cases - 1. conflict detection is
+ * OFF and found a tuple 2. conflict detection is ON,
+ * update_differ conflict is detected for the found
+ * tuple and the resolver is in favour of applying the
+ * update.
+ */
+
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+ ExecOpenIndices(partrelinfo, false);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot_part);
+ TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
+ ACL_UPDATE);
+ ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
+ localslot, remoteslot_part);
+ ExecCloseIndices(partrelinfo);
+ EvalPlanQualEnd(&epqstate);
+
+ }
+ else if (apply_remote)
+ {
+ /*
+ * Tuple is not found but update_missing resolver is
+ * in favour of applying the change as INSERT.
+ */
+ apply_handle_insert_internal(edata, partrelinfo,
+ remoteslot_part, newtup,
+ part_entry);
+ }
}
- else
+
+ /*
+ * Updated tuple doesn't satisfy the current partition's
+ * constraint.
+ *
+ * If conflict detection is OFF, proceed by always applying
+ * the update (as 'apply_remote' is by default true). If
+ * conflict detection is ON, 'apply_remote' can be OFF as well
+ * if the resolver for update_missing conflict conveys to skip
+ * the update.
+ */
+ else if (apply_remote)
{
/* Move the tuple into the new partition. */
@@ -3134,10 +3299,16 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
get_namespace_name(RelationGetNamespace(partrel_new)),
RelationGetRelationName(partrel_new));
- /* DELETE old tuple found in the old partition. */
- apply_handle_delete_internal(edata, partrelinfo,
- localslot,
- part_entry->localindexoid);
+ /*
+ * If tuple is found, delete it from old partition. We can
+ * reach this flow even for the case when the 'found' flag
+ * is false for 'update_missing' conflict and resolver is
+ * in favor of inserting the tuple.
+ */
+ if (found)
+ apply_handle_delete_internal(edata, partrelinfo,
+ localslot,
+ part_entry->localindexoid);
/* INSERT new tuple into the new partition. */
@@ -3153,19 +3324,25 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
map = ExecGetRootToChildMap(partrelinfo_new, estate);
if (map != NULL)
{
+ attrmap = map->attrMap;
remoteslot_part = execute_attr_map_slot(map->attrMap,
remoteslot,
remoteslot_part);
}
else
{
+ attrmap = NULL;
remoteslot_part = ExecCopySlot(remoteslot_part,
remoteslot);
slot_getallattrs(remoteslot);
}
MemoryContextSwitchTo(oldctx);
+ part_entry_new = logicalrep_partition_open(part_entry,
+ partrel_new,
+ attrmap);
apply_handle_insert_internal(edata, partrelinfo_new,
- remoteslot_part);
+ remoteslot_part, newtup,
+ part_entry_new);
}
}
break;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 9770752ea3..bc72b1f4fd 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,8 @@
#include "fmgr.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/memutils.h"
@@ -655,7 +657,8 @@ extern bool RelationFindReplTupleSeq(Relation rel, LockTupleMode lockmode,
TupleTableSlot *searchslot, TupleTableSlot *outslot);
extern void ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot);
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictsloty);
extern void ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
EState *estate, EPQState *epqstate,
TupleTableSlot *searchslot, TupleTableSlot *slot);
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index c6023e0914..b06c0f75a0 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -13,6 +13,8 @@
#include "catalog/objectaddress.h"
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/relcache.h"
#include "utils/timestamp.h"
@@ -74,10 +76,16 @@ typedef enum ConflictResolver
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
-extern void ReportApplyConflict(int elevel, ConflictType type,
+extern void ReportApplyConflict(ConflictType type, ConflictResolver resolver,
Relation localrel, Oid conflictidx,
- TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot);
+ TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot, bool apply_remote);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
extern void ExecConflictResolverStmt(ConflictResolverStmt *stmt);
+extern ConflictResolver GetConflictResolver(Relation localrel,
+ ConflictType type,
+ bool *apply_remote,
+ LogicalRepTupleData *newtup);
+extern bool CanCreateFullTuple(Relation localrel, LogicalRepTupleData *newtup);
#endif
diff --git a/src/test/subscription/meson.build b/src/test/subscription/meson.build
index c591cd7d61..00ade29b02 100644
--- a/src/test/subscription/meson.build
+++ b/src/test/subscription/meson.build
@@ -40,6 +40,7 @@ tests += {
't/031_column_list.pl',
't/032_subscribe_use_index.pl',
't/033_run_as_table_owner.pl',
+ 't/034_conflict_resolver.pl',
't/100_bugs.pl',
],
},
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 496a3c6cd9..c746191601 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -113,6 +113,10 @@ $node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
+# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
+$node_subscriber->safe_psql('postgres',"set conflict resolver 'error' for 'insert_exists'");
+
+
# Initial synchronization failure causes the subscription to be disabled.
$node_subscriber->poll_query_until('postgres',
"SELECT subenabled = false FROM pg_catalog.pg_subscription WHERE subname = 'sub'"
@@ -177,6 +181,9 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
+# Reset conflict resolver for 'insert_exist' conflict type to default.
+$node_subscriber->safe_psql('postgres',"reset conflict resolver for 'insert_exists'");
+
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
new file mode 100755
index 0000000000..0fb5ef9a86
--- /dev/null
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -0,0 +1,579 @@
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+###############################
+# Setup
+###############################
+
+# Initialize publisher node
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf',
+ qq(max_prepared_transactions = 10));
+$node_publisher->start;
+
+# Create subscriber node with track_commit_timestamp_set
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ qq(track_commit_timestamp = on));
+$node_subscriber->start;
+
+# Create table on publisher
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Create similar table on subscriber
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Setup logical replication
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE conf_tab");
+
+# Create the subscription
+my $appname = 'tap_sub';
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on);");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT conftype, confres FROM pg_conflict ORDER BY conftype"
+);
+is( $result, qq(delete_differ|remote_apply
+delete_missing|skip
+insert_exists|remote_apply
+update_differ|remote_apply
+update_missing|apply_or_skip),
+ "confirm that the default conflict resolvers are in place"
+);
+
+############################################
+# Test 'remote_apply' for 'insert_exists'
+############################################
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
+
+# Create conflicting data on the publisher
+my $log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=1);"
+);
+
+is($result, 'frompub', "remote data is kept");
+
+
+########################################
+# Test 'keep_local' for 'insert_exists'
+########################################
+
+# Change CONFLICT RESOLVER of insert_exists to keep_local
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'fromsub')");
+
+# Confirm that row is updated
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "data 2 from local is inserted");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that remote insert is ignored and the local row is kept
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "data from local is kept");
+
+###################################
+# Test 'error' for 'insert_exists'
+###################################
+
+# Change CONFLICT RESOLVER of insert_exists to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'insert_exists';"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'fromsub')");
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'frompub')");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Truncate table on subscriber to get rid of the error
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+###################################
+# Test 'skip' for 'delete_missing'
+###################################
+
+# Delete row on publisher that is not present on the subscriber and confirm that it is skipped
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=2);");
+
+# Confirm that the missing row is skipped because 'delete_missing' is set to 'skip'
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+####################################
+# Test 'error' for 'delete_missing'
+####################################
+
+# Change CONFLICT RESOLVER of delete_missing to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'delete_missing';"
+);
+
+# Capture the log offset before performing the delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+
+# Perform the delete on the publisher
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=1);");
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate table on subscriber
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+
+#########################################
+# Test 'remote_apply' for 'delete_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote delete the local updated row
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);"
+);
+
+is($result, '', "delete from remote is applied");
+
+#########################################
+# Test 'keep_local' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'delete_differ';"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);"
+);
+
+is($result, 'frompubnew', "update from remote is kept");
+
+#########################################
+# Test 'keep_local' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'keep_local' for 'update_differ';"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);"
+);
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'update_differ';"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###########################################
+# Test 'apply_or_skip' for 'update_missing'
+###########################################
+
+#test the apply part
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=3);"
+);
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the skip part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(4,'frompub',repeat('abcdefghij', 200));");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=4);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/DETAIL: Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update./, $log_offset);
+
+###########################################
+# Test 'apply_or_error' for 'update_missing'
+###########################################
+
+# test the apply part
+
+#Change CONFLICT RESOLVER of update_missing to apply_or_error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';"
+);
+
+# Create new row on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (5,'frompub');");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=5);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=5);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=5);"
+);
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the error part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(6,'frompub',repeat('abcdefghij', 200));");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=6);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=6);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###################################
+# Test 'skip' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to skip
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'skip' for 'update_missing';"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the update does not change anything on the subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);"
+);
+
+is($result, '', "update from remote is skipped on the subscriber");
+
+###################################
+# Test 'error' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to error
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'error' for 'update_missing';"
+);
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnewer' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/, $log_offset);
+
+done_testing();
--
2.34.1
[application/octet-stream] v6-0004-Manage-Clock-skew-and-implement-last_update_wins.patch (49.3K, ../../CAJpy0uDAMtrqXWFtxrXSwTNO8awFAoENtbo47Czj-67k3OVMyQ@mail.gmail.com/3-v6-0004-Manage-Clock-skew-and-implement-last_update_wins.patch)
download | inline diff:
From 691bd39b685677e2f316087c679eb442132d1a15 Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Tue, 16 Jul 2024 13:54:23 +0530
Subject: [PATCH v6 4/5] Manage Clock skew and implement last_update_wins
This patch attempts to manage clock skew between nodes by
introducing two new GUCs:
a) max_logical_rep_clock_skew
b) max_logical_rep_clock_skew_action
c) max_logical_rep_clock_skew_wait
If the timestamp of the currently replayed transaction is in the future
compared to the current time on the subscriber and the difference is
larger than 'max_logical_rep_clock_skew', then the action configured
in 'max_logical_rep_clock_skew_action' is performed by the apply worker.
If user configures 'wait' in 'max_logical_rep_clock_skew_action' and
actual clock skew is large while 'max_logical_rep_clock_skew' is small,
the apply worker may have to wait for a longer period to manage the clock
skew. To control this maximum wait time, a new GUC,
'max_logical_rep_clock_skew_wait', is provided. This allows the user to
set a cap on how long the apply worker should wait. If the computed wait
time exceeds this value, the apply worker will error out without waiting.
This patch also implements last_update_wins resolver.
Since conflict resolution for two phase commit transactions
using prepare-timestamp can result in data divergence, this patch
also restricts enabling two_phase and detect_conflict together
for a subscription
---
src/backend/commands/subscriptioncmds.c | 45 +++++
src/backend/executor/execReplication.c | 2 +-
.../replication/logical/applyparallelworker.c | 26 ++-
src/backend/replication/logical/conflict.c | 142 ++++++++++++----
src/backend/replication/logical/origin.c | 1 +
src/backend/replication/logical/worker.c | 155 ++++++++++++++++--
.../utils/activity/wait_event_names.txt | 1 +
src/backend/utils/misc/guc_tables.c | 40 +++++
src/backend/utils/misc/postgresql.conf.sample | 9 +-
src/include/catalog/pg_conflict.dat | 6 +-
src/include/replication/conflict.h | 6 +-
src/include/replication/logicalworker.h | 18 ++
src/include/replication/origin.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/include/utils/timestamp.h | 1 +
.../regress/expected/conflict_resolver.out | 16 +-
src/test/regress/expected/subscription.out | 2 +
src/test/subscription/t/029_on_error.pl | 52 ++++--
.../subscription/t/034_conflict_resolver.pl | 107 +++++++++++-
src/tools/pgindent/typedefs.list | 1 +
20 files changed, 556 insertions(+), 77 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 512b4273ae..7e3c7af160 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -14,6 +14,7 @@
#include "postgres.h"
+#include "access/commit_ts.h"
#include "access/htup_details.h"
#include "access/table.h"
#include "access/xact.h"
@@ -459,6 +460,22 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * Time based conflict resolution for two phase transactions can result in
+ * data divergence, so disallow enabling both together.
+ */
+ if (opts->detectconflict &&
+ IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ if (opts->twophase &&
+ IsSet(opts->specified_opts, SUBOPT_TWOPHASE_COMMIT))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ /*- translator: both %s are strings of the form "option = value" */
+ errmsg("%s and %s are mutually exclusive options",
+ "detect_conflict = true", "two_phase = true")));
+ }
}
/*
@@ -671,6 +688,13 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
elog(WARNING, "subscriptions created by regression test cases should have names starting with \"regress_\"");
#endif
+ /* Warn if detect_conflict is enabled and track_commit_timestamp is off */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
+
rel = table_open(SubscriptionRelationId, RowExclusiveLock);
/* Check if name is used */
@@ -1279,6 +1303,27 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
values[Anum_pg_subscription_subdetectconflict - 1] =
BoolGetDatum(opts.detectconflict);
replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+
+ /*
+ * Time based conflict resolution for two phase
+ * transactions can result in data divergence, so disallow
+ * enabling it when two_phase is enabled.
+ */
+ if (sub->twophasestate == LOGICALREP_TWOPHASE_STATE_ENABLED)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for a subscription that has two_phase enabled",
+ "detect_conflict")));
+
+ /*
+ * Warn if detect_conflict is enabled and
+ * track_commit_timestamp is off.
+ */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
}
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index b18723e56b..8d93a965f2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -630,7 +630,7 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
bool apply_remote = false;
GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
- resolver = GetConflictResolver(rel, CT_INSERT_EXISTS,
+ resolver = GetConflictResolver(*conflictslot, rel, CT_INSERT_EXISTS,
&apply_remote, NULL);
ReportApplyConflict(CT_INSERT_EXISTS, resolver, rel,
diff --git a/src/backend/replication/logical/applyparallelworker.c b/src/backend/replication/logical/applyparallelworker.c
index e7f7d4c5e4..10c7ca99df 100644
--- a/src/backend/replication/logical/applyparallelworker.c
+++ b/src/backend/replication/logical/applyparallelworker.c
@@ -312,6 +312,20 @@ pa_can_start(void)
if (!AllTablesyncsReady())
return false;
+ /*
+ * Don't start a new parallel worker if user has either configured max
+ * clock skew or if conflict detection and resolution is ON. In both cases
+ * we need commit timestamp in the beginning.
+ *
+ * XXX: For conflict reolution case, see if we can reduce the scope of
+ * this restriction to only such cases where time-based resolvers are
+ * actually being used.
+ */
+ if ((max_logical_rep_clock_skew > LR_CLOCK_SKEW_DEFAULT) ||
+ MySubscription->detectconflict)
+ return false;
+
+
return true;
}
@@ -696,9 +710,19 @@ pa_process_spooled_messages_if_required(void)
}
else if (fileset_state == FS_READY)
{
+ /*
+ * Currently we do not support starting parallel apply worker when
+ * either clock skew is configured or conflict resolution is
+ * configured, thus it is okay to pass 0 as origin-timestamp here.
+ *
+ * XXX: If in future, we support starting pa worker even with conflict
+ * detection enabled, then here we need to pass remote's
+ * commit/prepare/abort timestamp; we can get that info from leader
+ * worker in shared memory.
+ */
apply_spooled_messages(&MyParallelShared->fileset,
MyParallelShared->xid,
- InvalidXLogRecPtr);
+ InvalidXLogRecPtr, 0);
pa_set_fileset_state(MyParallelShared, FS_EMPTY);
}
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 574afe71f3..a8fa470cf7 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -44,6 +44,7 @@ const char *const ConflictTypeNames[] = {
const char *const ConflictResolverNames[] = {
[CR_REMOTE_APPLY] = "remote_apply",
[CR_KEEP_LOCAL] = "keep_local",
+ [CR_LAST_UPDATE_WINS] = "last_update_wins",
[CR_APPLY_OR_SKIP] = "apply_or_skip",
[CR_APPLY_OR_ERROR] = "apply_or_error",
[CR_SKIP] = "skip",
@@ -69,11 +70,11 @@ const char *const ConflictResolverNames[] = {
* friendly name for a resolver and thus has been added here.
*/
const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
- [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
- [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
[CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
[CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
- [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR}
};
/*
@@ -81,11 +82,11 @@ const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
* If this changes, change it in pg_conflict.dat as well.
*/
const int ConflictTypeDefaultResolvers[] = {
- [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
- [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_INSERT_EXISTS] = CR_LAST_UPDATE_WINS,
+ [CT_UPDATE_DIFFER] = CR_LAST_UPDATE_WINS,
[CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
[CT_DELETE_MISSING] = CR_SKIP,
- [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+ [CT_DELETE_DIFFER] = CR_LAST_UPDATE_WINS
};
@@ -201,6 +202,12 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
TupleTableSlot *conflictslot, bool apply_remote)
{
char *applymsg;
+ int errdet = 0;
+ char *local_ts;
+ char *remote_ts;
+
+ local_ts = pstrdup(timestamptz_to_str(localts));
+ remote_ts = pstrdup(timestamptz_to_str(replorigin_session_origin_timestamp));
if (apply_remote)
applymsg = "applying the remote changes.";
@@ -222,43 +229,63 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
if (resolver == CR_ERROR)
{
if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
+ errdet = errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx),
+ localorigin, localxmin, local_ts);
else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ errdet = errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ errdet = errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
}
+ else if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Key already exists, %s. The local tuple : origin=%u, timestamp=%s; The remote tuple : origin=%u, timestamp=%s.",
+ applymsg, localorigin, local_ts,
+ replorigin_session_origin, remote_ts);
else
- return errdetail("Key already exists, %s", applymsg);
+ errdet = errdetail("Key already exists, %s", applymsg);
}
+ break;
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
- localorigin, localxmin,
- timestamptz_to_str(localts), applymsg);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, local_ts, applymsg,
+ replorigin_session_origin, remote_ts);
+ else
+ errdet = errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin, local_ts, applymsg);
+ break;
case CT_UPDATE_MISSING:
if (resolver == CR_APPLY_OR_SKIP && !apply_remote)
- return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
+ errdet = errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
else if (resolver == CR_APPLY_OR_ERROR && !apply_remote)
- return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
+ errdet = errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
else if (apply_remote)
- return errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
- applymsg);
+ errdet = errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
+ applymsg);
else
- return errdetail("Did not find the row to be updated, %s",
- applymsg);
+ errdet = errdetail("Did not find the row to be updated, %s",
+ applymsg);
+ break;
case CT_DELETE_MISSING:
- return errdetail("Did not find the row to be deleted.");
+ errdet = errdetail("Did not find the row to be deleted.");
+ break;
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
- localorigin, localxmin,
- timestamptz_to_str(localts), applymsg);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, local_ts, applymsg,
+ replorigin_session_origin, remote_ts);
+ else
+ errdet = errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin, local_ts, applymsg);
+ break;
}
- return 0; /* silence compiler warning */
+ pfree(local_ts);
+ pfree(remote_ts);
+
+ return errdet;
}
/*
@@ -366,6 +393,15 @@ validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver
errmsg("%s is not a valid conflict resolver for conflict type %s",
conflict_resolver,
conflict_type));
+
+ if ((resolver == CR_LAST_UPDATE_WINS) && !track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ conflict_resolver, "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+
return type;
}
@@ -407,6 +443,42 @@ get_conflict_resolver_internal(ConflictType type)
return resolver;
}
+/*
+ * Compare the timestamps of given local tuple and the remote tuple to
+ * resolve the conflict.
+ *
+ * Returns true if remote tuple has the latest timestamp, false otherwise.
+ */
+static bool
+resolve_by_timestamp(TupleTableSlot *localslot)
+{
+ TransactionId local_xmin;
+ TimestampTz local_ts;
+ RepOriginId local_origin;
+ int ts_cmp;
+ uint64 local_system_identifier;
+
+ /* Get origin and timestamp info of the local tuple */
+ GetTupleCommitTs(localslot, &local_xmin, &local_origin, &local_ts);
+
+ /* Compare the timestamps of remote & local tuple to decide the winner */
+ ts_cmp = timestamptz_cmp_internal(replorigin_session_origin_timestamp,
+ local_ts);
+
+ if (ts_cmp == 0)
+ {
+ elog(LOG, "Timestamps of remote and local tuple are equal, comparing remote and local system identifiers");
+
+ /* Get current system's identifier */
+ local_system_identifier = GetSystemIdentifier();
+
+ return local_system_identifier <= replorigin_session_origin_sysid;
+ }
+ else
+ return (ts_cmp > 0);
+
+}
+
/*
* Check if a full tuple can be created from the new tuple.
* Return true if yes, false otherwise.
@@ -512,7 +584,8 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
* false otherwise.
*/
ConflictResolver
-GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
+GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
+ ConflictType type, bool *apply_remote,
LogicalRepTupleData *newtup)
{
ConflictResolver resolver;
@@ -521,6 +594,17 @@ GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
switch (resolver)
{
+ case CR_LAST_UPDATE_WINS:
+ if (!track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ ConflictResolverNames[resolver], "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+ else
+ *apply_remote = resolve_by_timestamp(localslot);
+ break;
case CR_REMOTE_APPLY:
*apply_remote = true;
break;
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 419e4814f0..bd8e6f0024 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -155,6 +155,7 @@ typedef struct ReplicationStateCtl
RepOriginId replorigin_session_origin = InvalidRepOriginId; /* assumed identity */
XLogRecPtr replorigin_session_origin_lsn = InvalidXLogRecPtr;
TimestampTz replorigin_session_origin_timestamp = 0;
+uint64 replorigin_session_origin_sysid = 0;
/*
* Base address into a shared memory array of replication states of size
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 9859f0f2bd..5f0af32e29 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -318,6 +318,20 @@ static uint32 parallel_stream_nchanges = 0;
/* Are we initializing an apply worker? */
bool InitializingApplyWorker = false;
+/*
+ * GUC support
+ */
+const struct config_enum_entry logical_rep_clock_skew_action_options[] = {
+ {"error", LR_CLOCK_SKEW_ACTION_ERROR, false},
+ {"wait", LR_CLOCK_SKEW_ACTION_WAIT, false},
+ {NULL, 0, false}
+};
+
+/* GUCs */
+int max_logical_rep_clock_skew = LR_CLOCK_SKEW_DEFAULT;
+int max_logical_rep_clock_skew_action = LR_CLOCK_SKEW_ACTION_ERROR;
+int max_logical_rep_clock_skew_wait = 300; /* 5 mins */
+
/*
* We enable skipping all data modification changes (INSERT, UPDATE, etc.) for
* the subscription if the remote transaction's finish LSN matches the subskiplsn.
@@ -987,6 +1001,95 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
ExecStoreVirtualTuple(slot);
}
+/*
+ * Manage clock skew between nodes.
+ *
+ * It checks if the remote timestamp is ahead of the local clock
+ * and if the difference exceeds max_logical_rep_clock_skew, it performs
+ * the action specified by the max_logical_rep_clock_skew_action.
+ */
+static void
+manage_clock_skew(TimestampTz origin_timestamp)
+{
+ TimestampTz current;
+ TimestampTz delayUntil;
+ long msecs;
+ int rc;
+
+ /* nothing to do if no max clock skew configured */
+ if (max_logical_rep_clock_skew == LR_CLOCK_SKEW_DEFAULT)
+ return;
+
+ current = GetCurrentTimestamp();
+
+ /*
+ * If the timestamp of the currently replayed transaction is in the future
+ * compared to the current time on the subscriber and the difference is
+ * larger than max_logical_rep_clock_skew, then perform the action
+ * specified by the max_logical_rep_clock_skew_action setting.
+ */
+ if (origin_timestamp > current &&
+ TimestampDifferenceExceeds(current, origin_timestamp,
+ max_logical_rep_clock_skew * 1000))
+ {
+ if (max_logical_rep_clock_skew_action == LR_CLOCK_SKEW_ACTION_ERROR)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg_internal("clock skew exceeds max_logical_rep_clock_skew (%d seconds)",
+ max_logical_rep_clock_skew)));
+
+ /* Perform the wait */
+ while (true)
+ {
+ delayUntil =
+ TimestampTzMinusSeconds(origin_timestamp,
+ max_logical_rep_clock_skew);
+
+ /* Exit without waiting if it's already past 'delayUntil' time */
+ msecs = TimestampDifferenceMilliseconds(GetCurrentTimestamp(),
+ delayUntil);
+ if (msecs <= 0)
+ break;
+
+ /* The wait time should not exceed max_logical_rep_clock_skew_wait */
+ if (msecs > (max_logical_rep_clock_skew_wait * 1000L))
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg_internal("clock skew wait time exceeds max_logical_rep_clock_skew_wait (%d seconds)",
+ max_logical_rep_clock_skew_wait)));
+
+ elog(DEBUG2, "delaying apply for %ld milliseconds to manage clock skew",
+ msecs);
+
+ /* Sleep until we are signaled or msecs have elapsed */
+ rc = WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ msecs,
+ WAIT_EVENT_LOGICAL_CLOCK_SKEW);
+
+ /* Exit the loop if msecs have elapsed */
+ if (rc & WL_TIMEOUT)
+ break;
+
+ if (rc & WL_LATCH_SET)
+ {
+ ResetLatch(MyLatch);
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ /*
+ * This might change max_logical_rep_clock_skew and
+ * max_logical_rep_clock_skew_wait.
+ */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+ }
+ }
+}
+
/*
* Handle BEGIN message.
*/
@@ -1008,6 +1111,15 @@ apply_handle_begin(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.committime);
+
+ /*
+ * Capture the commit timestamp of the remote transaction
+ * for time based conflict resolution purpose.
+ */
+ replorigin_session_origin_timestamp = begin_data.committime;
}
/*
@@ -1065,6 +1177,9 @@ apply_handle_begin_prepare(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.prepare_time);
}
/*
@@ -1305,7 +1420,8 @@ apply_handle_stream_prepare(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset,
- prepare_data.xid, prepare_data.prepare_lsn);
+ prepare_data.xid, prepare_data.prepare_lsn,
+ prepare_data.prepare_time);
/* Mark the transaction as prepared. */
apply_handle_prepare_internal(&prepare_data);
@@ -2002,7 +2118,8 @@ ensure_last_message(FileSet *stream_fileset, TransactionId xid, int fileno,
*/
void
apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn)
+ XLogRecPtr lsn,
+ TimestampTz origin_timestamp)
{
int nchanges;
char path[MAXPGPATH];
@@ -2055,6 +2172,16 @@ apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
end_replication_step();
+ /*
+ * If origin_timestamp is provided by caller, then check clock skew with
+ * respect to the passed time and take configured action.
+ */
+ if (origin_timestamp)
+ manage_clock_skew(origin_timestamp);
+
+ /* Capture the timestamp (prepare or commit) of the remote transaction */
+ replorigin_session_origin_timestamp = origin_timestamp;
+
/*
* Read the entries one by one and pass them through the same logic as in
* apply_dispatch.
@@ -2160,7 +2287,8 @@ apply_handle_stream_commit(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset, xid,
- commit_data.commit_lsn);
+ commit_data.commit_lsn,
+ commit_data.committime);
apply_handle_commit_internal(&commit_data);
@@ -2717,7 +2845,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(localrel, CT_UPDATE_DIFFER,
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_DIFFER,
&apply_remote, NULL);
ReportApplyConflict(CT_UPDATE_DIFFER, resolver, localrel,
@@ -2753,7 +2881,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(localrel, CT_UPDATE_MISSING,
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_MISSING,
&apply_remote, newtup);
ReportApplyConflict(CT_UPDATE_MISSING, resolver, localrel,
@@ -2906,7 +3034,7 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(localrel, CT_DELETE_DIFFER,
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_DIFFER,
&apply_remote, NULL);
ReportApplyConflict(CT_DELETE_DIFFER, resolver, localrel,
@@ -2932,7 +3060,7 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(localrel, CT_DELETE_MISSING,
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_MISSING,
&apply_remote, NULL);
/* Resolver is set to skip, thus report the conflict and skip */
@@ -3130,7 +3258,7 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(partrel, CT_UPDATE_MISSING,
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_MISSING,
&apply_remote, newtup);
ReportApplyConflict(CT_UPDATE_MISSING, resolver, partrel,
@@ -3162,7 +3290,7 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(partrel, CT_UPDATE_DIFFER,
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_DIFFER,
&apply_remote, NULL);
ReportApplyConflict(CT_UPDATE_DIFFER, resolver, partrel,
@@ -4682,6 +4810,7 @@ run_apply_worker()
TimeLineID startpointTLI;
char *err;
bool must_use_password;
+ char *replorigin_sysid;
slotname = MySubscription->slotname;
@@ -4722,10 +4851,12 @@ run_apply_worker()
MySubscription->name, err)));
/*
- * We don't really use the output identify_system for anything but it does
- * some initializations on the upstream so let's still call it.
+ * Call identify_system to do some initializations on the upstream and
+ * store the output as system identifier of the replication origin node.
*/
- (void) walrcv_identify_system(LogRepWorkerWalRcvConn, &startpointTLI);
+ replorigin_sysid = walrcv_identify_system(LogRepWorkerWalRcvConn,
+ &startpointTLI);
+ replorigin_session_origin_sysid = strtoul(replorigin_sysid, NULL, 10);
set_apply_error_context_origin(originname);
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index db37beeaae..a51f82169e 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -59,6 +59,7 @@ CHECKPOINTER_MAIN "Waiting in main loop of checkpointer process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_CLOCK_SKEW "Waiting in apply-begin of logical replication apply process to bring clock skew in permissible range."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot sync worker."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 630ed0f162..8c843c522e 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -67,6 +67,7 @@
#include "postmaster/walsummarizer.h"
#include "postmaster/walwriter.h"
#include "replication/logicallauncher.h"
+#include "replication/logicalworker.h"
#include "replication/slot.h"
#include "replication/slotsync.h"
#include "replication/syncrep.h"
@@ -492,6 +493,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry wal_sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry logical_rep_clock_skew_action_options[];
/*
* GUC option variables that are exported from this module
@@ -3649,6 +3651,33 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets maximum clock skew tolerance between logical "
+ "replication nodes beyond which action configured "
+ "in max_logical_rep_clock_skew_action is triggered."),
+ gettext_noop("-1 turns this check off."),
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew,
+ LR_CLOCK_SKEW_DEFAULT, LR_CLOCK_SKEW_DEFAULT, INT_MAX,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"max_logical_rep_clock_skew_wait", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets max limit on how long apply worker shall wait to "
+ "bring clock skew within permissible range of max_logical_rep_clock_skew. "
+ "If the computed wait time is more than this value, "
+ "apply worker will error out without waiting."),
+ gettext_noop("0 turns this limit off."),
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew_wait,
+ 300, 0, 3600,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
@@ -4915,6 +4944,17 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew_action", PGC_POSTMASTER, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the action to perform if a clock skew higher "
+ "than max_logical_rep_clock_skew is detected."),
+ NULL
+ },
+ &max_logical_rep_clock_skew_action,
+ LR_CLOCK_SKEW_ACTION_ERROR, logical_rep_clock_skew_action_options,
+ NULL, NULL, NULL
+ },
+
{
{"track_functions", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects function-level statistics on database activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 9ec9f97e92..f7a664a538 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -383,7 +383,14 @@
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
-
+#max_logical_rep_clock_skew = -1 # maximum clock skew tolerance between logical
+ # replication nodes beyond which action configured in
+ # 'max_logical_rep_clock_skew_action' is triggered.
+#max_logical_rep_clock_skew_action = error # error or wait
+ # (change requires restart)
+#max_logical_rep_clock_skew_wait = 300 # max limit on how long apply worker
+ # shall wait to bring clock skew within permissible
+ # range of max_logical_rep_clock_skew.
#------------------------------------------------------------------------------
# QUERY TUNING
diff --git a/src/include/catalog/pg_conflict.dat b/src/include/catalog/pg_conflict.dat
index 8d6183e3f2..1905151840 100644
--- a/src/include/catalog/pg_conflict.dat
+++ b/src/include/catalog/pg_conflict.dat
@@ -12,10 +12,10 @@
[
-{ conftype => 'insert_exists', confres => 'remote_apply' },
-{ conftype => 'update_differ', confres => 'remote_apply' },
+{ conftype => 'insert_exists', confres => 'last_update_wins' },
+{ conftype => 'update_differ', confres => 'last_update_wins' },
{ conftype => 'update_missing', confres => 'apply_or_skip' },
{ conftype => 'delete_missing', confres => 'skip' },
-{ conftype => 'delete_differ', confres => 'remote_apply' }
+{ conftype => 'delete_differ', confres => 'last_update_wins' }
]
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index b06c0f75a0..4e2853b5be 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -57,6 +57,9 @@ typedef enum ConflictResolver
/* Keep the local change */
CR_KEEP_LOCAL,
+ /* Apply the change with latest timestamp */
+ CR_LAST_UPDATE_WINS,
+
/* Apply the remote change; skip if it can not be applied */
CR_APPLY_OR_SKIP,
@@ -83,7 +86,8 @@ extern void ReportApplyConflict(ConflictType type, ConflictResolver resolver,
TupleTableSlot *conflictslot, bool apply_remote);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
extern void ExecConflictResolverStmt(ConflictResolverStmt *stmt);
-extern ConflictResolver GetConflictResolver(Relation localrel,
+extern ConflictResolver GetConflictResolver(TupleTableSlot *localslot,
+ Relation localrel,
ConflictType type,
bool *apply_remote,
LogicalRepTupleData *newtup);
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index a18d79d1b2..7cb03062ac 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,7 +14,25 @@
#include <signal.h>
+/*
+ * The default for max_logical_rep_clock_skew is -1, which means ignore clock
+ * skew (the check is turned off).
+ */
+#define LR_CLOCK_SKEW_DEFAULT -1
+
+/*
+ * Worker Clock Skew Action.
+ */
+typedef enum
+{
+ LR_CLOCK_SKEW_ACTION_ERROR,
+ LR_CLOCK_SKEW_ACTION_WAIT,
+} LogicalRepClockSkewAction;
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
+extern PGDLLIMPORT int max_logical_rep_clock_skew;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_action;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_wait;
extern void ApplyWorkerMain(Datum main_arg);
extern void ParallelApplyWorkerMain(Datum main_arg);
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 7189ba9e76..dcbbbdf6ea 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -36,6 +36,7 @@ typedef struct xl_replorigin_drop
extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+extern PGDLLIMPORT uint64 replorigin_session_origin_sysid;
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 515aefd519..dc9e067fac 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -267,7 +267,7 @@ extern void stream_stop_internal(TransactionId xid);
/* Common streaming function to apply all the spooled messages */
extern void apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn);
+ XLogRecPtr lsn, TimestampTz origin_timestamp);
extern void apply_dispatch(StringInfo s);
diff --git a/src/include/utils/timestamp.h b/src/include/utils/timestamp.h
index a6ce03ed46..53b828d89d 100644
--- a/src/include/utils/timestamp.h
+++ b/src/include/utils/timestamp.h
@@ -84,6 +84,7 @@ IntervalPGetDatum(const Interval *X)
/* Macros for doing timestamp arithmetic without assuming timestamp's units */
#define TimestampTzPlusMilliseconds(tz,ms) ((tz) + ((ms) * (int64) 1000))
#define TimestampTzPlusSeconds(tz,s) ((tz) + ((s) * (int64) 1000000))
+#define TimestampTzMinusSeconds(tz,s) ((tz) - ((s) * (int64) 1000000))
/* Set at postmaster start */
diff --git a/src/test/regress/expected/conflict_resolver.out b/src/test/regress/expected/conflict_resolver.out
index 53ecb430fa..c21486dbb4 100644
--- a/src/test/regress/expected/conflict_resolver.out
+++ b/src/test/regress/expected/conflict_resolver.out
@@ -1,11 +1,11 @@
--check default global resolvers in system catalog
select * from pg_conflict order by conftype;
- conftype | confres
-----------------+---------------
- delete_differ | remote_apply
+ conftype | confres
+----------------+------------------
+ delete_differ | last_update_wins
delete_missing | skip
- insert_exists | remote_apply
- update_differ | remote_apply
+ insert_exists | last_update_wins
+ update_differ | last_update_wins
update_missing | apply_or_skip
(5 rows)
@@ -43,11 +43,11 @@ RESET CONFLICT RESOLVER for 'delete_missing';
RESET CONFLICT RESOLVER for 'insert_exists';
--check resolvers are reset to default for delete_missing and insert_exists
select * from pg_conflict order by conftype;
- conftype | confres
-----------------+----------------
+ conftype | confres
+----------------+------------------
delete_differ | keep_local
delete_missing | skip
- insert_exists | remote_apply
+ insert_exists | last_update_wins
update_differ | keep_local
update_missing | apply_or_error
(5 rows)
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 3d8b8c5d32..844a5dfff8 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -433,6 +433,8 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
ERROR: detect_conflict requires a Boolean value
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index c746191601..6c1eb7e5a9 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -18,7 +18,7 @@ my $offset = 0;
# on the publisher.
sub test_skip_lsn
{
- my ($node_publisher, $node_subscriber, $nonconflict_data, $expected, $msg)
+ my ($node_publisher, $node_subscriber, $nonconflict_data, $expected, $msg, $conflict_detection)
= @_;
# Wait until a conflict occurs on the subscriber.
@@ -26,13 +26,25 @@ sub test_skip_lsn
"SELECT subenabled = FALSE FROM pg_subscription WHERE subname = 'sub'"
);
+ my $lsn;
+ my $contents = slurp_file($node_subscriber->logfile, $offset);
+
# Get the finish LSN of the error transaction, mapping the expected
# ERROR with its CONTEXT when retrieving this information.
- my $contents = slurp_file($node_subscriber->logfile, $offset);
- $contents =~
- qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
- or die "could not get error-LSN";
- my $lsn = $1;
+ if ($conflict_detection)
+ {
+ $contents =~
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ or die "could not get error-LSN";
+ $lsn = $1;
+ }
+ else
+ {
+ $contents =~
+ qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ or die "could not get error-LSN";
+ $lsn = $1;
+ }
# Set skip lsn.
$node_subscriber->safe_psql('postgres',
@@ -110,7 +122,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, detect_conflict = on)"
);
# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
@@ -147,7 +159,22 @@ INSERT INTO tbl VALUES (1, NULL);
COMMIT;
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(2, NULL)", "2", "test skipping transaction");
+ "(2, NULL)", "2", "test skipping transaction", 1);
+
+# Cleanup before we start PREPARE AND COMMIT PREPARED tests
+$node_subscriber->safe_psql('postgres', "TRUNCATE tbl");
+$node_publisher->safe_psql('postgres', "TRUNCATE tbl");
+
+# Drop subscription and recreate with two_phase enabled
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+);
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'sub');
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tbl VALUES (1, NULL)");
# Test for PREPARE and COMMIT PREPARED. Insert the same data to tbl and
# PREPARE the transaction, raising an error. Then skip the transaction.
@@ -160,7 +187,7 @@ PREPARE TRANSACTION 'gtx';
COMMIT PREPARED 'gtx';
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(3, NULL)", "3", "test skipping prepare and commit prepared ");
+ "(2, NULL)", "2", "test skipping prepare and commit prepared ", 0);
# Test for STREAM COMMIT. Insert enough rows to tbl to exceed the 64kB
# limit, also raising an error on the subscriber during applying spooled
@@ -173,17 +200,14 @@ INSERT INTO tbl SELECT i, sha256(i::text::bytea) FROM generate_series(1, 10000)
COMMIT;
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(4, sha256(4::text::bytea))",
- "4", "test skipping stream-commit");
+ "(3, sha256(4::text::bytea))",
+ "3", "test skipping stream-commit", 0);
$result = $node_subscriber->safe_psql('postgres',
"SELECT COUNT(*) FROM pg_prepared_xacts");
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
-# Reset conflict resolver for 'insert_exist' conflict type to default.
-$node_subscriber->safe_psql('postgres',"reset conflict resolver for 'insert_exists'");
-
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
index 0fb5ef9a86..3bde2750da 100755
--- a/src/test/subscription/t/034_conflict_resolver.pl
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -51,10 +51,10 @@ $node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
my $result = $node_subscriber->safe_psql('postgres',
"SELECT conftype, confres FROM pg_conflict ORDER BY conftype"
);
-is( $result, qq(delete_differ|remote_apply
+is( $result, qq(delete_differ|last_update_wins
delete_missing|skip
-insert_exists|remote_apply
-update_differ|remote_apply
+insert_exists|last_update_wins
+update_differ|last_update_wins
update_missing|apply_or_skip),
"confirm that the default conflict resolvers are in place"
);
@@ -63,6 +63,11 @@ update_missing|apply_or_skip),
# Test 'remote_apply' for 'insert_exists'
############################################
+# Change CONFLICT RESOLVER of insert_exists to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'remote_apply' for 'insert_exists';"
+);
+
# Create local data on the subscriber
$node_subscriber->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
@@ -146,6 +151,34 @@ $node_subscriber->wait_for_log(
$node_subscriber->safe_psql('postgres',
"TRUNCATE conf_tab;");
+############################################
+# Test 'last_update_wins' for 'insert_exists'
+############################################
+
+# Change CONFLICT RESOLVER of insert_exists to last_update_wins
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'last_update_wins' for 'insert_exists';"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (4,'fromsub')");
+
+# Create conflicting data on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (4,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=4);"
+);
+
+is($result, 'frompub', "remote data wins");
+
###################################
# Test 'skip' for 'delete_missing'
###################################
@@ -203,14 +236,48 @@ $node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
#########################################
-# Test 'remote_apply' for 'delete_differ'
+# Test 'last_update_wins' for 'delete_differ'
#########################################
+# Change CONFLICT RESOLVER of delete_differ to last_update_wins
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'last_update_wins' for 'delete_differ';"
+);
+
# Insert data in the publisher
$node_publisher->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
- INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (4,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=4);");
+
+# Create a conflicting delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "DELETE FROM conf_tab WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote delete the local updated row
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=4);"
+);
+
+is($result, '', "delete from remote wins");
+
+#########################################
+# Test 'remote_apply' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'remote_apply' for 'delete_differ';"
+);
# Modify data on the subscriber
$node_subscriber->safe_psql('postgres',
@@ -303,9 +370,14 @@ $node_subscriber->safe_psql('postgres',
$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
#########################################
-# Test 'remote_apply' for 'update_differ'
+# Test 'last_update_wins' for 'update_differ'
#########################################
+# Change CONFLICT RESOLVER of update_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "SET CONFLICT RESOLVER 'remote_apply' for 'update_differ';"
+);
+
# Insert data in the publisher
$node_publisher->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
@@ -331,6 +403,29 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, 'frompubnew', "update from remote is kept");
+#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub2' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew2' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/, $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);"
+);
+
+is($result, 'frompubnew2', "update from remote is kept");
+
#########################################
# Test 'keep_local' for 'update_differ'
#########################################
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1dfcc57fb1..ae3ee18764 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1567,6 +1567,7 @@ LogicalOutputPluginWriterPrepareWrite
LogicalOutputPluginWriterUpdateProgress
LogicalOutputPluginWriterWrite
LogicalRepBeginData
+LogicalRepClockSkewAction
LogicalRepCommitData
LogicalRepCommitPreparedTxnData
LogicalRepCtxStruct
--
2.34.1
[application/octet-stream] v6-0001-Detect-and-log-conflicts-in-logical-replication.patch (95.5K, ../../CAJpy0uDAMtrqXWFtxrXSwTNO8awFAoENtbo47Czj-67k3OVMyQ@mail.gmail.com/4-v6-0001-Detect-and-log-conflicts-in-logical-replication.patch)
download | inline diff:
From 0cbd31d6ac288ecc98cc6572fd80f78a9fd210e4 Mon Sep 17 00:00:00 2001
From: Hou Zhijie <[email protected]>
Date: Fri, 31 May 2024 14:20:45 +0530
Subject: [PATCH v6 1/5] Detect and log conflicts in logical replication
This patch adds a new parameter detect_conflict for CREATE and ALTER
subscription commands. This new parameter will decide if subscription will
go for confict detection. By default, conflict detection will be off for a
subscription.
When conflict detection is enabled, additional logging is triggered in the
following conflict scenarios:
insert_exists: Inserting a row that violates a NOT DEFERRABLE unique constraint.
update_differ: Updating a row that was previously modified by another origin.
update_missing: The tuple to be updated is missing.
delete_missing: The tuple to be deleted is missing.
delete_differ: Deleting a row that was previously modified by another origin.
For insert_exists conflict, the log can include origin and commit
timestamp details of the conflicting key with track_commit_timestamp
enabled.
update_differ and delete_differ conflicts can only be detected when
track_commit_timestamp is enabled.
---
doc/src/sgml/catalogs.sgml | 9 +
doc/src/sgml/logical-replication.sgml | 23 ++-
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 66 +++++++
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 3 +-
src/backend/commands/subscriptioncmds.c | 31 +++-
src/backend/executor/execIndexing.c | 14 +-
src/backend/executor/execReplication.c | 117 +++++++++++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/conflict.c | 191 ++++++++++++++++++++
src/backend/replication/logical/meson.build | 1 +
src/backend/replication/logical/worker.c | 79 ++++++--
src/bin/pg_dump/pg_dump.c | 17 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 6 +-
src/bin/psql/tab-complete.c | 14 +-
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/conflict.h | 47 +++++
src/test/regress/expected/subscription.out | 176 ++++++++++--------
src/test/regress/sql/subscription.sql | 15 ++
src/test/subscription/t/001_rep_changes.pl | 15 +-
src/test/subscription/t/013_partition.pl | 68 ++++---
src/test/subscription/t/029_on_error.pl | 5 +-
src/test/subscription/t/030_origin.pl | 43 +++++
src/tools/pgindent/typedefs.list | 1 +
26 files changed, 801 insertions(+), 152 deletions(-)
create mode 100644 src/backend/replication/logical/conflict.c
create mode 100644 src/include/replication/conflict.h
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b654fae1b2..b042a5a94a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8035,6 +8035,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subdetectconflict</structfield> <type>bool</type>
+ </para>
+ <para>
+ If true, the subscription is enabled for conflict detection.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subconninfo</structfield> <type>text</type>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index ccdd24312b..f078d6364b 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1565,7 +1565,9 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
stop. This is referred to as a <firstterm>conflict</firstterm>. When
replicating <command>UPDATE</command> or <command>DELETE</command>
operations, missing data will not produce a conflict and such operations
- will simply be skipped.
+ will simply be skipped, but note that this scenario can be reported in the server log
+ if <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ is enabled.
</para>
<para>
@@ -1634,6 +1636,25 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
linkend="sql-altersubscription"><command>ALTER SUBSCRIPTION ...
SKIP</command></link>.
</para>
+
+ <para>
+ Enabling both <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ and <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ on the subscriber can provide additional details regarding conflicting
+ rows, such as their origin and commit timestamp, in case of a unique
+ constraint violation conflict:
+<screen>
+ERROR: conflict insert_exists detected on relation "public.t"
+DETAIL: Key (a)=(1) already exists in unique index "t_pkey", which was modified by origin 0 in transaction 740 at 2024-06-26 10:47:04.727375+08.
+CONTEXT: processing remote data for replication origin "pg_16389" during message type "INSERT" for replication target relation "public.t" in transaction 740, finished at 0/14F7EC0
+</screen>
+ Users can use these information to make decisions on whether to retain
+ the local change or adopt the remote alteration. For instance, the
+ origin in above log indicates that the existing row was modified by a
+ local change, users can manually perform a remote-change-win resolution
+ by deleting the local row. Refer to <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ for other conflicts that will be logged when enabling <literal>detect_conflict</literal>.
+ </para>
</sect1>
<sect1 id="logical-replication-restrictions">
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 476f195622..5f6b83e415 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -228,8 +228,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<link linkend="sql-createsubscription-params-with-disable-on-error"><literal>disable_on_error</literal></link>,
<link linkend="sql-createsubscription-params-with-password-required"><literal>password_required</literal></link>,
<link linkend="sql-createsubscription-params-with-run-as-owner"><literal>run_as_owner</literal></link>,
- <link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>, and
- <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>.
+ <link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>,
+ <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>, and
+ <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>.
Only a superuser can set <literal>password_required = false</literal>.
</para>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 740b7d9421..caa523b9bd 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -428,6 +428,72 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createsubscription-params-with-detect-conflict">
+ <term><literal>detect_conflict</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the subscription is enabled for conflict detection.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ When conflict detection is enabled, additional logging is triggered
+ in the following scenarios:
+ <variablelist>
+ <varlistentry>
+ <term><literal>insert_exists</literal></term>
+ <listitem>
+ <para>
+ Inserting a row that violates a <literal>NOT DEFERRABLE</literal>
+ unique constraint. Note that to obtain the origin and commit
+ timestamp details of the conflicting key in the log, ensure that
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_differ</literal></term>
+ <listitem>
+ <para>
+ Updating a row that was previously modified by another origin. Note that this
+ conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be updated is not found.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be deleted is not found.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_differ</literal></term>
+ <listitem>
+ <para>
+ Deleting a row that was previously modified by another origin. Note that this
+ conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist></para>
</listitem>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..5a423f4fb0 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -72,6 +72,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->passwordrequired = subform->subpasswordrequired;
sub->runasowner = subform->subrunasowner;
sub->failover = subform->subfailover;
+ sub->detectconflict = subform->subdetectconflict;
/* Get conninfo */
datum = SysCacheGetAttrNotNull(SUBSCRIPTIONOID,
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 19cabc9a47..d084bfc48a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1356,7 +1356,8 @@ REVOKE ALL ON pg_subscription FROM public;
GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
subbinary, substream, subtwophasestate, subdisableonerr,
subpasswordrequired, subrunasowner, subfailover,
- subslotname, subsynccommit, subpublications, suborigin)
+ subdetectconflict, subslotname, subsynccommit,
+ subpublications, suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 16d83b3253..512b4273ae 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -70,8 +70,9 @@
#define SUBOPT_PASSWORD_REQUIRED 0x00000800
#define SUBOPT_RUN_AS_OWNER 0x00001000
#define SUBOPT_FAILOVER 0x00002000
-#define SUBOPT_LSN 0x00004000
-#define SUBOPT_ORIGIN 0x00008000
+#define SUBOPT_DETECT_CONFLICT 0x00004000
+#define SUBOPT_LSN 0x00008000
+#define SUBOPT_ORIGIN 0x00010000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -97,6 +98,7 @@ typedef struct SubOpts
bool passwordrequired;
bool runasowner;
bool failover;
+ bool detectconflict;
char *origin;
XLogRecPtr lsn;
} SubOpts;
@@ -159,6 +161,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->runasowner = false;
if (IsSet(supported_opts, SUBOPT_FAILOVER))
opts->failover = false;
+ if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT))
+ opts->detectconflict = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
@@ -316,6 +320,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_FAILOVER;
opts->failover = defGetBoolean(defel);
}
+ else if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT) &&
+ strcmp(defel->defname, "detect_conflict") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_DETECT_CONFLICT;
+ opts->detectconflict = defGetBoolean(defel);
+ }
else if (IsSet(supported_opts, SUBOPT_ORIGIN) &&
strcmp(defel->defname, "origin") == 0)
{
@@ -603,7 +616,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
SUBOPT_DISABLE_ON_ERR | SUBOPT_PASSWORD_REQUIRED |
- SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER | SUBOPT_ORIGIN);
+ SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -710,6 +724,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subpasswordrequired - 1] = BoolGetDatum(opts.passwordrequired);
values[Anum_pg_subscription_subrunasowner - 1] = BoolGetDatum(opts.runasowner);
values[Anum_pg_subscription_subfailover - 1] = BoolGetDatum(opts.failover);
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
values[Anum_pg_subscription_subconninfo - 1] =
CStringGetTextDatum(conninfo);
if (opts.slot_name)
@@ -1148,7 +1164,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
SUBOPT_STREAMING | SUBOPT_DISABLE_ON_ERR |
SUBOPT_PASSWORD_REQUIRED |
SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
- SUBOPT_ORIGIN);
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1258,6 +1274,13 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
replaces[Anum_pg_subscription_subfailover - 1] = true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
+ replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9f05b3654c..ef522778a2 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -207,8 +207,9 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
ii = BuildIndexInfo(indexDesc);
/*
- * If the indexes are to be used for speculative insertion, add extra
- * information required by unique index entries.
+ * If the indexes are to be used for speculative insertion or conflict
+ * detection in logical replication, add extra information required by
+ * unique index entries.
*/
if (speculative && ii->ii_Unique)
BuildSpeculativeIndexInfo(indexDesc, ii);
@@ -521,6 +522,11 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* possible that a conflicting tuple is inserted immediately
* after this returns. But this can be used for a pre-check
* before insertion.
+ *
+ * If the 'slot' holds a tuple with valid tid, this tuple will
+ * be ignored when checking conflict. This can help in scenarios
+ * where we want to re-check for conflicts after inserting a
+ * tuple.
* ----------------------------------------------------------------
*/
bool
@@ -536,11 +542,9 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
ExprContext *econtext;
Datum values[INDEX_MAX_KEYS];
bool isnull[INDEX_MAX_KEYS];
- ItemPointerData invalidItemPtr;
bool checkedIndex = false;
ItemPointerSetInvalid(conflictTid);
- ItemPointerSetInvalid(&invalidItemPtr);
/*
* Get information from the result relation info structure.
@@ -629,7 +633,7 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
satisfiesConstraint =
check_exclusion_or_unique_constraint(heapRelation, indexRelation,
- indexInfo, &invalidItemPtr,
+ indexInfo, &slot->tts_tid,
values, isnull, estate, false,
CEOUC_WAIT, true,
conflictTid);
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index d0a89cd577..f01927a933 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -23,6 +23,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "replication/conflict.h"
#include "replication/logicalrelation.h"
#include "storage/lmgr.h"
#include "utils/builtins.h"
@@ -480,6 +481,83 @@ retry:
return found;
}
+/*
+ * Find the tuple that violates the passed in unique index constraint
+ * (conflictindex).
+ *
+ * Return false if there is no conflict and *conflictslot is set to NULL.
+ * Otherwise return true, and the conflicting tuple is locked and returned
+ * in *conflictslot.
+ */
+static bool
+FindConflictTuple(ResultRelInfo *resultRelInfo, EState *estate,
+ Oid conflictindex, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot)
+{
+ Relation rel = resultRelInfo->ri_RelationDesc;
+ ItemPointerData conflictTid;
+ TM_FailureData tmfd;
+ TM_Result res;
+
+ *conflictslot = NULL;
+
+retry:
+ if (ExecCheckIndexConstraints(resultRelInfo, slot, estate,
+ &conflictTid, list_make1_oid(conflictindex)))
+ {
+ if (*conflictslot)
+ ExecDropSingleTupleTableSlot(*conflictslot);
+
+ *conflictslot = NULL;
+ return false;
+ }
+
+ *conflictslot = table_slot_create(rel, NULL);
+
+ PushActiveSnapshot(GetLatestSnapshot());
+
+ res = table_tuple_lock(rel, &conflictTid, GetLatestSnapshot(),
+ *conflictslot,
+ GetCurrentCommandId(false),
+ LockTupleShare,
+ LockWaitBlock,
+ 0 /* don't follow updates */ ,
+ &tmfd);
+
+ PopActiveSnapshot();
+
+ switch (res)
+ {
+ case TM_Ok:
+ break;
+ case TM_Updated:
+ /* XXX: Improve handling here */
+ if (ItemPointerIndicatesMovedPartitions(&tmfd.ctid))
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("tuple to be locked was already moved to another partition due to concurrent update, retrying")));
+ else
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent update, retrying")));
+ goto retry;
+ case TM_Deleted:
+ /* XXX: Improve handling here */
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent delete, retrying")));
+ goto retry;
+ case TM_Invisible:
+ elog(ERROR, "attempted to lock invisible tuple");
+ break;
+ default:
+ elog(ERROR, "unexpected table_tuple_lock status: %u", res);
+ break;
+ }
+
+ return true;
+}
+
/*
* Insert tuple represented in the slot to the relation, update the indexes,
* and execute any constraints and per-row triggers.
@@ -509,6 +587,8 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
+ List *conflictindexes;
+ bool conflict = false;
/* Compute stored generated columns */
if (rel->rd_att->constr &&
@@ -525,10 +605,43 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
- slot, estate, false, false,
- NULL, NIL, false);
+ slot, estate, false,
+ conflictindexes, &conflict,
+ conflictindexes, false);
+
+ /* Re-check all the unique indexes for potential conflicts */
+ foreach_oid(uniqueidx, (conflict ? conflictindexes : NIL))
+ {
+ TupleTableSlot *conflictslot;
+
+ /*
+ * Reports the conflict if any.
+ *
+ * Here, we attempt to find the conflict tuple. This operation may
+ * seem redundant with the unique violation check of indexam, but
+ * since we perform this only when we are detecting conflict in
+ * logical replication and encountering potential conflicts with
+ * any unique index constraints (which should not be frequent), so
+ * it's ok. Moreover, upon detecting a conflict, we will report an
+ * ERROR and restart the logical replication, so the additional
+ * cost of finding the tuple should be acceptable in this case.
+ */
+ if (list_member_oid(recheckIndexes, uniqueidx) &&
+ FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &conflictslot))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+
+ GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
+ ReportApplyConflict(ERROR, CT_INSERT_EXISTS, rel, uniqueidx,
+ xmin, origin, committs, conflictslot);
+ }
+ }
/* AFTER ROW INSERT Triggers */
ExecARInsertTriggers(estate, resultRelInfo, slot,
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index ba03eeff1c..1e08bbbd4e 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
applyparallelworker.o \
+ conflict.o \
decode.o \
launcher.o \
logical.o \
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
new file mode 100644
index 0000000000..b90e64b05b
--- /dev/null
+++ b/src/backend/replication/logical/conflict.c
@@ -0,0 +1,191 @@
+/*-------------------------------------------------------------------------
+ * conflict.c
+ * Functionality for detecting and logging conflicts.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/conflict.c
+ *
+ * This file contains the code for detecting and logging conflicts on
+ * the subscriber during logical replication.
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/commit_ts.h"
+#include "replication/conflict.h"
+#include "replication/origin.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+
+const char *const ConflictTypeNames[] = {
+ [CT_INSERT_EXISTS] = "insert_exists",
+ [CT_UPDATE_DIFFER] = "update_differ",
+ [CT_UPDATE_MISSING] = "update_missing",
+ [CT_DELETE_MISSING] = "delete_missing",
+ [CT_DELETE_DIFFER] = "delete_differ"
+};
+
+static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
+static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin,
+ RepOriginId localorigin,
+ TimestampTz localts,
+ TupleTableSlot *conflictslot);
+
+/*
+ * Get the xmin and commit timestamp data (origin and timestamp) associated
+ * with the provided local tuple.
+ *
+ * Return true if the commit timestamp data was found, false otherwise.
+ */
+bool
+GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts)
+{
+ Datum xminDatum;
+ bool isnull;
+
+ xminDatum = slot_getsysattr(localslot, MinTransactionIdAttributeNumber,
+ &isnull);
+ *xmin = DatumGetTransactionId(xminDatum);
+ Assert(!isnull);
+
+ /*
+ * The commit timestamp data is not available if track_commit_timestamp is
+ * disabled.
+ */
+ if (!track_commit_timestamp)
+ {
+ *localorigin = InvalidRepOriginId;
+ *localts = 0;
+ return false;
+ }
+
+ return TransactionIdGetCommitTsData(*xmin, localts, localorigin);
+}
+
+/*
+ * Report a conflict when applying remote changes.
+ */
+void
+ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot)
+{
+ ereport(elevel,
+ errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
+ errmsg("conflict %s detected on relation \"%s.%s\"",
+ ConflictTypeNames[type],
+ get_namespace_name(RelationGetNamespace(localrel)),
+ RelationGetRelationName(localrel)),
+ errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
+ localts, conflictslot));
+}
+
+/*
+ * Find all unique indexes to check for a conflict and store them into
+ * ResultRelInfo.
+ */
+void
+InitConflictIndexes(ResultRelInfo *relInfo)
+{
+ List *uniqueIndexes = NIL;
+
+ for (int i = 0; i < relInfo->ri_NumIndices; i++)
+ {
+ Relation indexRelation = relInfo->ri_IndexRelationDescs[i];
+
+ if (indexRelation == NULL)
+ continue;
+
+ /* Detect conflict only for unique indexes */
+ if (!relInfo->ri_IndexRelationInfo[i]->ii_Unique)
+ continue;
+
+ /* Don't support conflict detection for deferrable index */
+ if (!indexRelation->rd_index->indimmediate)
+ continue;
+
+ uniqueIndexes = lappend_oid(uniqueIndexes,
+ RelationGetRelid(indexRelation));
+ }
+
+ relInfo->ri_onConflictArbiterIndexes = uniqueIndexes;
+}
+
+/*
+ * Add an errdetail() line showing conflict detail.
+ */
+static int
+errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot)
+{
+ switch (type)
+ {
+ case CT_INSERT_EXISTS:
+ {
+ /*
+ * Bulid the index value string. If the return value is NULL,
+ * it indicates that the current user lacks permissions to
+ * view all the columns involved.
+ */
+ char *index_value = build_index_value_desc(conflictidx,
+ conflictslot);
+
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
+ case CT_UPDATE_DIFFER:
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ case CT_UPDATE_MISSING:
+ return errdetail("Did not find the row to be updated.");
+ case CT_DELETE_MISSING:
+ return errdetail("Did not find the row to be deleted.");
+ case CT_DELETE_DIFFER:
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ }
+
+ return 0; /* silence compiler warning */
+}
+
+/*
+ * Helper functions to construct a string describing the contents of an index
+ * entry. See BuildIndexValueDescription for details.
+ */
+static char *
+build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
+{
+ char *conflict_row;
+ Relation indexDesc;
+
+ if (!conflictslot)
+ return NULL;
+
+ /* Assume the index has been locked */
+ indexDesc = index_open(indexoid, NoLock);
+
+ slot_getallattrs(conflictslot);
+
+ conflict_row = BuildIndexValueDescription(indexDesc,
+ conflictslot->tts_values,
+ conflictslot->tts_isnull);
+
+ index_close(indexDesc, NoLock);
+
+ return conflict_row;
+}
diff --git a/src/backend/replication/logical/meson.build b/src/backend/replication/logical/meson.build
index 3dec36a6de..3d36249d8a 100644
--- a/src/backend/replication/logical/meson.build
+++ b/src/backend/replication/logical/meson.build
@@ -2,6 +2,7 @@
backend_sources += files(
'applyparallelworker.c',
+ 'conflict.c',
'decode.c',
'launcher.c',
'logical.c',
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index c0bda6269b..7e7de195ed 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -167,6 +167,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
+#include "replication/conflict.h"
#include "replication/logicallauncher.h"
#include "replication/logicalproto.h"
#include "replication/logicalrelation.h"
@@ -2461,7 +2462,10 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
EState *estate = edata->estate;
/* We must open indexes here. */
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, MySubscription->detectconflict);
+
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
@@ -2664,6 +2668,20 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
slot_modify_data(remoteslot, localslot, relmapentry, newtup);
@@ -2681,13 +2699,10 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2810,6 +2825,20 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/* If found delete it. */
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
EvalPlanQualSetSlot(&epqstate, localslot);
/* Do the actual delete. */
@@ -2821,13 +2850,10 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/*
* The tuple to be deleted could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be deleted "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2994,6 +3020,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ResultRelInfo *partrelinfo_new;
Relation partrel_new;
bool found;
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3005,16 +3034,28 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing
* except for emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation's partition \"%s\"",
- RelationGetRelationName(partrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING,
+ partrel, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL);
+
return;
}
+ /*
+ * If conflict detection is enabled, check whether the local
+ * tuple was modified by a different origin. If detected,
+ * report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
+ InvalidOid, localxmin, localorigin,
+ localts, NULL);
+
/*
* Apply the update to the local tuple, putting the result in
* remoteslot_part.
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b8b1888bd3..829f9b3e88 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4760,6 +4760,7 @@ getSubscriptions(Archive *fout)
int i_suboriginremotelsn;
int i_subenabled;
int i_subfailover;
+ int i_subdetectconflict;
int i,
ntups;
@@ -4832,11 +4833,17 @@ getSubscriptions(Archive *fout)
if (fout->remoteVersion >= 170000)
appendPQExpBufferStr(query,
- " s.subfailover\n");
+ " s.subfailover,\n");
else
appendPQExpBuffer(query,
- " false AS subfailover\n");
+ " false AS subfailover,\n");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ " s.subdetectconflict\n");
+ else
+ appendPQExpBuffer(query,
+ " false AS subdetectconflict\n");
appendPQExpBufferStr(query,
"FROM pg_subscription s\n");
@@ -4875,6 +4882,7 @@ getSubscriptions(Archive *fout)
i_suboriginremotelsn = PQfnumber(res, "suboriginremotelsn");
i_subenabled = PQfnumber(res, "subenabled");
i_subfailover = PQfnumber(res, "subfailover");
+ i_subdetectconflict = PQfnumber(res, "subdetectconflict");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4921,6 +4929,8 @@ getSubscriptions(Archive *fout)
pg_strdup(PQgetvalue(res, i, i_subenabled));
subinfo[i].subfailover =
pg_strdup(PQgetvalue(res, i, i_subfailover));
+ subinfo[i].subdetectconflict =
+ pg_strdup(PQgetvalue(res, i, i_subdetectconflict));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -5161,6 +5171,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subfailover, "t") == 0)
appendPQExpBufferStr(query, ", failover = true");
+ if (strcmp(subinfo->subdetectconflict, "t") == 0)
+ appendPQExpBufferStr(query, ", detect_conflict = true");
+
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 4b2e5870a9..bbd7cbeff6 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -671,6 +671,7 @@ typedef struct _SubscriptionInfo
char *suborigin;
char *suboriginremotelsn;
char *subfailover;
+ char *subdetectconflict;
} SubscriptionInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 7c9a1f234c..fef1ad0d70 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6539,7 +6539,7 @@ describeSubscriptions(const char *pattern, bool verbose)
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
false, false, false, false, false, false, false, false, false, false,
- false};
+ false, false};
if (pset.sversion < 100000)
{
@@ -6607,6 +6607,10 @@ describeSubscriptions(const char *pattern, bool verbose)
appendPQExpBuffer(&buf,
", subfailover AS \"%s\"\n",
gettext_noop("Failover"));
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ ", subdetectconflict AS \"%s\"\n",
+ gettext_noop("Detect conflict"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index d453e224d9..219fac7e71 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1946,9 +1946,10 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit");
+ COMPLETE_WITH("binary", "detect_conflict", "disable_on_error",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
COMPLETE_WITH("lsn");
@@ -3363,9 +3364,10 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit", "two_phase");
+ "detect_conflict", "disable_on_error", "enabled",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..17daf11dc7 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -98,6 +98,9 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool subdetectconflict; /* True if replication should perform
+ * conflict detection */
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* Connection string to the publisher */
text subconninfo BKI_FORCE_NOT_NULL;
@@ -151,6 +154,7 @@ typedef struct Subscription
* (i.e. the main slot and the table sync
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool detectconflict; /* True if conflict detection is enabled */
char *conninfo; /* Connection string to the publisher */
char *slotname; /* Name of the replication slot */
char *synccommit; /* Synchronous commit setting for worker */
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
new file mode 100644
index 0000000000..a9f521aaca
--- /dev/null
+++ b/src/include/replication/conflict.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * conflict.h
+ * Exports for conflict detection and log
+ *
+ * Copyright (c) 2012-2024, PostgreSQL Global Development Group
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef CONFLICT_H
+#define CONFLICT_H
+
+#include "access/xlogdefs.h"
+#include "executor/tuptable.h"
+#include "nodes/execnodes.h"
+#include "utils/relcache.h"
+#include "utils/timestamp.h"
+
+/*
+ * Conflict types that could be encountered when applying remote changes.
+ */
+typedef enum
+{
+ /* The row to be inserted violates unique constraint */
+ CT_INSERT_EXISTS,
+
+ /* The row to be updated was modified by a different origin */
+ CT_UPDATE_DIFFER,
+
+ /* The row to be updated is missing */
+ CT_UPDATE_MISSING,
+
+ /* The row to be deleted is missing */
+ CT_DELETE_MISSING,
+
+ /* The row to be deleted was modified by a different origin */
+ CT_DELETE_DIFFER,
+} ConflictType;
+
+extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts);
+extern void ReportApplyConflict(int elevel, ConflictType type,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot);
+extern void InitConflictIndexes(ResultRelInfo *relInfo);
+
+#endif
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 5c2f1ee517..3d8b8c5d32 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -116,18 +116,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -145,10 +145,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -157,10 +157,10 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = 'newname');
ALTER SUBSCRIPTION regress_testsub SET (password_required = false);
ALTER SUBSCRIPTION regress_testsub SET (run_as_owner = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (password_required = true);
@@ -176,10 +176,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -188,10 +188,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -223,10 +223,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -255,19 +255,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -279,27 +279,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -314,10 +314,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -332,10 +332,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -371,10 +371,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
--fail - alter of two_phase option not supported.
@@ -383,10 +383,10 @@ ERROR: unrecognized subscription parameter: "two_phase"
-- but can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -396,10 +396,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -412,18 +412,42 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+ERROR: detect_conflict requires a Boolean value
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | t | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 3e5ba4cb8c..a77b196704 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -290,6 +290,21 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
-- let's do some tests with pg_create_subscription rather than superuser
SET SESSION AUTHORIZATION regress_subscription_user3;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 471e981962..78c0307165 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -331,11 +331,12 @@ is( $result, qq(1|bar
2|baz),
'update works with REPLICA IDENTITY FULL and a primary key');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber->append_conf('postgresql.conf', "log_min_messages = debug1");
-$node_subscriber->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub SET (detect_conflict = true)"
+);
$node_subscriber->safe_psql('postgres', "DELETE FROM tab_full_pk");
@@ -352,10 +353,10 @@ $node_publisher->wait_for_catchup('tap_sub');
my $logfile = slurp_file($node_subscriber->logfile, $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation "tab_full_pk"/,
+ qr/conflict update_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be updated./m,
'update target row is missing');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab_full_pk"/,
+ qr/conflict delete_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be deleted./m,
'delete target row is missing');
$node_subscriber->append_conf('postgresql.conf',
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 29580525a9..7a66a06b51 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -343,12 +343,12 @@ $result =
$node_subscriber2->safe_psql('postgres', "SELECT a FROM tab1 ORDER BY 1");
is($result, qq(), 'truncate of tab1 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = true)"
+);
$node_publisher->safe_psql('postgres',
"INSERT INTO tab1 VALUES (1, 'foo'), (4, 'bar'), (10, 'baz')");
@@ -372,21 +372,21 @@ $node_publisher->wait_for_catchup('sub2');
my $logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab1_2_2"/,
+ qr/conflict update_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_1"/,
+ qr/conflict delete_missing detected on relation "public.tab1_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_2_2"/,
+ qr/conflict delete_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_def"/,
+ qr/conflict delete_missing detected on relation "public.tab1_def".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_def');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = false)"
+);
# Tests for replication using root table identity and schema
@@ -773,12 +773,12 @@ pub_tab2|3|yyy
pub_tab2|5|zzz
xxx_c|6|aaa), 'inserts into tab2 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = true)"
+);
$node_subscriber1->safe_psql('postgres', "DELETE FROM tab2");
@@ -796,15 +796,35 @@ $node_publisher->wait_for_catchup('sub2');
$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab2_1"/,
+ qr/conflict update_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab2_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab2_1"/,
+ qr/conflict delete_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab2_1');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+# Enable the track_commit_timestamp to detect the conflict when attempting
+# to update a row that was previously modified by a different origin.
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = on');
+$node_subscriber1->restart;
+
+$node_subscriber1->safe_psql('postgres', "INSERT INTO tab2 VALUES (3, 'yyy')");
+$node_publisher->safe_psql('postgres',
+ "UPDATE tab2 SET b = 'quux' WHERE a = 3");
+
+$node_publisher->wait_for_catchup('sub_viaroot');
+
+$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
+ok( $logfile =~
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/,
+ 'updating a tuple that was modified by a different origin');
+
+# The remaining tests no longer test conflict detection.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = false)"
+);
+
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_subscriber1->restart;
# Test that replication continues to work correctly after altering the
# partition of a partitioned target table.
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 0ab57a4b5b..496a3c6cd9 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -30,7 +30,7 @@ sub test_skip_lsn
# ERROR with its CONTEXT when retrieving this information.
my $contents = slurp_file($node_subscriber->logfile, $offset);
$contents =~
- qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
or die "could not get error-LSN";
my $lsn = $1;
@@ -83,6 +83,7 @@ $node_subscriber->append_conf(
'postgresql.conf',
qq[
max_prepared_transactions = 10
+track_commit_timestamp = on
]);
$node_subscriber->start;
@@ -109,7 +110,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
# Initial synchronization failure causes the subscription to be disabled.
diff --git a/src/test/subscription/t/030_origin.pl b/src/test/subscription/t/030_origin.pl
index 056561f008..8c929c07c7 100644
--- a/src/test/subscription/t/030_origin.pl
+++ b/src/test/subscription/t/030_origin.pl
@@ -27,9 +27,14 @@ my $stderr;
my $node_A = PostgreSQL::Test::Cluster->new('node_A');
$node_A->init(allows_streaming => 'logical');
$node_A->start;
+
# node_B
my $node_B = PostgreSQL::Test::Cluster->new('node_B');
$node_B->init(allows_streaming => 'logical');
+
+# Enable the track_commit_timestamp to detect the conflict when attempting to
+# update a row that was previously modified by a different origin.
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = on');
$node_B->start;
# Create table on node_A
@@ -139,6 +144,44 @@ is($result, qq(),
'Remote data originating from another node (not the publisher) is not replicated when origin parameter is none'
);
+###############################################################################
+# Check that the conflict can be detected when attempting to update or
+# delete a row that was previously modified by a different source.
+###############################################################################
+
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = true);
+ DELETE FROM tab;");
+
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (32);");
+
+# The update should update the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "UPDATE tab SET a = 33 WHERE a = 32;");
+
+$node_B->wait_for_log(
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_B->safe_psql('postgres', "DELETE FROM tab;");
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (33);");
+
+# The delete should remove the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "DELETE FROM tab WHERE a = 33;");
+
+$node_B->wait_for_log(
+ qr/Deleting a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_A->wait_for_catchup($subname_BA);
+$node_B->wait_for_catchup($subname_AB);
+
+# The remaining tests no longer test conflict detection.
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = false);");
+
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_B->restart;
+
###############################################################################
# Specifying origin = NONE indicates that the publisher should only replicate the
# changes that are generated locally from node_B, but in this case since the
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b4d7f9217c..2098ed7467 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -467,6 +467,7 @@ ConditionVariableMinimallyPadded
ConditionalStack
ConfigData
ConfigVariable
+ConflictType
ConnCacheEntry
ConnCacheKey
ConnParams
--
2.34.1
[application/octet-stream] v6-0002-DDL-command-to-configure-Global-Conflict-Resolver.patch (26.5K, ../../CAJpy0uDAMtrqXWFtxrXSwTNO8awFAoENtbo47Czj-67k3OVMyQ@mail.gmail.com/5-v6-0002-DDL-command-to-configure-Global-Conflict-Resolver.patch)
download | inline diff:
From 3540c8d1b24be2170842d06baa13f3ab40018be1 Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Wed, 3 Jul 2024 10:23:58 +0530
Subject: [PATCH v6 2/5] DDL command to configure Global Conflict Resolvers
This patch adds new DDL commands to configure resolvers
at global level:
a) To set global resolver for a given conflict_type:
SET CONFLICT RESOLVER 'conflict_resolver' FOR 'conflict_type'
b) To reset to default:
RESET CONFLICT RESOLVER FOR 'conflict_type'
A new catalog table pg_conflict has been created to store
global conflict_type and conflict_resolver configurations given by
above DDL command. Initially, this catalog table holds default
configuration for resolvers which is overwritten by the ones given
by the user in 'SET CONFLICT RESOLVER' command.
'RESET CONFLICT RESOLVER' command will reset the resolver to default.
This patch provides support to configure resolvers for below conflict
types:
- insert_exists
- update_differ
- update_missing
- delete_missing
- delete_differ
---
src/backend/parser/gram.y | 42 +++-
src/backend/replication/logical/conflict.c | 218 +++++++++++++++++-
src/backend/tcop/utility.c | 15 ++
src/include/catalog/Makefile | 4 +-
src/include/catalog/meson.build | 2 +
src/include/catalog/pg_conflict.dat | 21 ++
src/include/catalog/pg_conflict.h | 43 ++++
src/include/nodes/parsenodes.h | 12 +
src/include/parser/kwlist.h | 1 +
src/include/replication/conflict.h | 38 ++-
src/include/tcop/cmdtaglist.h | 1 +
.../regress/expected/conflict_resolver.out | 54 +++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/conflict_resolver.sql | 28 +++
src/tools/pgindent/typedefs.list | 2 +
15 files changed, 476 insertions(+), 7 deletions(-)
create mode 100644 src/include/catalog/pg_conflict.dat
create mode 100644 src/include/catalog/pg_conflict.h
create mode 100644 src/test/regress/expected/conflict_resolver.out
create mode 100644 src/test/regress/sql/conflict_resolver.sql
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a043fd4c66..42726fe3a6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -292,7 +292,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
AlterRoleStmt AlterRoleSetStmt AlterPolicyStmt AlterStatsStmt
AlterDefaultPrivilegesStmt DefACLAction
AnalyzeStmt CallStmt ClosePortalStmt ClusterStmt CommentStmt
- ConstraintsSetStmt CopyStmt CreateAsStmt CreateCastStmt
+ ConflictResolverStmt ConstraintsSetStmt CopyStmt CreateAsStmt CreateCastStmt
CreateDomainStmt CreateExtensionStmt CreateGroupStmt CreateOpClassStmt
CreateOpFamilyStmt AlterOpFamilyStmt CreatePLangStmt
CreateSchemaStmt CreateSeqStmt CreateStmt CreateStatsStmt CreateTableSpaceStmt
@@ -613,6 +613,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> opt_check_option
%type <str> opt_provider security_label
+%type <str> conflict_type conflict_resolver
%type <target> xml_attribute_el
%type <list> xml_attribute_list xml_attributes
@@ -772,7 +773,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
RANGE READ REAL REASSIGN RECHECK RECURSIVE REF_P REFERENCES REFERENCING
REFRESH REINDEX RELATIVE_P RELEASE RENAME REPEATABLE REPLACE REPLICA
- RESET RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
+ RESET RESOLVER RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
ROUTINE ROUTINES ROW ROWS RULE
SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT
@@ -1039,6 +1040,7 @@ stmt:
| ClosePortalStmt
| ClusterStmt
| CommentStmt
+ | ConflictResolverStmt
| ConstraintsSetStmt
| CopyStmt
| CreateAmStmt
@@ -7266,7 +7268,41 @@ comment_text:
| NULL_P { $$ = NULL; }
;
+/*****************************************************************************
+ *
+ * SET CONFLICT RESOLVER <conflict_resolver> FOR <conflict_type>
+ * RESET CONFLICT RESOLVER FOR <conflict_type>
+ *
+ *****************************************************************************/
+
+ConflictResolverStmt:
+ SET CONFLICT RESOLVER conflict_resolver FOR conflict_type
+ {
+ ConflictResolverStmt *n = makeNode(ConflictResolverStmt);
+
+ n->conflict_resolver = $4;
+ n->conflict_type = $6;
+ n->isReset = false;
+ $$ = (Node *) n;
+ }
+ | RESET CONFLICT RESOLVER FOR conflict_type
+ {
+ ConflictResolverStmt *n = makeNode(ConflictResolverStmt);
+ n->conflict_type = $5;
+ n->isReset = true;
+ $$ = (Node *) n;
+ }
+ ;
+conflict_type:
+ Sconst { $$ = $1; }
+ | NULL_P { $$ = NULL; }
+ ;
+
+conflict_resolver:
+ Sconst { $$ = $1; }
+ | NULL_P { $$ = NULL; }
+ ;
/*****************************************************************************
*
* SECURITY LABEL [FOR <provider>] ON <object> IS <label>
@@ -17797,6 +17833,7 @@ unreserved_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
@@ -18428,6 +18465,7 @@ bare_label_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index b90e64b05b..93d9659752 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -1,13 +1,13 @@
/*-------------------------------------------------------------------------
* conflict.c
- * Functionality for detecting and logging conflicts.
+ * Functionality for detecting, logging and configuring conflicts.
*
* Copyright (c) 2024, PostgreSQL Global Development Group
*
* IDENTIFICATION
* src/backend/replication/logical/conflict.c
*
- * This file contains the code for detecting and logging conflicts on
+ * This file contains the code for detecting and handling conflicts on
* the subscriber during logical replication.
*-------------------------------------------------------------------------
*/
@@ -15,10 +15,19 @@
#include "postgres.h"
#include "access/commit_ts.h"
+#include "access/genam.h"
+#include "access/htup_details.h"
+#include "access/skey.h"
+#include "access/table.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_conflict.h"
#include "replication/conflict.h"
#include "replication/origin.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/timestamp.h"
const char *const ConflictTypeNames[] = {
[CT_INSERT_EXISTS] = "insert_exists",
@@ -28,6 +37,54 @@ const char *const ConflictTypeNames[] = {
[CT_DELETE_DIFFER] = "delete_differ"
};
+const char *const ConflictResolverNames[] = {
+ [CR_REMOTE_APPLY] = "remote_apply",
+ [CR_KEEP_LOCAL] = "keep_local",
+ [CR_APPLY_OR_SKIP] = "apply_or_skip",
+ [CR_APPLY_OR_ERROR] = "apply_or_error",
+ [CR_SKIP] = "skip",
+ [CR_ERROR] = "error"
+};
+
+#define CONFLICT_TYPE_MAX_RESOLVERS 4
+
+/*
+ * Valid conflict resolvers for each conflict type.
+ *
+ * XXX: If we do not want to maintain different resolvers such as
+ * apply_or_skip and apply_or_error for update_missing conflict,
+ * then we can retain remote_apply and keep_local only. Then these
+ * resolvers in context of update_missing will mean:
+ *
+ * keep_local: do not apply the update as INSERT.
+ * remote_apply: apply the update as INSERT. If we could not apply,
+ * then log and skip.
+ *
+ * Similarly SKIP can be replaced with KEEP_LOCAL for both update_missing
+ * and delete_missing conflicts. For missing rows, 'SKIP' sounds more user
+ * friendly name for a resolver and thus has been added here.
+ */
+const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
+ [CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+};
+
+/*
+ * Default conflict resolver for each conflict type.
+ * If this changes, change it in pg_conflict.dat as well.
+ */
+const int ConflictTypeDefaultResolvers[] = {
+ [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
+ [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
+ [CT_DELETE_MISSING] = CR_SKIP,
+ [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+
+};
+
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
TransactionId localxmin,
@@ -35,6 +92,7 @@ static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
TimestampTz localts,
TupleTableSlot *conflictslot);
+
/*
* Get the xmin and commit timestamp data (origin and timestamp) associated
* with the provided local tuple.
@@ -189,3 +247,159 @@ build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
return conflict_row;
}
+
+/*
+ * Validate the conflict type and resolver.
+ *
+ * Return ConflictType enum value corresponding to given conflict_type.
+ */
+static ConflictType
+validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver,
+ bool isReset)
+{
+ ConflictType type;
+ ConflictResolver resolver;
+ bool valid = false;
+ int i;
+
+ /* Check conflict type validity */
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ if (strcmp(ConflictTypeNames[type], conflict_type) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict type", conflict_type));
+
+ /* If it is RESET statement, we are done */
+ if (isReset)
+ return type;
+
+ /* Reset */
+ valid = false;
+
+ /* Check conflict resolver validity. */
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_resolver) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ /*
+ * XXX: Shall we give supported resolvers list for given type in error
+ * hint?
+ */
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver", conflict_resolver));
+
+ /* Reset */
+ valid = false;
+
+ /* Check if conflict resolver is a valid one for the given conflict type */
+ for (i = 0; i < CONFLICT_TYPE_MAX_RESOLVERS; i++)
+ {
+ if (ConflictTypeResolverMap[type][i] == resolver)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ /*
+ * XXX: Shall we give supported resolvers list for given type in error
+ * hint?
+ */
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver for conflict type %s",
+ conflict_resolver,
+ conflict_type));
+ return type;
+}
+
+/*
+ * Execute Conflict Resolver Stmt
+ *
+ * Validate conflict_type and conflict_resolver given by the user
+ * and save the info in pg_conflict system catalog.
+ */
+void
+ExecConflictResolverStmt(ConflictResolverStmt *stmt)
+{
+ Relation pg_conflict;
+ Datum values[Natts_pg_conflict];
+ bool nulls[Natts_pg_conflict];
+ bool replaces[Natts_pg_conflict];
+ ScanKeyData keys[1];
+ SysScanDesc scan;
+ HeapTuple oldtup;
+ HeapTuple newtup = NULL;
+ ConflictType type;
+
+ type = validate_conflict_type_and_resolver(stmt->conflict_type,
+ stmt->conflict_resolver,
+ stmt->isReset);
+
+
+ /* Prepare to update a tuple in pg_conflict system catalog */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_conflict_conftype - 1] =
+ CStringGetTextDatum(stmt->conflict_type);
+
+ if (stmt->isReset)
+ {
+ /* Set resolver to default */
+ ConflictResolver resolver = ConflictTypeDefaultResolvers[type];
+
+ values[Anum_pg_conflict_confres - 1] =
+ CStringGetTextDatum(ConflictResolverNames[resolver]);
+ }
+ else
+ values[Anum_pg_conflict_confres - 1] =
+ CStringGetTextDatum(stmt->conflict_resolver);
+
+ /* Use the index to search for a matching old tuple */
+ ScanKeyInit(&keys[0],
+ Anum_pg_conflict_conftype,
+ BTEqualStrategyNumber, F_TEXTEQ,
+ values[Anum_pg_conflict_conftype - 1]);
+
+ pg_conflict = table_open(ConflictResRelationId, RowExclusiveLock);
+
+ scan = systable_beginscan(pg_conflict, ConflictTypeIndexId, true,
+ NULL, 1, keys);
+
+ oldtup = systable_getnext(scan);
+ if (HeapTupleIsValid(oldtup))
+ {
+ replaces[Anum_pg_conflict_confres - 1] = true;
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(pg_conflict),
+ values, nulls, replaces);
+ CatalogTupleUpdate(pg_conflict, &oldtup->t_self, newtup);
+ }
+ else
+ Assert(oldtup); /* We must have a tuple for given
+ * conflict_type */
+
+ systable_endscan(scan);
+
+ if (newtup != NULL)
+ heap_freetuple(newtup);
+
+ table_close(pg_conflict, RowExclusiveLock);
+}
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index fa66b8017e..4ddf4aaff0 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -59,6 +59,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/conflict.h"
#include "rewrite/rewriteDefine.h"
#include "storage/fd.h"
#include "tcop/utility.h"
@@ -163,6 +164,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_AlterUserMappingStmt:
case T_CommentStmt:
case T_CompositeTypeStmt:
+ case T_ConflictResolverStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
case T_CreateConversionStmt:
@@ -1062,6 +1064,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
break;
}
+ case T_ConflictResolverStmt:
+ /* no event triggers for conflict types and resolvers */
+ ExecConflictResolverStmt((ConflictResolverStmt *) parsetree);
+ break;
+
default:
/* All other statement types have event trigger support */
ProcessUtilitySlow(pstate, pstmt, queryString,
@@ -2659,6 +2666,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_COMMENT;
break;
+ case T_ConflictResolverStmt:
+ tag = CMDTAG_CONFLICT_RESOLVER;
+ break;
+
case T_SecLabelStmt:
tag = CMDTAG_SECURITY_LABEL;
break;
@@ -3334,6 +3345,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_ConflictResolverStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_SecLabelStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile
index 167f91a6e3..8619d73e5a 100644
--- a/src/include/catalog/Makefile
+++ b/src/include/catalog/Makefile
@@ -81,7 +81,8 @@ CATALOG_HEADERS := \
pg_publication_namespace.h \
pg_publication_rel.h \
pg_subscription.h \
- pg_subscription_rel.h
+ pg_subscription_rel.h \
+ pg_conflict.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h)
@@ -97,6 +98,7 @@ POSTGRES_BKI_DATA = \
pg_cast.dat \
pg_class.dat \
pg_collation.dat \
+ pg_conflict.dat \
pg_conversion.dat \
pg_database.dat \
pg_language.dat \
diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index f70d1daba5..4d6732a303 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -69,6 +69,7 @@ catalog_headers = [
'pg_publication_rel.h',
'pg_subscription.h',
'pg_subscription_rel.h',
+ 'pg_conflict.h',
]
# The .dat files we need can just be listed alphabetically.
@@ -81,6 +82,7 @@ bki_data = [
'pg_cast.dat',
'pg_class.dat',
'pg_collation.dat',
+ 'pg_conflict.dat',
'pg_conversion.dat',
'pg_database.dat',
'pg_language.dat',
diff --git a/src/include/catalog/pg_conflict.dat b/src/include/catalog/pg_conflict.dat
new file mode 100644
index 0000000000..8d6183e3f2
--- /dev/null
+++ b/src/include/catalog/pg_conflict.dat
@@ -0,0 +1,21 @@
+#----------------------------------------------------------------------
+#
+# pg_conflict.dat
+# Initial contents of the pg_conflict system catalog.
+#
+# Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_conflict.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ conftype => 'insert_exists', confres => 'remote_apply' },
+{ conftype => 'update_differ', confres => 'remote_apply' },
+{ conftype => 'update_missing', confres => 'apply_or_skip' },
+{ conftype => 'delete_missing', confres => 'skip' },
+{ conftype => 'delete_differ', confres => 'remote_apply' }
+
+]
diff --git a/src/include/catalog/pg_conflict.h b/src/include/catalog/pg_conflict.h
new file mode 100644
index 0000000000..e3fe3e6d30
--- /dev/null
+++ b/src/include/catalog/pg_conflict.h
@@ -0,0 +1,43 @@
+/* -------------------------------------------------------------------------
+ *
+ * pg_conflict.h
+ * definition of the "global conflict detection" system catalog
+ * (pg_conflict)
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_conflict.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ * -------------------------------------------------------------------------
+ */
+#ifndef PG_CONFLICT_H
+#define PG_CONFLICT_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_conflict_d.h"
+
+/* ----------------
+ * pg_conflict definition. cpp turns this into
+ * typedef struct FormData_pg_conflict
+ * ----------------
+ */
+CATALOG(pg_conflict,8688,ConflictResRelationId)
+{
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text conftype BKI_FORCE_NOT_NULL; /* conflict type */
+ text confres BKI_FORCE_NOT_NULL; /* conflict resolver */
+#endif
+} FormData_pg_conflict;
+
+DECLARE_TOAST(pg_conflict, 8689, 8690);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_conflict_type_index, 8691, ConflictTypeIndexId, pg_conflict, btree(conftype text_ops));
+
+MAKE_SYSCACHE(CONFLICTTYPE, pg_conflict_type_index, 16);
+
+#endif /* PG_CONFLICT_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 85a62b538e..1d41169d5e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3262,6 +3262,18 @@ typedef struct CommentStmt
char *comment; /* Comment to insert, or NULL to remove */
} CommentStmt;
+/* ----------------------
+ * CONFLICT RESOLVER Statement
+ * ----------------------
+ */
+typedef struct ConflictResolverStmt
+{
+ NodeTag type;
+ char *conflict_type;
+ char *conflict_resolver;
+ bool isReset;
+} ConflictResolverStmt;
+
/* ----------------------
* SECURITY LABEL Statement
* ----------------------
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f7fe834cf4..21e5cb162d 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -377,6 +377,7 @@ PG_KEYWORD("repeatable", REPEATABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replace", REPLACE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replica", REPLICA, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("reset", RESET, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("resolver", RESOLVER, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restart", RESTART, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restrict", RESTRICT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("return", RETURN, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index a9f521aaca..c6023e0914 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -10,6 +10,7 @@
#define CONFLICT_H
#include "access/xlogdefs.h"
+#include "catalog/objectaddress.h"
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
#include "utils/relcache.h"
@@ -36,6 +37,41 @@ typedef enum
CT_DELETE_DIFFER,
} ConflictType;
+/* Min and max conflict type */
+#define CT_MIN CT_INSERT_EXISTS
+#define CT_MAX CT_DELETE_DIFFER
+
+/*
+ * Conflict resolvers that can be used to resolve various conflicts.
+ *
+ * See ConflictTypeResolverMap in conflcit.c to find out which all
+ * resolvers are supported for each conflict type.
+ */
+typedef enum ConflictResolver
+{
+ /* Apply the remote change */
+ CR_REMOTE_APPLY = 1,
+
+ /* Keep the local change */
+ CR_KEEP_LOCAL,
+
+ /* Apply the remote change; skip if it can not be applied */
+ CR_APPLY_OR_SKIP,
+
+ /* Apply the remote change; emit error if it can not be applied */
+ CR_APPLY_OR_ERROR,
+
+ /* Skip applying the change */
+ CR_SKIP,
+
+ /* Error out */
+ CR_ERROR,
+} ConflictResolver;
+
+/* Min and max conflict resolver */
+#define CR_MIN CR_REMOTE_APPLY
+#define CR_MAX CR_ERROR
+
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
extern void ReportApplyConflict(int elevel, ConflictType type,
@@ -43,5 +79,5 @@ extern void ReportApplyConflict(int elevel, ConflictType type,
TransactionId localxmin, RepOriginId localorigin,
TimestampTz localts, TupleTableSlot *conflictslot);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
-
+extern void ExecConflictResolverStmt(ConflictResolverStmt *stmt);
#endif
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index 7fdcec6dd9..562478e232 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -80,6 +80,7 @@ PG_CMDTAG(CMDTAG_CLUSTER, "CLUSTER", false, false, false)
PG_CMDTAG(CMDTAG_COMMENT, "COMMENT", true, false, false)
PG_CMDTAG(CMDTAG_COMMIT, "COMMIT", false, false, false)
PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
+PG_CMDTAG(CMDTAG_CONFLICT_RESOLVER, "CONFLICT RESOLVER", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
diff --git a/src/test/regress/expected/conflict_resolver.out b/src/test/regress/expected/conflict_resolver.out
new file mode 100644
index 0000000000..53ecb430fa
--- /dev/null
+++ b/src/test/regress/expected/conflict_resolver.out
@@ -0,0 +1,54 @@
+--check default global resolvers in system catalog
+select * from pg_conflict order by conftype;
+ conftype | confres
+----------------+---------------
+ delete_differ | remote_apply
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | remote_apply
+ update_missing | apply_or_skip
+(5 rows)
+
+--
+-- Test of SET/RESET CONFLICT RESOLVER with invalid names
+--
+SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
+ERROR: aaaa is not a valid conflict type
+SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
+ERROR: bbbbb is not a valid conflict resolver
+SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
+ERROR: remote_apply is not a valid conflict resolver for conflict type delete_missing
+RESET CONFLICT RESOLVER for 'ct'; -- fail
+ERROR: ct is not a valid conflict type
+--
+-- Test of SET/RESET CONFLICT RESOLVER with valid names
+--
+SET CONFLICT RESOLVER 'error' for 'delete_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
+SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
+SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
+--check new resolvers are saved
+select * from pg_conflict order by conftype;
+ conftype | confres
+----------------+----------------
+ delete_differ | keep_local
+ delete_missing | error
+ insert_exists | keep_local
+ update_differ | keep_local
+ update_missing | apply_or_error
+(5 rows)
+
+RESET CONFLICT RESOLVER for 'delete_missing';
+RESET CONFLICT RESOLVER for 'insert_exists';
+--check resolvers are reset to default for delete_missing and insert_exists
+select * from pg_conflict order by conftype;
+ conftype | confres
+----------------+----------------
+ delete_differ | keep_local
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | keep_local
+ update_missing | apply_or_error
+(5 rows)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 2429ec2bba..76ea2d359b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -48,7 +48,7 @@ test: create_index create_index_spgist create_view index_including index_includi
# ----------
# Another group of parallel tests
# ----------
-test: create_aggregate create_function_sql create_cast constraints triggers select inherit typed_table vacuum drop_if_exists updatable_views roleattributes create_am hash_func errors infinite_recurse
+test: create_aggregate create_function_sql create_cast conflict_resolver constraints triggers select inherit typed_table vacuum drop_if_exists updatable_views roleattributes create_am hash_func errors infinite_recurse
# ----------
# sanity_check does a vacuum, affecting the sort order of SELECT *
diff --git a/src/test/regress/sql/conflict_resolver.sql b/src/test/regress/sql/conflict_resolver.sql
new file mode 100644
index 0000000000..f83d14b229
--- /dev/null
+++ b/src/test/regress/sql/conflict_resolver.sql
@@ -0,0 +1,28 @@
+--check default global resolvers in system catalog
+select * from pg_conflict order by conftype;
+
+--
+-- Test of SET/RESET CONFLICT RESOLVER with invalid names
+--
+SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
+SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
+SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
+RESET CONFLICT RESOLVER for 'ct'; -- fail
+
+--
+-- Test of SET/RESET CONFLICT RESOLVER with valid names
+--
+SET CONFLICT RESOLVER 'error' for 'delete_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
+SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
+SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
+SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
+
+--check new resolvers are saved
+select * from pg_conflict order by conftype;
+
+RESET CONFLICT RESOLVER for 'delete_missing';
+RESET CONFLICT RESOLVER for 'insert_exists';
+
+--check resolvers are reset to default for delete_missing and insert_exists
+select * from pg_conflict order by conftype;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2098ed7467..1dfcc57fb1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -467,6 +467,8 @@ ConditionVariableMinimallyPadded
ConditionalStack
ConfigData
ConfigVariable
+ConflictResolver
+ConflictResolverStmt
ConflictType
ConnCacheEntry
ConnCacheKey
--
2.34.1
[application/octet-stream] v6-0005-Configure-table-level-conflict-resolvers.patch (48.9K, ../../CAJpy0uDAMtrqXWFtxrXSwTNO8awFAoENtbo47Czj-67k3OVMyQ@mail.gmail.com/6-v6-0005-Configure-table-level-conflict-resolvers.patch)
download | inline diff:
From ed9ec14fa145a4fedf0571bd1098264d89657f20 Mon Sep 17 00:00:00 2001
From: Shveta Malik <[email protected]>
Date: Wed, 17 Jul 2024 10:13:38 +0530
Subject: [PATCH v6 5/5] Configure table level conflict resolvers
This patch provides support for configuring table level conflict
resolvers using ALTER TABLE cmd.
Syntax to SET resolvers:
ALTER TABLE <name> SET CONFLICT RESOLVER <resolver1> on <conflict_type1>,
SET CONFLICT RESOLVER <resolver2> on <conflict_type2>, ...;
A new catalog table pg_conflict_rel has been created to store
table-level conflict_type and conflict_resolver configurations given by
above DDL command.
Syntax to RESET resolvers:
ALTER TABLE <name> RESET CONFLICT RESOLVER on <conflict_type1>,
RESET CONFLICT RESOLVER on <conflict_type2>, ...;
Above RESET command will remove entry for that particular conflict_type for
the given table from pg_conflict_rel catalog table.
---
src/backend/catalog/dependency.c | 6 +
src/backend/catalog/objectaddress.c | 30 ++
src/backend/commands/tablecmds.c | 70 ++++
src/backend/parser/gram.y | 30 ++
src/backend/replication/logical/conflict.c | 330 +++++++++++++++++-
src/include/catalog/Makefile | 4 +-
src/include/catalog/meson.build | 1 +
src/include/catalog/pg_conflict.h | 2 +-
src/include/catalog/pg_conflict_rel.h | 58 +++
src/include/nodes/parsenodes.h | 2 +
src/include/replication/conflict.h | 12 +
.../regress/expected/conflict_resolver.out | 245 ++++++++++++-
src/test/regress/expected/oidjoins.out | 1 +
src/test/regress/sql/conflict_resolver.sql | 161 ++++++++-
14 files changed, 928 insertions(+), 24 deletions(-)
create mode 100644 src/include/catalog/pg_conflict_rel.h
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0489cbabcb..9dd78d9094 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -31,6 +31,7 @@
#include "catalog/pg_auth_members.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_conflict_rel.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -79,6 +80,7 @@
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
#include "parser/parsetree.h"
+#include "replication/conflict.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/fmgroids.h"
@@ -1444,6 +1446,10 @@ doDeletion(const ObjectAddress *object, int flags)
RemovePublicationById(object->objectId);
break;
+ case ConflictRelRelationId:
+ RemoveTableConflictById(object->objectId);
+ break;
+
case CastRelationId:
case CollationRelationId:
case ConversionRelationId:
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 85a7b7e641..1456eedf46 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -30,6 +30,7 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
+#include "catalog/pg_conflict_rel.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
#include "catalog/pg_default_acl.h"
@@ -3005,6 +3006,35 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case ConflictRelRelationId:
+ {
+ HeapTuple confTup;
+ Datum typeDatum;
+ Datum resDatum;
+ char *contype;
+ char *conres;
+
+ confTup = SearchSysCache1(CONFLICTRELOID,
+ ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(confTup))
+ {
+ if (!missing_ok)
+ elog(ERROR, "cache lookup failed for table conflict %u",
+ object->objectId);
+ break;
+ }
+
+ typeDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, confTup,
+ Anum_pg_conflict_rel_confrtype);
+ resDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, confTup,
+ Anum_pg_conflict_rel_confrres);
+ contype = TextDatumGetCString(typeDatum);
+ conres = TextDatumGetCString(resDatum);
+ ReleaseSysCache(confTup);
+ appendStringInfo(&buffer, _("conflict_resolver %s on conflict_type %s"),
+ conres, contype);
+ break;
+ }
case ConstraintRelationId:
{
HeapTuple conTup;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 721d24783b..4ebd80950a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -83,6 +83,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/conflict.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -662,6 +663,11 @@ static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab,
AlterTableUtilityContext *context);
static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation rel,
PartitionCmd *cmd, AlterTableUtilityContext *context);
+static void ATExecSetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode);
+static void
+ ATExecResetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode);
/* ----------------------------------------------------------------
* DefineRelation
@@ -1245,6 +1251,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
CloneForeignKeyConstraints(NULL, parent, rel);
+ /* Inherit conflict resolvers configuration from parent. */
+ InheritTableConflictResolvers(rel, parent);
+
table_close(parent, NoLock);
}
@@ -4543,6 +4552,8 @@ AlterTableGetLockLevel(List *cmds)
case AT_SetExpression:
case AT_DropExpression:
case AT_SetCompression:
+ case AT_SetConflictResolver:
+ case AT_ResetConflictResolver:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -5116,6 +5127,15 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_ResetConflictResolver: /* RESET CONFLICT RESOLVER */
+ case AT_SetConflictResolver: /* SET CONFLICT RESOLVER */
+ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE);
+ /* Recursion occurs during execution phase */
+ /* No command-specific prep needed except saving recurse flag */
+ if (recurse)
+ cmd->recurse = true;
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5528,6 +5548,14 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
ATExecMergePartitions(wqueue, tab, rel, (PartitionCmd *) cmd->def,
context);
break;
+ case AT_SetConflictResolver:
+ ATExecSetConflictResolver(rel, (ConflictResolverStmt *) cmd->def,
+ cmd->recurse, false, lockmode);
+ break;
+ case AT_ResetConflictResolver:
+ ATExecResetConflictResolver(rel, (ConflictResolverStmt *) cmd->def,
+ cmd->recurse, false, lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -6528,6 +6556,10 @@ alter_table_type_to_string(AlterTableType cmdtype)
return "ALTER COLUMN ... DROP IDENTITY";
case AT_ReAddStatistics:
return NULL; /* not real grammar */
+ case AT_SetConflictResolver:
+ return "SET CONFLICT RESOLVER";
+ case AT_ResetConflictResolver:
+ return "RESET CONFLICT RESOLVER";
}
return NULL;
@@ -15650,6 +15682,13 @@ CreateInheritance(Relation child_rel, Relation parent_rel, bool ispartition)
/* Match up the constraints and bump coninhcount as needed */
MergeConstraintsIntoExisting(child_rel, parent_rel);
+ /*
+ * Inherit resolvers configuration from parent if not explicitly set for
+ * child partition.
+ */
+ if (ispartition)
+ InheritTableConflictResolvers(child_rel, parent_rel);
+
/*
* OK, it looks valid. Make the catalog entries that show inheritance.
*/
@@ -16238,6 +16277,10 @@ RemoveInheritance(Relation child_rel, Relation parent_rel, bool expect_detached)
systable_endscan(scan);
table_close(catalogRelation, RowExclusiveLock);
+ /* Find inherited conflict resolvers and disinherit them */
+ if (is_partitioning)
+ ResetResolversInheritance(child_rel);
+
drop_parent_dependency(RelationGetRelid(child_rel),
RelationRelationId,
RelationGetRelid(parent_rel),
@@ -20775,3 +20818,30 @@ ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Keep the lock until commit. */
table_close(newPartRel, NoLock);
}
+
+/*
+ * ALTER TABLE <name> SET CONFLICT RESOLVER ...
+ */
+static void
+ATExecSetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ ValidateConflictTypeAndResolver(stmt->conflict_type, stmt->conflict_resolver, false);
+
+ SetTableConflictResolver(NULL, rel, stmt->conflict_type,
+ stmt->conflict_resolver,
+ recurse, recursing, lockmode);
+}
+
+/*
+ * ALTER TABLE <name> RESET CONFLICT RESOLVER ...
+ */
+static void
+ATExecResetConflictResolver(Relation rel, ConflictResolverStmt *stmt,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ ValidateConflictTypeAndResolver(stmt->conflict_type, stmt->conflict_resolver, true);
+
+ ResetTableConflictResolver(rel, stmt->conflict_type,
+ recurse, recursing, lockmode);
+}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 42726fe3a6..628b9d10e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -3015,6 +3015,36 @@ alter_table_cmd:
n->subtype = AT_GenericOptions;
n->def = (Node *) $1;
+ $$ = (Node *) n;
+ }
+ /*
+ * ALTER TABLE <name> SET CONFLICT RESOLVER <conflict_resolver>
+ * on <conflict_type>
+ */
+ | SET CONFLICT RESOLVER conflict_resolver ON conflict_type
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ ConflictResolverStmt *c = makeNode(ConflictResolverStmt);
+
+ c->conflict_resolver = $4;
+ c->conflict_type = $6;
+
+ n->subtype = AT_SetConflictResolver;
+ n->def = (Node *) c;
+
+ $$ = (Node *) n;
+ }
+ /* ALTER TABLE <name> RESET CONFLICT RESOLVER on <conflict_type> */
+ | RESET CONFLICT RESOLVER ON conflict_type
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ ConflictResolverStmt *c = makeNode(ConflictResolverStmt);
+
+ c->conflict_type = $5;
+
+ n->subtype = AT_ResetConflictResolver;
+ n->def = (Node *) c;
+
$$ = (Node *) n;
}
;
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index a8fa470cf7..7ec9fd6dd8 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -19,8 +19,11 @@
#include "access/htup_details.h"
#include "access/skey.h"
#include "access/table.h"
+#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_conflict.h"
+#include "catalog/pg_conflict_rel.h"
+#include "catalog/pg_inherits.h"
#include "executor/executor.h"
#include "replication/conflict.h"
#include "replication/logicalproto.h"
@@ -28,8 +31,10 @@
#include "replication/origin.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/syscache.h"
#include "utils/timestamp.h"
#include "utils/syscache.h"
@@ -320,9 +325,9 @@ build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
*
* Return ConflictType enum value corresponding to given conflict_type.
*/
-static ConflictType
-validate_conflict_type_and_resolver(char *conflict_type, char *conflict_resolver,
- bool isReset)
+ConflictType
+ValidateConflictTypeAndResolver(char *conflict_type, char *conflict_resolver,
+ bool isReset)
{
ConflictType type;
ConflictResolver resolver;
@@ -521,10 +526,9 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
HeapTuple newtup = NULL;
ConflictType type;
- type = validate_conflict_type_and_resolver(stmt->conflict_type,
- stmt->conflict_resolver,
- stmt->isReset);
-
+ type = ValidateConflictTypeAndResolver(stmt->conflict_type,
+ stmt->conflict_resolver,
+ stmt->isReset);
/* Prepare to update a tuple in pg_conflict system catalog */
memset(values, 0, sizeof(values));
@@ -552,7 +556,7 @@ ExecConflictResolverStmt(ConflictResolverStmt *stmt)
BTEqualStrategyNumber, F_TEXTEQ,
values[Anum_pg_conflict_conftype - 1]);
- pg_conflict = table_open(ConflictResRelationId, RowExclusiveLock);
+ pg_conflict = table_open(ConflictRelationId, RowExclusiveLock);
scan = systable_beginscan(pg_conflict, ConflictTypeIndexId, true,
NULL, 1, keys);
@@ -632,3 +636,313 @@ GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
return resolver;
}
+
+/*
+ * Update the table level conflict resolver for a conflict_type in
+ * pg_conflict_rel system catalog.
+ */
+void
+SetTableConflictResolver(Relation pg_conf_rel, Relation rel, char *conflict_type, char *conflict_resolver,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ Relation pg_conflict_rel;
+ Datum values[Natts_pg_conflict_rel];
+ bool nulls[Natts_pg_conflict_rel];
+ bool replaces[Natts_pg_conflict_rel];
+ HeapTuple oldtup;
+ HeapTuple newtup = NULL;
+ Oid relid = rel->rd_id;
+
+ /* Prepare to update a tuple. */
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ if (!pg_conf_rel)
+ pg_conflict_rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+ else
+ pg_conflict_rel = pg_conf_rel;
+
+ values[Anum_pg_conflict_rel_confrrelid - 1] = ObjectIdGetDatum(relid);
+ values[Anum_pg_conflict_rel_confrtype - 1] = CStringGetTextDatum(conflict_type);
+ values[Anum_pg_conflict_rel_confrres - 1] = CStringGetTextDatum(conflict_resolver);
+ values[Anum_pg_conflict_rel_confrinherited - 1] = BoolGetDatum(recursing);
+
+ oldtup = SearchSysCache2(CONFLICTRELTYPE,
+ values[Anum_pg_conflict_rel_confrrelid - 1],
+ values[Anum_pg_conflict_rel_confrtype - 1]);
+ if (HeapTupleIsValid(oldtup))
+ {
+ Form_pg_conflict_rel confForm = (Form_pg_conflict_rel) GETSTRUCT(oldtup);
+
+ /*
+ * Update resolver for recursing=false cases.
+ *
+ * recursing=true indicates it is a child parittion table and if it
+ * already has a resolver set then overwrite it only if it is an
+ * inherited resolver. We should not overwrite non-inherited resolvers
+ * for child partition tables during recursion.
+ */
+ if (!recursing || (recursing && confForm->confrinherited))
+ {
+ replaces[Anum_pg_conflict_rel_confrres - 1] = true;
+ replaces[Anum_pg_conflict_rel_confrinherited - 1] = !recursing;
+
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(pg_conflict_rel),
+ values, nulls, replaces);
+ CatalogTupleUpdate(pg_conflict_rel, &oldtup->t_self, newtup);
+ }
+ else
+ {
+ /*
+ * If we did not update resolver for this child table, do not
+ * update for child's child tables as well.
+ */
+ recurse = false;
+ }
+
+ ReleaseSysCache(oldtup);
+ }
+ /* If we didn't find an old tuple, insert a new one */
+ else
+ {
+ ObjectAddress myself,
+ referenced;
+ Oid conflict_oid;
+
+ conflict_oid = GetNewOidWithIndex(pg_conflict_rel, ConflictRelOidIndexId,
+ Anum_pg_conflict_rel_oid);
+ values[Anum_pg_conflict_rel_oid - 1] = ObjectIdGetDatum(conflict_oid);
+
+ newtup = heap_form_tuple(RelationGetDescr(pg_conflict_rel),
+ values, nulls);
+ CatalogTupleInsert(pg_conflict_rel, newtup);
+
+ /* Add dependency on the relation */
+ ObjectAddressSet(myself, ConflictRelRelationId, conflict_oid);
+ ObjectAddressSet(referenced, RelationRelationId, relid);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
+ }
+
+ if (HeapTupleIsValid(newtup))
+ heap_freetuple(newtup);
+
+ if (!pg_conf_rel)
+ table_close(pg_conflict_rel, RowExclusiveLock);
+
+ if (recurse && (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ List *children;
+ ListCell *lc;
+
+ children =
+ find_inheritance_children(RelationGetRelid(rel), lockmode);
+
+ foreach(lc, children)
+ {
+ Relation childrel;
+
+ /* find_inheritance_children already got lock */
+ childrel = table_open(lfirst_oid(lc), NoLock);
+ SetTableConflictResolver(NULL, childrel, conflict_type, conflict_resolver, recurse, true, lockmode);
+ table_close(childrel, NoLock);
+ }
+ }
+}
+
+/*
+ * Reset table level conflict_type configuration.
+ *
+ * Removes table's resolver configuration for given conflict_type from
+ * pg_conflict_rel system catalog.
+ */
+void
+ResetTableConflictResolver(Relation rel, char *conflict_type,
+ bool recurse, bool recursing, LOCKMODE lockmode)
+{
+ Oid relid = rel->rd_id;
+ HeapTuple tup;
+ ObjectAddress confobj;
+ Form_pg_conflict_rel confForm;
+
+ tup = SearchSysCache2(CONFLICTRELTYPE,
+ ObjectIdGetDatum(relid),
+ CStringGetTextDatum(conflict_type));
+
+ /* Nothing to reset */
+ if (!HeapTupleIsValid(tup))
+ return;
+
+ confForm = (Form_pg_conflict_rel) GETSTRUCT(tup);
+
+ if (confForm->confrinherited && !recursing)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TABLE_DEFINITION),
+ errmsg("cannot reset inherited resolver for conflict type \"%s\" of relation \"%s\"",
+ conflict_type, RelationGetRelationName(rel))));
+
+ /*
+ * While resetting resolver on parent table, reset resolver for child
+ * partition table only if it is inherited one.
+ *
+ * OTOH, if user has invoked RESET directly on child partition table,
+ * ensure we reset only non-inherited resolvers. Inherited resolvers can
+ * not be reset alone on child table, RESET has to come through parent
+ * table.
+ */
+ if ((recursing && confForm->confrinherited) ||
+ (!recursing && !confForm->confrinherited))
+ {
+ ObjectAddressSet(confobj, ConflictRelRelationId, confForm->oid);
+ performDeletion(&confobj, DROP_CASCADE, 0);
+ }
+ else
+ {
+ /*
+ * If we did not reset resolver for this child table, do not reset for
+ * child's child tables as well.
+ */
+ recurse = false;
+ }
+
+ ReleaseSysCache(tup);
+
+ if (recurse && (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ List *children;
+ ListCell *lc;
+
+ children =
+ find_inheritance_children(RelationGetRelid(rel), lockmode);
+
+ foreach(lc, children)
+ {
+ Relation childrel;
+
+ /* find_inheritance_children already got lock */
+ childrel = table_open(lfirst_oid(lc), NoLock);
+ ResetTableConflictResolver(childrel, conflict_type, recurse, true, lockmode);
+ table_close(childrel, NoLock);
+ }
+ }
+}
+
+/*
+ * Inherit conflict resolvers from parent paritioned table.
+ *
+ * If child partition table has resolvers set already for some conflict
+ * types, do not overwrite those, inherit rest from parent.
+ *
+ * Used during attach partition operation.
+ */
+void
+InheritTableConflictResolvers(Relation child_rel, Relation parent_rel)
+{
+ Relation pg_conflict_rel;
+ SysScanDesc parent_scan;
+ ScanKeyData parent_key;
+ HeapTuple parent_tuple;
+ Oid parent_relid = RelationGetRelid(parent_rel);
+
+ Assert(parent_rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
+
+ pg_conflict_rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&parent_key,
+ Anum_pg_conflict_rel_confrrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(parent_relid));
+
+ parent_scan = systable_beginscan(pg_conflict_rel, InvalidOid, false, NULL, 1, &parent_key);
+
+ while (HeapTupleIsValid(parent_tuple = systable_getnext(parent_scan)))
+ {
+ Datum typeDatum;
+ Datum resDatum;
+ char *parent_conftype;
+ char *parent_confres;
+
+ typeDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, parent_tuple,
+ Anum_pg_conflict_rel_confrtype);
+ parent_conftype = TextDatumGetCString(typeDatum);
+
+ resDatum = SysCacheGetAttrNotNull(CONFLICTRELOID, parent_tuple,
+ Anum_pg_conflict_rel_confrres);
+ parent_confres = TextDatumGetCString(resDatum);
+
+ SetTableConflictResolver(pg_conflict_rel, child_rel, parent_conftype, parent_confres, true, true, AccessExclusiveLock);
+ }
+
+ systable_endscan(parent_scan);
+ table_close(pg_conflict_rel, RowExclusiveLock);
+}
+
+/*
+ * Reset inheritance of conflict resolvers.
+ *
+ * Used during detach partition operation. Detach partition
+ * will not remove inherited resolvers but will mark them
+ * as non inherited.
+ */
+void
+ResetResolversInheritance(Relation rel)
+{
+ Relation pg_conflict_rel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+ Oid relid = RelationGetRelid(rel);
+
+ Assert(rel->rd_rel->relispartition);
+
+ pg_conflict_rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_conflict_rel_confrrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ scan = systable_beginscan(pg_conflict_rel, InvalidOid, false, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_conflict_rel conf =
+ (Form_pg_conflict_rel) GETSTRUCT(tuple);
+
+ if (conf->confrinherited)
+ {
+ HeapTuple copy_tuple = heap_copytuple(tuple);
+ Form_pg_conflict_rel copy_conf =
+ (Form_pg_conflict_rel) GETSTRUCT(copy_tuple);
+
+ copy_conf->confrinherited = false;
+ CatalogTupleUpdate(pg_conflict_rel, ©_tuple->t_self, copy_tuple);
+ heap_freetuple(copy_tuple);
+
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(pg_conflict_rel, RowExclusiveLock);
+}
+
+/*
+ * Remove the conflict resolver configuration by table conflict oid.
+ */
+void
+RemoveTableConflictById(Oid confid)
+{
+ Relation rel;
+ HeapTuple tup;
+
+ rel = table_open(ConflictRelRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(CONFLICTRELOID, ObjectIdGetDatum(confid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for table conflict %u", confid);
+
+ CatalogTupleDelete(rel, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ table_close(rel, RowExclusiveLock);
+}
diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile
index 8619d73e5a..e192375fa0 100644
--- a/src/include/catalog/Makefile
+++ b/src/include/catalog/Makefile
@@ -82,7 +82,9 @@ CATALOG_HEADERS := \
pg_publication_rel.h \
pg_subscription.h \
pg_subscription_rel.h \
- pg_conflict.h
+ pg_conflict.h \
+ pg_conflict_rel.h
+
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h)
diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index 4d6732a303..28924f5df8 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -70,6 +70,7 @@ catalog_headers = [
'pg_subscription.h',
'pg_subscription_rel.h',
'pg_conflict.h',
+ 'pg_conflict_rel.h',
]
# The .dat files we need can just be listed alphabetically.
diff --git a/src/include/catalog/pg_conflict.h b/src/include/catalog/pg_conflict.h
index e3fe3e6d30..c080a0d0be 100644
--- a/src/include/catalog/pg_conflict.h
+++ b/src/include/catalog/pg_conflict.h
@@ -26,7 +26,7 @@
* typedef struct FormData_pg_conflict
* ----------------
*/
-CATALOG(pg_conflict,8688,ConflictResRelationId)
+CATALOG(pg_conflict,8688,ConflictRelationId)
{
#ifdef CATALOG_VARLEN /* variable-length fields start here */
text conftype BKI_FORCE_NOT_NULL; /* conflict type */
diff --git a/src/include/catalog/pg_conflict_rel.h b/src/include/catalog/pg_conflict_rel.h
new file mode 100644
index 0000000000..636398baf1
--- /dev/null
+++ b/src/include/catalog/pg_conflict_rel.h
@@ -0,0 +1,58 @@
+/* -------------------------------------------------------------------------
+ *
+ * pg_conflict_rel.h
+ * definition of the "table level conflict detection" system
+ * catalog (pg_conflict_rel)
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_conflict_rel.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ * -------------------------------------------------------------------------
+ */
+#ifndef PG_CONFLICT_REL_H
+#define PG_CONFLICT_REL_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_conflict_rel_d.h"
+
+/* ----------------
+ * pg_conflict_rel definition. cpp turns this into
+ * typedef struct FormData_pg_conflict_rel
+ * ----------------
+ */
+CATALOG(pg_conflict_rel,8881,ConflictRelRelationId)
+{
+ Oid oid; /* OID of the object itself */
+ Oid confrrelid BKI_LOOKUP(pg_class); /* Oid of the relation
+ * having resolver */
+ bool confrinherited; /* Is this an inherited configuration */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text confrtype BKI_FORCE_NOT_NULL; /* conflict type */
+ text confrres BKI_FORCE_NOT_NULL; /* conflict resolver */
+#endif
+} FormData_pg_conflict_rel;
+
+/* ----------------
+ * Form_pg_conflict_rel corresponds to a pointer to a row with
+ * the format of pg_conflict_rel relation.
+ * ----------------
+ */
+typedef FormData_pg_conflict_rel * Form_pg_conflict_rel;
+
+DECLARE_TOAST(pg_conflict_rel, 8882, 8883);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_conflict_rel_oid_index, 8884, ConflictRelOidIndexId, pg_conflict_rel, btree(oid oid_ops));
+
+DECLARE_UNIQUE_INDEX(pg_conflict_rel_type_index, 8885, ConflictRelTypeIndexId, pg_conflict_rel, btree(confrrelid oid_ops, confrtype text_ops));
+
+MAKE_SYSCACHE(CONFLICTRELOID, pg_conflict_rel_oid_index, 256);
+MAKE_SYSCACHE(CONFLICTRELTYPE, pg_conflict_rel_type_index, 256);
+
+#endif /* PG_CONFLICT_REL_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 1d41169d5e..7677cf023b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -2419,6 +2419,8 @@ typedef enum AlterTableType
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
AT_ReAddStatistics, /* internal to commands/tablecmds.c */
+ AT_SetConflictResolver, /* SET CONFLICT RESOLVER */
+ AT_ResetConflictResolver, /* RESET CONFLICT RESOLVER */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index 4e2853b5be..207c4cd9a4 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -92,4 +92,16 @@ extern ConflictResolver GetConflictResolver(TupleTableSlot *localslot,
bool *apply_remote,
LogicalRepTupleData *newtup);
extern bool CanCreateFullTuple(Relation localrel, LogicalRepTupleData *newtup);
+extern ConflictType ValidateConflictTypeAndResolver(char *conflict_type,
+ char *conflict_resolver,
+ bool isReset);
+extern void SetTableConflictResolver(Relation pg_conf_rel, Relation rel, char *conflict_type,
+ char *conflict_resolver, bool recurse,
+ bool recursing, LOCKMODE lockmode);
+extern void ResetTableConflictResolver(Relation rel, char *conflict_type,
+ bool recurse, bool recursing, LOCKMODE lockmode);
+extern void RemoveTableConflictById(Oid confid);
+extern void InheritTableConflictResolvers(Relation child_rel, Relation parent_rel);
+extern void ResetResolversInheritance(Relation rel);
+
#endif
diff --git a/src/test/regress/expected/conflict_resolver.out b/src/test/regress/expected/conflict_resolver.out
index c21486dbb4..802a7583d4 100644
--- a/src/test/regress/expected/conflict_resolver.out
+++ b/src/test/regress/expected/conflict_resolver.out
@@ -1,5 +1,8 @@
+--
+-- Test for configuration of global resolvers
+--
--check default global resolvers in system catalog
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
conftype | confres
----------------+------------------
delete_differ | last_update_wins
@@ -9,9 +12,7 @@ select * from pg_conflict order by conftype;
update_missing | apply_or_skip
(5 rows)
---
-- Test of SET/RESET CONFLICT RESOLVER with invalid names
---
SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
ERROR: aaaa is not a valid conflict type
SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
@@ -20,16 +21,14 @@ SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
ERROR: remote_apply is not a valid conflict resolver for conflict type delete_missing
RESET CONFLICT RESOLVER for 'ct'; -- fail
ERROR: ct is not a valid conflict type
---
-- Test of SET/RESET CONFLICT RESOLVER with valid names
---
SET CONFLICT RESOLVER 'error' for 'delete_missing';
SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
--check new resolvers are saved
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
conftype | confres
----------------+----------------
delete_differ | keep_local
@@ -42,7 +41,7 @@ select * from pg_conflict order by conftype;
RESET CONFLICT RESOLVER for 'delete_missing';
RESET CONFLICT RESOLVER for 'insert_exists';
--check resolvers are reset to default for delete_missing and insert_exists
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
conftype | confres
----------------+------------------
delete_differ | keep_local
@@ -52,3 +51,235 @@ select * from pg_conflict order by conftype;
update_missing | apply_or_error
(5 rows)
+--
+-- Test for configuration of table level resolvers
+--
+-- Test for ALTER TABLE..SET/RESET CONFLICT RESOLVER
+CREATE TABLE ptntable (
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+CREATE TABLE ptntable_1 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-02-01') TO ('2006-03-01');
+CREATE TABLE ptntable_2 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01')
+ PARTITION BY RANGE (logdate);
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'keep_local' ON 'insert_exists';
+--Expect 8 entries. 2 for parent and 2 for each child partition (inherited
+--resolvers)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | keep_local | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | keep_local | t
+ ptntable_2 | delete_missing | error | t
+ ptntable_2 | insert_exists | keep_local | t
+ ptntable_2_1 | delete_missing | error | t
+ ptntable_2_1 | insert_exists | keep_local | t
+(8 rows)
+
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'skip' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ';
+--For both ptntable_2 and ptntable_2_1, expect:
+--The resolver for 'delete_missing' changed from 'error' to 'skip'
+--The 'update_differ'configuration added
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | keep_local | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | keep_local | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | keep_local | t
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | keep_local | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+--Set resolvers on parent table again. It should not overwrite non inherited
+--entries for ptntable_2 and corresponding entries for ptntable_2_1
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'insert_exists';
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | t
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+--Reset of inherited entry ON child alone should result in error
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'insert_exists';
+ERROR: cannot reset inherited resolver for conflict type "insert_exists" of relation "ptntable_2"
+--Reset of non-inherited entry ON child alone should suceed
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'update_differ';
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | t
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+(8 rows)
+
+DROP TABLE ptntable_2;
+--Expect entries removed for both ptntable_2 and its child ptntable_2_1
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+(4 rows)
+
+-- Test for ALTER TABLE..SPLIT PARTITION and ALTER TABLE..MERGE PARTITIONS
+ALTER TABLE ptntable SPLIT PARTITION ptntable_1 INTO
+ (PARTITION ptntable_1_1 FOR VALUES FROM ('2006-02-01') TO ('2006-02-10'),
+ PARTITION ptntable_1_10 FOR VALUES FROM ('2006-02-10') TO ('2006-02-20'),
+ PARTITION ptntable_1_20 FOR VALUES FROM ('2006-02-20') TO ('2006-03-01'));
+--Expect split partitions to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+---------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1_1 | delete_missing | error | t
+ ptntable_1_1 | insert_exists | remote_apply | t
+ ptntable_1_10 | delete_missing | error | t
+ ptntable_1_10 | insert_exists | remote_apply | t
+ ptntable_1_20 | delete_missing | error | t
+ ptntable_1_20 | insert_exists | remote_apply | t
+(8 rows)
+
+ALTER TABLE ptntable MERGE PARTITIONS (ptntable_1_1, ptntable_1_10, ptntable_1_20)
+ INTO ptntable_1;
+--Expect merged partition to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+(4 rows)
+
+-- Test for ALTER TABLE..ATTACH PARTITION and CREATE TABLE..PARTITION OF
+CREATE TABLE ptntable_2(
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ',
+ SET CONFLICT RESOLVER 'skip' ON 'delete_missing';
+--Expect ptntable_2_1 to inherit resolvers of ptntable_2
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+--Expect 4 new entries for ptntable_2 and its child ptntable_2_1
+--Entried for ptntable_2_1 should be marked as inherited.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(8 rows)
+
+--Attach ptntable_2 to ptntable now
+ALTER TABLE ptntable ATTACH PARTITION ptntable_2
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01');
+--For both ptntable_2 and ptntable_2_1, expect:
+--The 'insert_exists' configuration inherited
+--Resolver for 'delete_missing' not overwritten
+--Resolver for 'update_differ' retained.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | t
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+-- Test for ALTER TABLE..DETACH PARTITION
+ALTER TABLE ptntable DETACH PARTITION ptntable_2;
+--All resolvers of ptntable_2 should be marked as non-inherited
+--All resolvers for ptntable_2_1's should still be marked as inherited (as
+--they are still inherited from ptntable_2)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+--------------+----------------+--------------+----------------
+ ptntable | delete_missing | error | f
+ ptntable | insert_exists | remote_apply | f
+ ptntable_1 | delete_missing | error | t
+ ptntable_1 | insert_exists | remote_apply | t
+ ptntable_2 | delete_missing | skip | f
+ ptntable_2 | insert_exists | remote_apply | f
+ ptntable_2 | update_differ | remote_apply | f
+ ptntable_2_1 | delete_missing | skip | t
+ ptntable_2_1 | insert_exists | remote_apply | t
+ ptntable_2_1 | update_differ | remote_apply | t
+(10 rows)
+
+DROP TABLE ptntable_2;
+DROP TABLE ptntable;
+--Expect no resolvers
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+ relname | confrtype | confrres | confrinherited
+---------+-----------+----------+----------------
+(0 rows)
+
diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out
index 215eb899be..8caf852600 100644
--- a/src/test/regress/expected/oidjoins.out
+++ b/src/test/regress/expected/oidjoins.out
@@ -266,3 +266,4 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid}
NOTICE: checking pg_subscription {subowner} => pg_authid {oid}
NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid}
NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid}
+NOTICE: checking pg_conflict_rel {confrrelid} => pg_class {oid}
diff --git a/src/test/regress/sql/conflict_resolver.sql b/src/test/regress/sql/conflict_resolver.sql
index f83d14b229..b4574d1df4 100644
--- a/src/test/regress/sql/conflict_resolver.sql
+++ b/src/test/regress/sql/conflict_resolver.sql
@@ -1,17 +1,19 @@
+--
+-- Test for configuration of global resolvers
+--
+
--check default global resolvers in system catalog
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
---
-- Test of SET/RESET CONFLICT RESOLVER with invalid names
---
+
SET CONFLICT RESOLVER 'keep_local' for 'aaaa'; -- fail
SET CONFLICT RESOLVER 'bbbbb' for 'delete_missing'; -- fail
SET CONFLICT RESOLVER 'remote_apply' for 'delete_missing'; -- fail
RESET CONFLICT RESOLVER for 'ct'; -- fail
---
-- Test of SET/RESET CONFLICT RESOLVER with valid names
---
+
SET CONFLICT RESOLVER 'error' for 'delete_missing';
SET CONFLICT RESOLVER 'keep_local' for 'insert_exists';
SET CONFLICT RESOLVER 'keep_local' for 'update_differ';
@@ -19,10 +21,155 @@ SET CONFLICT RESOLVER 'apply_or_error' for 'update_missing';
SET CONFLICT RESOLVER 'keep_local' for 'delete_differ';
--check new resolvers are saved
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
RESET CONFLICT RESOLVER for 'delete_missing';
RESET CONFLICT RESOLVER for 'insert_exists';
--check resolvers are reset to default for delete_missing and insert_exists
-select * from pg_conflict order by conftype;
+SELECT * FROM pg_conflict order by conftype;
+
+--
+-- Test for configuration of table level resolvers
+--
+
+
+-- Test for ALTER TABLE..SET/RESET CONFLICT RESOLVER
+
+CREATE TABLE ptntable (
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+
+CREATE TABLE ptntable_1 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-02-01') TO ('2006-03-01');
+
+CREATE TABLE ptntable_2 PARTITION OF ptntable
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01')
+ PARTITION BY RANGE (logdate);
+
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'keep_local' ON 'insert_exists';
+
+--Expect 8 entries. 2 for parent and 2 for each child partition (inherited
+--resolvers)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'skip' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ';
+
+--For both ptntable_2 and ptntable_2_1, expect:
+--The resolver for 'delete_missing' changed from 'error' to 'skip'
+--The 'update_differ'configuration added
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+--Set resolvers on parent table again. It should not overwrite non inherited
+--entries for ptntable_2 and corresponding entries for ptntable_2_1
+ALTER TABLE ptntable SET CONFLICT RESOLVER 'error' ON 'delete_missing',
+ SET CONFLICT RESOLVER 'remote_apply' ON 'insert_exists';
+
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+--Reset of inherited entry ON child alone should result in error
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'insert_exists';
+
+--Reset of non-inherited entry ON child alone should suceed
+ALTER TABLE ptntable_2 RESET CONFLICT RESOLVER ON 'update_differ';
+
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+DROP TABLE ptntable_2;
+
+--Expect entries removed for both ptntable_2 and its child ptntable_2_1
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+
+-- Test for ALTER TABLE..SPLIT PARTITION and ALTER TABLE..MERGE PARTITIONS
+
+ALTER TABLE ptntable SPLIT PARTITION ptntable_1 INTO
+ (PARTITION ptntable_1_1 FOR VALUES FROM ('2006-02-01') TO ('2006-02-10'),
+ PARTITION ptntable_1_10 FOR VALUES FROM ('2006-02-10') TO ('2006-02-20'),
+ PARTITION ptntable_1_20 FOR VALUES FROM ('2006-02-20') TO ('2006-03-01'));
+
+--Expect split partitions to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+ALTER TABLE ptntable MERGE PARTITIONS (ptntable_1_1, ptntable_1_10, ptntable_1_20)
+ INTO ptntable_1;
+
+--Expect merged partition to inherit resolvers FROM parent
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+
+-- Test for ALTER TABLE..ATTACH PARTITION and CREATE TABLE..PARTITION OF
+
+CREATE TABLE ptntable_2(
+ city_id int not null,
+ logdate date not null,
+ peaktemp int,
+ unitsales int
+) PARTITION BY RANGE (logdate);
+
+ALTER TABLE ptntable_2 SET CONFLICT RESOLVER 'remote_apply' ON 'update_differ',
+ SET CONFLICT RESOLVER 'skip' ON 'delete_missing';
+
+--Expect ptntable_2_1 to inherit resolvers of ptntable_2
+CREATE TABLE ptntable_2_1 PARTITION OF ptntable_2
+ FOR VALUES FROM ('2006-03-20') TO ('2006-04-01');
+
+--Expect 4 new entries for ptntable_2 and its child ptntable_2_1
+--Entried for ptntable_2_1 should be marked as inherited.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+--Attach ptntable_2 to ptntable now
+ALTER TABLE ptntable ATTACH PARTITION ptntable_2
+ FOR VALUES FROM ('2006-03-01') TO ('2006-04-01');
+
+--For both ptntable_2 and ptntable_2_1, expect:
+--The 'insert_exists' configuration inherited
+--Resolver for 'delete_missing' not overwritten
+--Resolver for 'update_differ' retained.
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+
+-- Test for ALTER TABLE..DETACH PARTITION
+
+ALTER TABLE ptntable DETACH PARTITION ptntable_2;
+
+--All resolvers of ptntable_2 should be marked as non-inherited
+--All resolvers for ptntable_2_1's should still be marked as inherited (as
+--they are still inherited from ptntable_2)
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
+
+DROP TABLE ptntable_2;
+DROP TABLE ptntable;
+
+--Expect no resolvers
+SELECT relname, confrtype, confrres, confrinherited FROM pg_class pc,
+ (SELECT confrrelid, confrtype, confrres, confrinherited FROM pg_conflict_rel)conf
+ WHERE conf.confrrelid= pc.oid order by relname, confrtype;
--
2.34.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
@ 2024-07-26 04:20 ` Ajin Cherian <[email protected]>
2024-07-30 04:19 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-30 10:33 ` Re: Conflict Detection and Resolution Dilip Kumar <[email protected]>
0 siblings, 2 replies; 27+ messages in thread
From: Ajin Cherian @ 2024-07-26 04:20 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Nisha Moond <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Dilip Kumar <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>
On Wed, Jul 17, 2024 at 4:01 PM shveta malik <[email protected]> wrote:
>
> Please find v6 patch-set. Changes are:
>
>
> Please find v7 patch-set, the changes are:
Patch 0001 - Reflects v5 of Conflict Detection patch in [1].
Patch 0002:
a) Removed global CONFLICT RESOLVER syntax and logic.
b) Added new syntax for creating CONFLICT RESOLVERs at the subscription
level.
Syntax for CREATE SUBSCRIPTION:
CREATE SUBSCRIPTION <subname> CONNECTION <conninfo> PUBLICATION <pubname>
CONFLICT RESOLVER
(conflict_type1 = resolver1, conflict_type2 = resolver2, conflict_type3 =
resolver3, ...);
Syntax for ALTER SUBSCRIPTION:
ALTER SUBSCRIPTION <subname> CONFLICT RESOLVER
(conflict_type1 = resolver1, conflict_type2 = resolver2, conflict_type3 =
resolver3, ...);
Patch 0003 - Supports subscription-level resolvers for conflict resolution.
Patch 0004 - Modified last_update_win related test cases to reflect the new
syntax.
Patch 0005 - Dropped for the time being; will rebase and post in the next
version.
Thanks to Shveta for design discussions and thanks to Nisha for helping in
rebasing the patch and helping in testing and stabilizing the patch by
providing comments off-list.
[1] -
https://www.postgresql.org/message-id/[email protected]....
Attachments:
[application/octet-stream] v7-0002-Add-CONFLICT-RESOLVERS-into-the-syntax-for-CREATE.patch (35.6K, ../../CAFPTHDbfEGJrL_063JC=qrxJLzN60CQZ=wv0yB8BYYBuiqa6BA@mail.gmail.com/3-v7-0002-Add-CONFLICT-RESOLVERS-into-the-syntax-for-CREATE.patch)
download | inline diff:
From e6651190f22de06689313475c04cc6f2c195f8d5 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Thu, 25 Jul 2024 00:12:40 -0400
Subject: [PATCH v7 2/4] Add CONFLICT RESOLVERS into the syntax for CREATE and
ALTER SUBSCRIPTION
This patch provides support for configuring subscriptions with conflict resolvers
Syntax for CREATE SUBSCRIPTION:
CREATE SUBSCRIPTION <subname> CONNECTION <conninfo> PUBLICATION <pubname> CONFLICT RESOLVER
(conflict_type1 = resolver1, conflict_type2 = resolver2, conflict_type3 = resolver3,...);
Syntax for ALTER SUBSCRIPTION:
ALTER SUBSCRIPTION <subname> CONFLICT RESOLVER
(conflict_type1 = resolver1, conflict_type2 = resolver2, conflict_type3 = resolver3,...);
---
src/backend/commands/subscriptioncmds.c | 110 +++++++
src/backend/parser/gram.y | 25 +-
src/backend/replication/logical/conflict.c | 310 ++++++++++++++++++
src/include/catalog/Makefile | 3 +-
src/include/catalog/meson.build | 1 +
.../catalog/pg_subscription_conflict.h | 56 ++++
src/include/nodes/parsenodes.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/replication/conflict.h | 49 +++
src/test/regress/expected/oidjoins.out | 1 +
src/test/regress/expected/subscription.out | 69 ++++
src/test/regress/sql/subscription.sql | 35 ++
12 files changed, 659 insertions(+), 4 deletions(-)
create mode 100644 src/include/catalog/pg_subscription_conflict.h
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index b2bc095c71..0e40a2226d 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -28,6 +28,7 @@
#include "catalog/pg_database_d.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
+#include "catalog/pg_subscription_conflict.h"
#include "catalog/pg_type.h"
#include "commands/dbcommands.h"
#include "commands/defrem.h"
@@ -37,6 +38,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "pgstat.h"
+#include "replication/conflict.h"
#include "replication/logicallauncher.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
@@ -452,6 +454,46 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
}
+/*
+ * Parsing function for conflict resolvers in CREATE SUBSCRIPTION command
+ */
+static void
+parse_subscription_conflict_resolvers(List *stmtresolvers,
+ ConflictTypeResolver * resolvers)
+{
+ ListCell *lc;
+ ConflictType type;
+ bool valid = false;
+
+ if (!stmtresolvers)
+ return;
+
+ foreach(lc, stmtresolvers)
+ {
+ DefElem *defel = (DefElem *) lfirst(lc);
+
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ if (strcmp(resolvers[type].conflict_type, defel->defname) == 0)
+ {
+ resolvers[type].resolver = defGetString(defel);
+ validate_conflict_type_and_resolver(resolvers[type].conflict_type,
+ resolvers[type].resolver);
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict type", defel->defname)));
+
+ /* reset the flag for the next conflict type */
+ valid = false;
+ }
+}
+
/*
* Add publication names from the list to a string.
*/
@@ -596,6 +638,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
bits32 supported_opts;
SubOpts opts = {0};
AclResult aclresult;
+ ConflictTypeResolver conflictResolvers[CT_MAX + 1];
+ bool skip_conflict_resolvers = false;
/*
* Parse and check options.
@@ -611,6 +655,33 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
+ if (!opts.detectconflict)
+ {
+ /*
+ * If conflict resolvers are set but detect_conflict is not enabled,
+ * issue warning and ignore resolvers.
+ */
+ if (stmt->resolvers)
+ ereport(WARNING,
+ (errmsg("Ignoring given CONFLICT RESOLVERS as detect_conflict is not enabled.")));
+ skip_conflict_resolvers = true;
+ }
+ else
+ {
+ /* If conflict resolvers are not set, use default values */
+ if (!stmt->resolvers)
+ {
+ ereport(WARNING,
+ (errmsg("Will use default resolvers configuration as detect_conflict is ON but resolvers are not given")));
+ }
+
+ /*
+ * Parse and check conflict resolvers. Initialize with default values
+ */
+ SetDefaultResolvers(conflictResolvers);
+ parse_subscription_conflict_resolvers(stmt->resolvers, conflictResolvers);
+ }
+
/*
* Since creating a replication slot is not transactional, rolling back
* the transaction leaves the created replication slot. So we cannot run
@@ -742,6 +813,10 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_create(originname);
+ /* Update the Conflict Resolvers in pg_subscription_conflict */
+ if (!skip_conflict_resolvers)
+ SetSubConflictResolver(subid, conflictResolvers);
+
/*
* Connect to remote side to execute requested commands and fetch table
* info.
@@ -1374,6 +1449,18 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_DETECT_CONFLICT))
{
+ ConflictTypeResolver conflictResolvers[CT_MAX + 1];
+
+ if (!opts.detectconflict)
+ RemoveSubscriptionConflictBySubid(subid);
+ else
+ {
+ ereport(WARNING,
+ (errmsg("Using default conflict resolvers")));
+ SetDefaultResolvers(conflictResolvers);
+ SetSubConflictResolver(subid, conflictResolvers);
+ }
+
values[Anum_pg_subscription_subdetectconflict - 1] =
BoolGetDatum(opts.detectconflict);
replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
@@ -1604,6 +1691,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
update_tuple = true;
break;
}
+ case ALTER_SUBSCRIPTION_CONFLICT_RESOLVERS:
+ {
+ List *conflict_resolvers = NIL;
+
+ /* make sure that detect_conflict is enabled, else throw error */
+ if (!sub->detectconflict)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set conflict resolvers when detect_conflict is not enabled")));
+
+ /* get list of conflict types and resolvers and validate them */
+ conflict_resolvers = GetAndValidateSubsConflictResolverList(stmt->resolvers);
+
+ /*
+ * Update the conflict resolvers for the corresponding
+ * conflict types in the pg_subscription_conflict catalog
+ */
+ UpdateSubConflictResolvers(conflict_resolvers, subid);
+ break;
+ }
default:
elog(ERROR, "unrecognized ALTER SUBSCRIPTION kind %d",
@@ -1855,6 +1962,9 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
/* Remove any associated relation synchronization states. */
RemoveSubscriptionRel(subid, InvalidOid);
+ /* Remove any associated conflict resolvers */
+ RemoveSubscriptionConflictBySubid(subid);
+
/* Remove the origin tracking if exists. */
ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_drop_by_name(originname, true, false);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a043fd4c66..a690b46c6a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -426,7 +426,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
OptTableElementList TableElementList OptInherit definition
OptTypedTableElementList TypedTableElementList
reloptions opt_reloptions
- OptWith opt_definition func_args func_args_list
+ OptWith opt_definition opt_resolver_definition func_args func_args_list
func_args_with_defaults func_args_with_defaults_list
aggr_args aggr_args_list
func_as createfunc_opt_list opt_createfunc_opt_list alterfunc_opt_list
@@ -772,7 +772,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
RANGE READ REAL REASSIGN RECHECK RECURSIVE REF_P REFERENCES REFERENCING
REFRESH REINDEX RELATIVE_P RELEASE RENAME REPEATABLE REPLACE REPLICA
- RESET RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
+ RESET RESOLVER RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
ROUTINE ROUTINES ROW ROWS RULE
SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT
@@ -8824,6 +8824,11 @@ opt_definition:
| /*EMPTY*/ { $$ = NIL; }
;
+opt_resolver_definition:
+ CONFLICT RESOLVER definition { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
table_func_column: param_name func_type
{
FunctionParameter *n = makeNode(FunctionParameter);
@@ -10758,7 +10763,7 @@ AlterPublicationStmt:
*****************************************************************************/
CreateSubscriptionStmt:
- CREATE SUBSCRIPTION name CONNECTION Sconst PUBLICATION name_list opt_definition
+ CREATE SUBSCRIPTION name CONNECTION Sconst PUBLICATION name_list opt_definition opt_resolver_definition
{
CreateSubscriptionStmt *n =
makeNode(CreateSubscriptionStmt);
@@ -10766,6 +10771,7 @@ CreateSubscriptionStmt:
n->conninfo = $5;
n->publication = $7;
n->options = $8;
+ n->resolvers = $9;
$$ = (Node *) n;
}
;
@@ -10872,6 +10878,17 @@ AlterSubscriptionStmt:
n->options = $5;
$$ = (Node *) n;
}
+ | ALTER SUBSCRIPTION name opt_resolver_definition
+ {
+ AlterSubscriptionStmt *n =
+ makeNode(AlterSubscriptionStmt);
+
+ n->kind = ALTER_SUBSCRIPTION_CONFLICT_RESOLVERS;
+ n->subname = $3;
+ n->resolvers = $4;
+ $$ = (Node *) n;
+ }
+
;
/*****************************************************************************
@@ -17797,6 +17814,7 @@ unreserved_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
@@ -18428,6 +18446,7 @@ bare_label_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index b7dc73cfce..104a8ba3cd 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -15,9 +15,23 @@
#include "postgres.h"
#include "access/commit_ts.h"
+#include "access/heaptoast.h"
+#include "access/heapam.h"
+#include "access/table.h"
+#include "access/tableam.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_subscription.h"
+#include "catalog/pg_subscription_conflict.h"
+#include "catalog/pg_subscription_conflict_d.h"
+#include "catalog/pg_inherits.h"
+#include "commands/defrem.h"
#include "replication/conflict.h"
#include "replication/origin.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
#include "utils/rel.h"
const char *const ConflictTypeNames[] = {
@@ -30,6 +44,55 @@ const char *const ConflictTypeNames[] = {
[CT_DELETE_DIFFER] = "delete_differ"
};
+const char *const ConflictResolverNames[] = {
+ [CR_REMOTE_APPLY] = "remote_apply",
+ [CR_KEEP_LOCAL] = "keep_local",
+ [CR_APPLY_OR_SKIP] = "apply_or_skip",
+ [CR_APPLY_OR_ERROR] = "apply_or_error",
+ [CR_SKIP] = "skip",
+ [CR_ERROR] = "error"
+};
+
+#define CONFLICT_TYPE_MAX_RESOLVERS 4
+
+/*
+ * Valid conflict resolvers for each conflict type.
+ *
+ * XXX: If we do not want to maintain different resolvers such as
+ * apply_or_skip and apply_or_error for update_missing conflict,
+ * then we can retain remote_apply and keep_local only. Then these
+ * resolvers in context of update_missing will mean:
+ *
+ * keep_local: do not apply the update as INSERT.
+ * remote_apply: apply the update as INSERT. If we could not apply,
+ * then log and skip.
+ *
+ * Similarly SKIP can be replaced with KEEP_LOCAL for both update_missing
+ * and delete_missing conflicts. For missing rows, 'SKIP' sounds more user
+ * friendly name for a resolver and thus has been added here.
+ */
+const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
+ [CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+};
+
+/*
+ * Default conflict resolver for each conflict type.
+ */
+const int ConflictTypeDefaultResolvers[] = {
+ [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
+ [CT_UPDATE_EXISTS] = CR_REMOTE_APPLY,
+ [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
+ [CT_DELETE_MISSING] = CR_SKIP,
+ [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+
+};
+
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
TransactionId localxmin,
@@ -192,3 +255,250 @@ build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
return conflict_row;
}
+
+/*
+ * Set default values for CONFLICT RESOLVERS for each conflict type
+ */
+void
+SetDefaultResolvers(ConflictTypeResolver * conflictResolvers)
+{
+ ConflictType type;
+
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ conflictResolvers[type].conflict_type = ConflictTypeNames[type];
+ conflictResolvers[type].resolver =
+ ConflictResolverNames[ConflictTypeDefaultResolvers[type]];
+ }
+}
+
+/*
+ * Validate the conflict type and resolver.
+ */
+void
+validate_conflict_type_and_resolver(const char *conflict_type,
+ const char *conflict_resolver)
+{
+ ConflictType type;
+ ConflictResolver resolver;
+ bool valid = false;
+ int i;
+
+ /* Check conflict type validity */
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ if (strcmp(ConflictTypeNames[type], conflict_type) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict type", conflict_type));
+
+ /* Reset */
+ valid = false;
+
+ /* Check conflict resolver validity. */
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_resolver) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver", conflict_resolver));
+
+ /* Reset */
+ valid = false;
+
+ /* Check if conflict resolver is a valid one for the given conflict type */
+ for (i = 0; i < CONFLICT_TYPE_MAX_RESOLVERS; i++)
+ {
+ if (ConflictTypeResolverMap[type][i] == resolver)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver for conflict type %s",
+ conflict_resolver,
+ conflict_type));
+
+}
+
+/*
+ * Extract the conflict type and conflict resolvers from the
+ * ALTER SUBSCRIPTION command and return a list of ConflictTypeResolver nodes.
+ */
+List *
+GetAndValidateSubsConflictResolverList(List *stmtresolvers)
+{
+ ListCell *lc;
+ ConflictTypeResolver *CTR = NULL;
+ List *res = NIL;
+
+ foreach(lc, stmtresolvers)
+ {
+ DefElem *defel = (DefElem *) lfirst(lc);
+
+ CTR = palloc(sizeof(ConflictTypeResolver));
+ CTR->conflict_type = defel->defname;
+ CTR->resolver = defGetString(defel);
+
+ /*
+ * Validate the conflict type and that the resolver is valid for that
+ * conflict type
+ */
+ validate_conflict_type_and_resolver(CTR->conflict_type, CTR->resolver);
+
+ res = lappend(res, CTR);
+ }
+
+ return res;
+}
+
+/*
+ * Update the Subscription's conflict resolver for a conflict type in
+ * pg_subscription_conflict system catalog
+ */
+void
+UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid)
+{
+ ListCell *lc;
+ Datum values[Natts_pg_subscription_conflict];
+ bool nulls[Natts_pg_subscription_conflict];
+ bool replaces[Natts_pg_subscription_conflict];
+ HeapTuple oldtup;
+ HeapTuple newtup = NULL;
+ Relation pg_subscription_conflict;
+
+ /* Prepare to update a tuple. */
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+ memset(values, 0, sizeof(values));
+
+ pg_subscription_conflict = table_open(SubscriptionConflictId, RowExclusiveLock);
+
+ foreach(lc, conflict_resolvers)
+ {
+ ConflictTypeResolver *CTR = (ConflictTypeResolver *) lfirst(lc);
+
+ /* set up subid and conflict_type to search in cache */
+ values[Anum_pg_subscription_conflict_confsubid - 1] = ObjectIdGetDatum(subid);
+ values[Anum_pg_subscription_conflict_confrtype - 1] = CStringGetTextDatum(CTR->conflict_type);
+
+ oldtup = SearchSysCache2(SUBSCRIPTIONCONFLICTSUBOID,
+ values[Anum_pg_subscription_conflict_confsubid - 1],
+ values[Anum_pg_subscription_conflict_confrtype - 1]);
+
+ if (HeapTupleIsValid(oldtup))
+ {
+ /* Update the new resolver */
+ values[Anum_pg_subscription_conflict_confrres - 1] = CStringGetTextDatum(CTR->resolver);
+ replaces[Anum_pg_subscription_conflict_confrres - 1] = true;
+
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(pg_subscription_conflict),
+ values, nulls, replaces);
+ CatalogTupleUpdate(pg_subscription_conflict, &oldtup->t_self, newtup);
+ ReleaseSysCache(oldtup);
+ heap_freetuple(newtup);
+ }
+ else
+ elog(ERROR, "cache lookup failed for table conflict %s for subid %u",
+ CTR->conflict_type, subid);
+
+ }
+
+ table_close(pg_subscription_conflict, RowExclusiveLock);
+}
+
+/*
+ * Set Conflict Resolvers on the subscription
+ */
+void
+SetSubConflictResolver(Oid subId, ConflictTypeResolver * resolvers)
+{
+ Relation pg_subscription_conflict;
+ Datum values[Natts_pg_subscription_conflict];
+ bool nulls[Natts_pg_subscription_conflict];
+ bool replaces[Natts_pg_subscription_conflict];
+ HeapTuple newtup = NULL;
+ ConflictType type;
+ Oid conflict_oid;
+
+ pg_subscription_conflict = table_open(SubscriptionConflictId, RowExclusiveLock);
+
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ /* Prepare to update a tuple. */
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_subscription_conflict_confsubid - 1] = ObjectIdGetDatum(subId);
+ values[Anum_pg_subscription_conflict_confrtype - 1] =
+ CStringGetTextDatum(resolvers[type].conflict_type);
+ values[Anum_pg_subscription_conflict_confrres - 1] =
+ CStringGetTextDatum(resolvers[type].resolver);
+
+ /* Get a new oid and update the tuple into catalog */
+ conflict_oid = GetNewOidWithIndex(pg_subscription_conflict, SubscriptionConflictOidIndexId,
+ Anum_pg_subscription_conflict_oid);
+ values[Anum_pg_subscription_conflict_oid - 1] = ObjectIdGetDatum(conflict_oid);
+ newtup = heap_form_tuple(RelationGetDescr(pg_subscription_conflict),
+ values, nulls);
+ CatalogTupleInsert(pg_subscription_conflict, newtup);
+ heap_freetuple(newtup);
+ }
+
+ table_close(pg_subscription_conflict, RowExclusiveLock);
+}
+
+/*
+ * Remove the subscription conflict resolvers for the subscription id
+ */
+void
+RemoveSubscriptionConflictBySubid(Oid subid)
+{
+ Relation rel;
+ HeapTuple tup;
+ TableScanDesc scan;
+ ScanKeyData skey[1];
+ int nkeys = 0;
+
+
+ rel = table_open(SubscriptionConflictId, RowExclusiveLock);
+
+ /*
+ * Search using the subid, this should return all conflict resolvers for
+ * this sub
+ */
+ ScanKeyInit(&skey[nkeys++],
+ Anum_pg_subscription_conflict_confsubid,
+ BTEqualStrategyNumber,
+ F_OIDEQ,
+ ObjectIdGetDatum(subid));
+
+ scan = table_beginscan_catalog(rel, nkeys, skey);
+
+ /* Iterate through the tuples and delete them */
+ while (HeapTupleIsValid(tup = heap_getnext(scan, ForwardScanDirection)))
+ {
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+
+ table_endscan(scan);
+ table_close(rel, RowExclusiveLock);
+}
diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile
index 167f91a6e3..f2611c1424 100644
--- a/src/include/catalog/Makefile
+++ b/src/include/catalog/Makefile
@@ -81,7 +81,8 @@ CATALOG_HEADERS := \
pg_publication_namespace.h \
pg_publication_rel.h \
pg_subscription.h \
- pg_subscription_rel.h
+ pg_subscription_rel.h \
+ pg_subscription_conflict.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h)
diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index f70d1daba5..959e1d9ded 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -69,6 +69,7 @@ catalog_headers = [
'pg_publication_rel.h',
'pg_subscription.h',
'pg_subscription_rel.h',
+ 'pg_subscription_conflict.h',
]
# The .dat files we need can just be listed alphabetically.
diff --git a/src/include/catalog/pg_subscription_conflict.h b/src/include/catalog/pg_subscription_conflict.h
new file mode 100644
index 0000000000..ad82f69288
--- /dev/null
+++ b/src/include/catalog/pg_subscription_conflict.h
@@ -0,0 +1,56 @@
+/* -------------------------------------------------------------------------
+ *
+ * pg_subscription_conflict.h
+ * definition of the "subscription conflict resolver" system
+ * catalog (pg_subscription_conflict)
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_subscription_conflict.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ * -------------------------------------------------------------------------
+ */
+#ifndef PG_SUBSCRIPTION_CONFLICT_H
+#define PG_SUBSCRIPTION_CONFLICT_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_subscription_conflict_d.h"
+
+/* ----------------
+ * pg_subscription_conflict definition. cpp turns this into
+ * typedef struct FormData_pg_subscription_conflict
+ * ----------------
+ */
+CATALOG(pg_subscription_conflict,8881,SubscriptionConflictId)
+{
+ Oid oid; /* OID of the object itself */
+ Oid confsubid BKI_LOOKUP(pg_subscription); /* OID of subscription */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text confrtype BKI_FORCE_NOT_NULL; /* conflict type */
+ text confrres BKI_FORCE_NOT_NULL; /* conflict resolver */
+#endif
+} FormData_pg_subscription_conflict;
+
+/* ----------------
+ * Form_pg_subscription_conflict corresponds to a pointer to a row with
+ * the format of pg_subscription_conflict relation.
+ * ----------------
+ */
+typedef FormData_pg_subscription_conflict * Form_pg_subscription_conflict;
+
+DECLARE_TOAST(pg_subscription_conflict, 8882, 8883);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_subscription_conflict_oid_index, 8884, SubscriptionConflictOidIndexId, pg_subscription_conflict, btree(oid oid_ops));
+
+DECLARE_UNIQUE_INDEX(pg_subscription_conflict_sub_index, 8885, SubscriptionConflictSubIndexId, pg_subscription_conflict, btree(confsubid oid_ops, confrtype text_ops));
+
+MAKE_SYSCACHE(SUBSCRIPTIONCONFLICTOID, pg_subscription_conflict_oid_index, 256);
+MAKE_SYSCACHE(SUBSCRIPTIONCONFLICTSUBOID, pg_subscription_conflict_sub_index, 256);
+
+#endif /* PG_SUBSCRIPTION_CONFLICT_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 85a62b538e..2e1eefc3a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -4203,6 +4203,7 @@ typedef struct CreateSubscriptionStmt
char *conninfo; /* Connection string to publisher */
List *publication; /* One or more publication to subscribe to */
List *options; /* List of DefElem nodes */
+ List *resolvers; /* List of conflict resolvers */
} CreateSubscriptionStmt;
typedef enum AlterSubscriptionType
@@ -4215,6 +4216,7 @@ typedef enum AlterSubscriptionType
ALTER_SUBSCRIPTION_REFRESH,
ALTER_SUBSCRIPTION_ENABLED,
ALTER_SUBSCRIPTION_SKIP,
+ ALTER_SUBSCRIPTION_CONFLICT_RESOLVERS,
} AlterSubscriptionType;
typedef struct AlterSubscriptionStmt
@@ -4225,6 +4227,7 @@ typedef struct AlterSubscriptionStmt
char *conninfo; /* Connection string to publisher */
List *publication; /* One or more publication to subscribe to */
List *options; /* List of DefElem nodes */
+ List *resolvers; /* List of conflict resolvers */
} AlterSubscriptionStmt;
typedef struct DropSubscriptionStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f7fe834cf4..21e5cb162d 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -377,6 +377,7 @@ PG_KEYWORD("repeatable", REPEATABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replace", REPLACE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replica", REPLICA, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("reset", RESET, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("resolver", RESOLVER, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restart", RESTART, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restrict", RESTRICT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("return", RETURN, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index 3a7260d3c1..91ab4908b1 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -39,6 +39,47 @@ typedef enum
CT_DELETE_DIFFER,
} ConflictType;
+/* Min and max conflict type */
+#define CT_MIN CT_INSERT_EXISTS
+#define CT_MAX CT_DELETE_DIFFER
+
+/*
+ * Conflict resolvers that can be used to resolve various conflicts.
+ *
+ * See ConflictTypeResolverMap in conflcit.c to find out which all
+ * resolvers are supported for each conflict type.
+ */
+typedef enum ConflictResolver
+{
+ /* Apply the remote change */
+ CR_REMOTE_APPLY = 1,
+
+ /* Keep the local change */
+ CR_KEEP_LOCAL,
+
+ /* Apply the remote change; skip if it can not be applied */
+ CR_APPLY_OR_SKIP,
+
+ /* Apply the remote change; emit error if it can not be applied */
+ CR_APPLY_OR_ERROR,
+
+ /* Skip applying the change */
+ CR_SKIP,
+
+ /* Error out */
+ CR_ERROR,
+} ConflictResolver;
+
+/* Min and max conflict resolver */
+#define CR_MIN CR_REMOTE_APPLY
+#define CR_MAX CR_ERROR
+
+typedef struct ConflictTypeResolver
+{
+ const char *conflict_type;
+ const char *resolver;
+} ConflictTypeResolver;
+
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
extern void ReportApplyConflict(int elevel, ConflictType type,
@@ -46,5 +87,13 @@ extern void ReportApplyConflict(int elevel, ConflictType type,
TransactionId localxmin, RepOriginId localorigin,
TimestampTz localts, TupleTableSlot *conflictslot);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
+extern void SetSubConflictResolver(Oid subId, ConflictTypeResolver * resolvers);
+extern void RemoveSubscriptionConflictById(Oid confid);
+extern void RemoveSubscriptionConflictBySubid(Oid confid);
+extern List *GetAndValidateSubsConflictResolverList(List *stmtresolvers);
+extern void UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid);
+extern void validate_conflict_type_and_resolver(const char *conflict_type,
+ const char *conflict_resolver);
+extern void SetDefaultResolvers(ConflictTypeResolver * conflictResolvers);
#endif
diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out
index 215eb899be..42bf2cce92 100644
--- a/src/test/regress/expected/oidjoins.out
+++ b/src/test/regress/expected/oidjoins.out
@@ -266,3 +266,4 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid}
NOTICE: checking pg_subscription {subowner} => pg_authid {oid}
NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid}
NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid}
+NOTICE: checking pg_subscription_conflict {confsubid} => pg_subscription {oid}
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index cc9337ce73..1bff69e1d9 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -430,6 +430,7 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
ERROR: detect_conflict requires a Boolean value
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: Will use default resolvers configuration as detect_conflict is ON but resolvers are not given
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
@@ -439,6 +440,18 @@ HINT: To initiate replication, you must manually create the replication slot, e
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | t | off | dbname=regress_doesnotexist | 0/0
(1 row)
+-- confirm that the default conflict resolvers have been set
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+----------------+---------------
+ delete_differ | remote_apply
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | remote_apply
+ update_exists | remote_apply
+ update_missing | apply_or_skip
+(6 rows)
+
ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
\dRs+
List of subscriptions
@@ -447,6 +460,62 @@ ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
+-- confirm that the conflict resolvers have been dropped
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+-----------+----------
+(0 rows)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail - invalid conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = foo);
+ERROR: foo is not a valid conflict resolver
+-- fail - invalid conflict types
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (foo = 'keep_local');
+ERROR: foo is not a valid conflict type
+-- creating subscription with detect_conflict = false should not create conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+-----------+----------
+(0 rows)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- creating subscription with detect_conflict = false but conflict resolvers specified
+-- will result in conflict resolvers being ignored
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false) CONFLICT RESOLVER (insert_exists = 'keep_local');
+WARNING: Ignoring given CONFLICT RESOLVERS as detect_conflict is not enabled.
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+-----------+----------
+(0 rows)
+
+-- setting detect_conflict to true will set default conflict resolvers
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = true);
+WARNING: Using default conflict resolvers
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+----------------+---------------
+ delete_differ | remote_apply
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | remote_apply
+ update_exists | remote_apply
+ update_missing | apply_or_skip
+(6 rows)
+
+-- fail - altering with invalid conflict type
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (foo = 'keep_local');
+ERROR: foo is not a valid conflict type
+-- fail - altering with invalid conflict resolver
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'foo');
+ERROR: foo is not a valid conflict resolver
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
-- let's do some tests with pg_create_subscription rather than superuser
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 5c740fd263..d5fb082b09 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -295,10 +295,45 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
\dRs+
+-- confirm that the default conflict resolvers have been set
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
\dRs+
+-- confirm that the conflict resolvers have been dropped
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
+-- fail - invalid conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = foo);
+
+-- fail - invalid conflict types
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (foo = 'keep_local');
+
+-- creating subscription with detect_conflict = false should not create conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false);
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
+-- creating subscription with detect_conflict = false but conflict resolvers specified
+-- will result in conflict resolvers being ignored
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false) CONFLICT RESOLVER (insert_exists = 'keep_local');
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
+-- setting detect_conflict to true will set default conflict resolvers
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = true);
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
+-- fail - altering with invalid conflict type
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (foo = 'keep_local');
+
+-- fail - altering with invalid conflict resolver
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'foo');
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
--
2.34.1
[application/octet-stream] v7-0003-Conflict-resolvers-for-insert-update-and-delete.patch (55.8K, ../../CAFPTHDbfEGJrL_063JC=qrxJLzN60CQZ=wv0yB8BYYBuiqa6BA@mail.gmail.com/4-v7-0003-Conflict-resolvers-for-insert-update-and-delete.patch)
download | inline diff:
From ac4879d814fc85bf59fd8a66e1d7b01f6d67d62c Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Thu, 25 Jul 2024 13:44:17 +0530
Subject: [PATCH v7 3/4] Conflict resolvers for insert, update, and delete
This patch introduces support for handling conflicts with the following resolutions:
- For INSERT conflicts:
- insert_exists: remote_apply, keep_local, error
- For UPDATE conflicts:
- update_differ: remote_apply, keep_local, error
- update_missing: apply_or_skip, apply_or_error, skip, error
- For DELETE conflicts:
- delete_missing: skip, error
- delete_differ: remote_apply, keep_local, error
---
src/backend/executor/execReplication.c | 56 +-
src/backend/replication/logical/conflict.c | 207 ++++++-
src/backend/replication/logical/worker.c | 366 ++++++++---
src/include/executor/executor.h | 5 +-
src/include/replication/conflict.h | 13 +-
src/test/subscription/meson.build | 1 +
src/test/subscription/t/029_on_error.pl | 9 +
.../subscription/t/034_conflict_resolver.pl | 576 ++++++++++++++++++
8 files changed, 1104 insertions(+), 129 deletions(-)
create mode 100755 src/test/subscription/t/034_conflict_resolver.pl
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 0680bc86fd..fb96bb7005 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -590,8 +590,9 @@ ReCheckConflictIndexes(ResultRelInfo *resultRelInfo, EState *estate,
TransactionId xmin;
GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
- ReportApplyConflict(ERROR, type, resultRelInfo->ri_RelationDesc, uniqueidx,
- xmin, origin, committs, conflictslot);
+ ReportApplyConflict(type, CR_ERROR, resultRelInfo->ri_RelationDesc,
+ uniqueidx, xmin, origin, committs,
+ conflictslot, false);
}
}
}
@@ -604,7 +605,8 @@ ReCheckConflictIndexes(ResultRelInfo *resultRelInfo, EState *estate,
*/
void
ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot)
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot, Oid subid)
{
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
@@ -640,11 +642,55 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
+ /*
+ * If caller has passed non null conflictslot, check all the unique
+ * indexes for potential conflicts. If the configured resolver is in
+ * favour of apply, give the conflicted tuple information in
+ * conflictslot.
+ */
+ if (conflictslot)
+ {
+ foreach_oid(uniqueidx, conflictindexes)
+ {
+ /*
+ * Return if any conflict is found other than one with 'ERROR'
+ * resolver configured. In case of 'ERROR' resolver, emit
+ * error here; otherwise return to caller for resolutions.
+ */
+ if (FindConflictTuple(resultRelInfo, estate, uniqueidx,
+ slot, &(*conflictslot)))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+ ConflictResolver resolver;
+ bool apply_remote = false;
+
+ GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
+ resolver = GetConflictResolver(rel, CT_INSERT_EXISTS,
+ &apply_remote, NULL, subid);
+
+ ReportApplyConflict(CT_INSERT_EXISTS, resolver, rel,
+ uniqueidx, xmin, origin, committs,
+ *conflictslot, apply_remote);
+
+ /* Nothing to apply, free the resources */
+ if (!apply_remote)
+ {
+ ExecDropSingleTupleTableSlot(*conflictslot);
+ *conflictslot = NULL;
+ }
+
+ return;
+ }
+ }
+ }
+
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
- conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
-
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, false,
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 104a8ba3cd..50f12a0efd 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -26,13 +26,17 @@
#include "catalog/pg_subscription_conflict_d.h"
#include "catalog/pg_inherits.h"
#include "commands/defrem.h"
+#include "executor/executor.h"
#include "replication/conflict.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "replication/origin.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/syscache.h"
#include "utils/rel.h"
+#include "utils/syscache.h"
const char *const ConflictTypeNames[] = {
[CT_INSERT_EXISTS] = "insert_exists",
@@ -94,11 +98,13 @@ const int ConflictTypeDefaultResolvers[] = {
};
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
-static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin,
+static int errdetail_apply_conflict(ConflictType type,
+ ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
RepOriginId localorigin,
TimestampTz localts,
- TupleTableSlot *conflictslot);
+ TupleTableSlot *conflictslot,
+ bool apply_remote);
/*
* Get the xmin and commit timestamp data (origin and timestamp) associated
@@ -133,22 +139,32 @@ GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
}
/*
- * Report a conflict when applying remote changes.
+ * Report conflict and resolution applied while applying remote changes.
*/
void
-ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
- Oid conflictidx, TransactionId localxmin,
- RepOriginId localorigin, TimestampTz localts,
- TupleTableSlot *conflictslot)
+ReportApplyConflict(ConflictType type, ConflictResolver resolver,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot,
+ bool apply_remote)
{
+ int elevel;
+
+ if (resolver == CR_ERROR ||
+ (resolver == CR_APPLY_OR_ERROR && !apply_remote))
+ elevel = ERROR;
+ else
+ elevel = LOG;
+
ereport(elevel,
errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
- errmsg("conflict %s detected on relation \"%s.%s\"",
+ errmsg("conflict %s detected on relation \"%s.%s\". Resolution: %s",
ConflictTypeNames[type],
get_namespace_name(RelationGetNamespace(localrel)),
- RelationGetRelationName(localrel)),
- errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
- localts, conflictslot));
+ RelationGetRelationName(localrel),
+ ConflictResolverNames[resolver]),
+ errdetail_apply_conflict(type, resolver, conflictidx, localxmin, localorigin,
+ localts, conflictslot, apply_remote));
}
/*
@@ -183,13 +199,21 @@ InitConflictIndexes(ResultRelInfo *relInfo)
}
/*
- * Add an errdetail() line showing conflict detail.
+ * Add an errdetail() line showing conflict and resolution details.
*/
static int
-errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot)
+errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot, bool apply_remote)
{
+ char *applymsg;
+
+ if (apply_remote)
+ applymsg = "applying the remote changes.";
+ else
+ applymsg = "ignoring the remote changes.";
+
switch (type)
{
case CT_INSERT_EXISTS:
@@ -203,27 +227,43 @@ errdetail_apply_conflict(ConflictType type, Oid conflictidx,
char *index_value = build_index_value_desc(conflictidx,
conflictslot);
- if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
- else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ if (resolver == CR_ERROR)
+ {
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ return errdetail("Key already exists, %s", applymsg);
}
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin,
+ timestamptz_to_str(localts), applymsg);
case CT_UPDATE_MISSING:
- return errdetail("Did not find the row to be updated.");
+ if (resolver == CR_APPLY_OR_SKIP && !apply_remote)
+ return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
+ else if (resolver == CR_APPLY_OR_ERROR && !apply_remote)
+ return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
+ else if (apply_remote)
+ return errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
+ applymsg);
+ else
+ return errdetail("Did not find the row to be updated, %s",
+ applymsg);
case CT_DELETE_MISSING:
return errdetail("Did not find the row to be deleted.");
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin,
+ timestamptz_to_str(localts), applymsg);
}
return 0; /* silence compiler warning */
@@ -339,6 +379,69 @@ validate_conflict_type_and_resolver(const char *conflict_type,
}
+/*
+ * Get the conflict resolver configured at subscription level for
+ * for the given conflict type.
+ */
+static ConflictResolver
+get_conflict_resolver_internal(ConflictType type, Oid subid)
+{
+
+ ConflictResolver resolver;
+ HeapTuple tuple;
+ Datum datum;
+ char *conflict_res;
+
+ /*
+ * XXX: Currently, we fetch the conflict resolver from cache for each
+ * conflict detection. If needed, we can keep the info in global variable
+ * and fetch from cache only once after cache invalidation.
+ */
+ tuple = SearchSysCache2(SUBSCRIPTIONCONFLICTSUBOID,
+ ObjectIdGetDatum(subid),
+ CStringGetTextDatum(ConflictTypeNames[type]));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for conflict type %u", type);
+
+ datum = SysCacheGetAttrNotNull(SUBSCRIPTIONCONFLICTSUBOID,
+ tuple, Anum_pg_subscription_conflict_confrres);
+
+ conflict_res = TextDatumGetCString(datum);
+
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_res) == 0)
+ break;
+ }
+
+ ReleaseSysCache(tuple);
+ return resolver;
+}
+
+/*
+ * Check if a full tuple can be created from the new tuple.
+ * Return true if yes, false otherwise.
+ */
+static bool
+can_create_full_tuple(Relation localrel,
+ LogicalRepTupleData *newtup)
+{
+ int i;
+ int local_att = RelationGetNumberOfAttributes(localrel);
+
+ if (newtup->ncols != local_att)
+ return false;
+
+ for (i = 0; i < newtup->ncols; i++)
+ {
+ if (newtup->colstatus[i] == LOGICALREP_COLUMN_UNCHANGED)
+ return false;
+ }
+
+ return true;
+}
+
/*
* Extract the conflict type and conflict resolvers from the
* ALTER SUBSCRIPTION command and return a list of ConflictTypeResolver nodes.
@@ -502,3 +605,47 @@ RemoveSubscriptionConflictBySubid(Oid subid)
table_endscan(scan);
table_close(rel, RowExclusiveLock);
}
+
+/*
+ * Find the resolver of the conflict type set under the given subscription.
+ *
+ * Set 'apply_remote' to true if remote tuple should be applied,
+ * false otherwise.
+ */
+ConflictResolver
+GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
+ LogicalRepTupleData *newtup, Oid subid)
+{
+ ConflictResolver resolver;
+
+ resolver = get_conflict_resolver_internal(type, subid);
+
+ switch (resolver)
+ {
+ case CR_REMOTE_APPLY:
+ *apply_remote = true;
+ break;
+ case CR_APPLY_OR_SKIP:
+ if (can_create_full_tuple(localrel, newtup))
+ *apply_remote = true;
+ else
+ *apply_remote = false;
+ break;
+ case CR_APPLY_OR_ERROR:
+ if (can_create_full_tuple(localrel, newtup))
+ *apply_remote = true;
+ else
+ *apply_remote = false;
+ break;
+ case CR_KEEP_LOCAL:
+ case CR_SKIP:
+ case CR_ERROR:
+ *apply_remote = false;
+ break;
+ default:
+ elog(ERROR, "Conflict %s is detected! Unrecogonized conflict resolution method",
+ ConflictTypeNames[type]);
+ }
+
+ return resolver;
+}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc3f80e8b0..36172b2460 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -382,7 +382,9 @@ static void send_feedback(XLogRecPtr recvpos, bool force, bool requestReply);
static void apply_handle_commit_internal(LogicalRepCommitData *commit_data);
static void apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot);
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry);
static void apply_handle_update_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
TupleTableSlot *remoteslot,
@@ -2428,10 +2430,10 @@ apply_handle_insert(StringInfo s)
/* For a partitioned table, insert the tuple into a partition. */
if (rel->localrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
apply_handle_tuple_routing(edata,
- remoteslot, NULL, CMD_INSERT);
+ remoteslot, &newtup, CMD_INSERT);
else
apply_handle_insert_internal(edata, edata->targetRelInfo,
- remoteslot);
+ remoteslot, &newtup, rel);
finish_edata(edata);
@@ -2454,9 +2456,13 @@ apply_handle_insert(StringInfo s)
static void
apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot)
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry)
{
EState *estate = edata->estate;
+ MemoryContext oldctx;
+ TupleTableSlot *conflictslot = NULL;
/* We must open indexes here. */
ExecOpenIndices(relinfo, MySubscription->detectconflict);
@@ -2466,7 +2472,36 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
- ExecSimpleRelationInsert(relinfo, estate, remoteslot);
+
+ if (MySubscription->detectconflict)
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, &conflictslot,
+ MyLogicalRepWorker->subid);
+ else
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, NULL, InvalidOid);
+
+
+ /* Apply remote tuple by converting INSERT to UPDATE */
+ if (conflictslot)
+ {
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, conflictslot, rel_entry, newtup);
+ MemoryContextSwitchTo(oldctx);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
+
+ /* Do the update */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, conflictslot,
+ remoteslot);
+
+ EvalPlanQualEnd(&epqstate);
+ ExecDropSingleTupleTableSlot(conflictslot);
+ }
/* Cleanup. */
ExecCloseIndices(relinfo);
@@ -2648,6 +2683,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
TupleTableSlot *localslot;
bool found;
MemoryContext oldctx;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, MySubscription->detectconflict);
@@ -2671,38 +2708,71 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* If conflict detection is enabled, check whether the local tuple was
- * modified by a different origin. If detected, report the conflict.
+ * modified by a different origin. If detected, report the conflict
+ * and configured resolver.
*/
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
- /* Process and store remote tuple in the slot */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
- MemoryContextSwitchTo(oldctx);
+ ReportApplyConflict(CT_UPDATE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- EvalPlanQualSetSlot(&epqstate, remoteslot);
+ /*
+ * Apply the change if configured resolver is in favor of that, else
+ * ignore the remote update.
+ */
+ if (apply_remote)
+ {
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ MemoryContextSwitchTo(oldctx);
- if (MySubscription->detectconflict)
- InitConflictIndexes(relinfo);
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
- /* Do the actual update. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
- ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
- remoteslot);
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
+
+ /* Do the actual update. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
+ remoteslot);
+ }
}
else
{
/*
- * The tuple to be updated could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be updated could not be found. Report the conflict. If
+ * the configured resolver is in favor of applying the change, convert
+ * UPDATE to INSERT and apply the change.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup,
+ MyLogicalRepWorker->subid);
+
+ ReportApplyConflict(CT_UPDATE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot, relmapentry, newtup);
+ slot_fill_defaults(relmapentry, estate, remoteslot);
+ MemoryContextSwitchTo(oldctx);
+
+ if (apply_remote)
+ apply_handle_insert_internal(edata, relinfo, remoteslot,
+ newtup, relmapentry);
+ }
}
/* Cleanup. */
@@ -2815,6 +2885,8 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
EPQState epqstate;
TupleTableSlot *localslot;
bool found;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2836,24 +2908,44 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_DELETE_DIFFER,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
+
+ ReportApplyConflict(CT_DELETE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- EvalPlanQualSetSlot(&epqstate, localslot);
+ if (apply_remote)
+ {
+ EvalPlanQualSetSlot(&epqstate, localslot);
+
+ /* Do the actual delete. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
+ ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
+ }
- /* Do the actual delete. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
- ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
}
else
{
/*
- * The tuple to be deleted could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be deleted could not be found. Based on resolver
+ * configured, either skip and log a message or emit an error.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_DELETE_MISSING,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
+
+ /* Resolver is set to skip, thus report the conflict and skip */
+ if (!apply_remote)
+ ReportApplyConflict(CT_DELETE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+ }
}
/* Cleanup. */
@@ -2986,19 +3078,21 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
}
MemoryContextSwitchTo(oldctx);
- /* Check if we can do the update or delete on the leaf partition. */
- if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ if (operation == CMD_INSERT || operation == CMD_UPDATE ||
+ operation == CMD_DELETE)
{
- part_entry = logicalrep_partition_open(relmapentry, partrel,
- attrmap);
- check_relation_updatable(part_entry);
+ part_entry = logicalrep_partition_open(relmapentry, partrel, attrmap);
+
+ /* Check if we can do the update or delete on the leaf partition */
+ if (operation != CMD_INSERT)
+ check_relation_updatable(part_entry);
}
switch (operation)
{
case CMD_INSERT:
apply_handle_insert_internal(edata, partrelinfo,
- remoteslot_part);
+ remoteslot_part, newtup, part_entry);
break;
case CMD_DELETE:
@@ -3023,6 +3117,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
RepOriginId localorigin;
TransactionId localxmin;
TimestampTz localts;
+ LogicalRepRelMapEntry *part_entry_new = NULL;
+ ConflictResolver resolver;
+ bool apply_remote = true;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3032,38 +3129,81 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
if (!found)
{
/*
- * The tuple to be updated could not be found. Do nothing
- * except for emitting a log message.
+ * The tuple to be updated could not be found. Report the
+ * conflict and resolver. And take action based on the
+ * configured resolver.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING,
- partrel, InvalidOid,
- InvalidTransactionId,
- InvalidRepOriginId, 0, NULL);
-
- return;
+ {
+ resolver = GetConflictResolver(partrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup,
+ MyLogicalRepWorker->subid);
+
+ ReportApplyConflict(CT_UPDATE_MISSING, resolver, partrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+
+ if (apply_remote)
+ {
+ /*
+ * Resolver is in favour of applying the remote
+ * changes. Prepare the slot for the INSERT.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot_part, part_entry, newtup);
+ slot_fill_defaults(part_entry, estate, remoteslot_part);
+ MemoryContextSwitchTo(oldctx);
+ }
+ }
}
+ else
+ {
+ /*
+ * The tuple to be updated is found. If conflict detection
+ * is enabled, check whether the local tuple was modified
+ * by a different origin. If detected, report and resolve
+ * the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ {
+ resolver = GetConflictResolver(partrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
- /*
- * If conflict detection is enabled, check whether the local
- * tuple was modified by a different origin. If detected,
- * report the conflict.
- */
- if (MySubscription->detectconflict &&
- GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
- localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
- InvalidOid, localxmin, localorigin,
- localts, NULL);
+ ReportApplyConflict(CT_UPDATE_DIFFER, resolver, partrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- /*
- * Apply the update to the local tuple, putting the result in
- * remoteslot_part.
- */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
- MemoryContextSwitchTo(oldctx);
+ if (apply_remote)
+ {
+ /*
+ * We can reach here in two cases:
+ *
+ * 1. If we found a tuple but conflict detection is
+ * OFF
+ *
+ * 2. If we found a tuple and resolver is in favor of
+ * applying the change when conflict detection is ON
+ *
+ * Putting the result in remoteslot_part.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
+ MemoryContextSwitchTo(oldctx);
+ }
+ else
+
+ /*
+ * apply_remote can be toggled if resolver for
+ * update_differ is set to skip. Ignore remote update.
+ */
+ return;
+
+ }
/*
* Does the updated tuple still satisfy the current
@@ -3073,27 +3213,59 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ExecPartitionCheck(partrelinfo, remoteslot_part, estate,
false))
{
- /*
- * Yes, so simply UPDATE the partition. We don't call
- * apply_handle_update_internal() here, which would
- * normally do the following work, to avoid repeating some
- * work already done above to find the local tuple in the
- * partition.
- */
- EPQState epqstate;
-
- EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(partrelinfo, false);
-
- EvalPlanQualSetSlot(&epqstate, remoteslot_part);
- TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
- ACL_UPDATE);
- ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
- localslot, remoteslot_part);
- ExecCloseIndices(partrelinfo);
- EvalPlanQualEnd(&epqstate);
+ if (found && apply_remote)
+ {
+ /*
+ * Yes, so simply UPDATE the partition. We don't call
+ * apply_handle_update_internal() here, which would
+ * normally do the following work, to avoid repeating
+ * some work already done above to find the local
+ * tuple in the partition.
+ *
+ * Do the update in cases - 1. conflict detection is
+ * OFF and found a tuple 2. conflict detection is ON,
+ * update_differ conflict is detected for the found
+ * tuple and the resolver is in favour of applying the
+ * update.
+ */
+
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+ ExecOpenIndices(partrelinfo, false);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot_part);
+ TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
+ ACL_UPDATE);
+ ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
+ localslot, remoteslot_part);
+ ExecCloseIndices(partrelinfo);
+ EvalPlanQualEnd(&epqstate);
+
+ }
+ else if (apply_remote)
+ {
+ /*
+ * Tuple is not found but update_missing resolver is
+ * in favour of applying the change as INSERT.
+ */
+ apply_handle_insert_internal(edata, partrelinfo,
+ remoteslot_part, newtup,
+ part_entry);
+ }
}
- else
+
+ /*
+ * Updated tuple doesn't satisfy the current partition's
+ * constraint.
+ *
+ * If conflict detection is OFF, proceed by always applying
+ * the update (as 'apply_remote' is by default true). If
+ * conflict detection is ON, 'apply_remote' can be OFF as well
+ * if the resolver for update_missing conflict conveys to skip
+ * the update.
+ */
+ else if (apply_remote)
{
/* Move the tuple into the new partition. */
@@ -3134,10 +3306,16 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
get_namespace_name(RelationGetNamespace(partrel_new)),
RelationGetRelationName(partrel_new));
- /* DELETE old tuple found in the old partition. */
- apply_handle_delete_internal(edata, partrelinfo,
- localslot,
- part_entry->localindexoid);
+ /*
+ * If tuple is found, delete it from old partition. We can
+ * reach this flow even for the case when the 'found' flag
+ * is false for 'update_missing' conflict and resolver is
+ * in favor of inserting the tuple.
+ */
+ if (found)
+ apply_handle_delete_internal(edata, partrelinfo,
+ localslot,
+ part_entry->localindexoid);
/* INSERT new tuple into the new partition. */
@@ -3153,19 +3331,25 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
map = ExecGetRootToChildMap(partrelinfo_new, estate);
if (map != NULL)
{
+ attrmap = map->attrMap;
remoteslot_part = execute_attr_map_slot(map->attrMap,
remoteslot,
remoteslot_part);
}
else
{
+ attrmap = NULL;
remoteslot_part = ExecCopySlot(remoteslot_part,
remoteslot);
slot_getallattrs(remoteslot);
}
MemoryContextSwitchTo(oldctx);
+ part_entry_new = logicalrep_partition_open(part_entry,
+ partrel_new,
+ attrmap);
apply_handle_insert_internal(edata, partrelinfo_new,
- remoteslot_part);
+ remoteslot_part, newtup,
+ part_entry_new);
}
}
break;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 9770752ea3..7d4a698c83 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,8 @@
#include "fmgr.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/memutils.h"
@@ -655,7 +657,8 @@ extern bool RelationFindReplTupleSeq(Relation rel, LockTupleMode lockmode,
TupleTableSlot *searchslot, TupleTableSlot *outslot);
extern void ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot);
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictsloty, Oid subid);
extern void ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
EState *estate, EPQState *epqstate,
TupleTableSlot *searchslot, TupleTableSlot *slot);
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index 91ab4908b1..8ce20c5bb4 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -12,6 +12,8 @@
#include "access/xlogdefs.h"
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/relcache.h"
#include "utils/timestamp.h"
@@ -82,10 +84,11 @@ typedef struct ConflictTypeResolver
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
-extern void ReportApplyConflict(int elevel, ConflictType type,
+extern void ReportApplyConflict(ConflictType type, ConflictResolver resolver,
Relation localrel, Oid conflictidx,
TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot);
+ TimestampTz localts, TupleTableSlot *conflictslot,
+ bool apply_remote);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
extern void SetSubConflictResolver(Oid subId, ConflictTypeResolver * resolvers);
extern void RemoveSubscriptionConflictById(Oid confid);
@@ -95,5 +98,11 @@ extern void UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid);
extern void validate_conflict_type_and_resolver(const char *conflict_type,
const char *conflict_resolver);
extern void SetDefaultResolvers(ConflictTypeResolver * conflictResolvers);
+extern ConflictResolver GetConflictResolver(Relation localrel,
+ ConflictType type,
+ bool *apply_remote,
+ LogicalRepTupleData *newtup,
+ Oid subid);
+extern bool CanCreateFullTuple(Relation localrel, LogicalRepTupleData *newtup);
#endif
diff --git a/src/test/subscription/meson.build b/src/test/subscription/meson.build
index c591cd7d61..00ade29b02 100644
--- a/src/test/subscription/meson.build
+++ b/src/test/subscription/meson.build
@@ -40,6 +40,7 @@ tests += {
't/031_column_list.pl',
't/032_subscribe_use_index.pl',
't/033_run_as_table_owner.pl',
+ 't/034_conflict_resolver.pl',
't/100_bugs.pl',
],
},
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 496a3c6cd9..e6f07fac03 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -113,6 +113,11 @@ $node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
+# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub CONFLICT RESOLVER (insert_exists=error)");
+
+
# Initial synchronization failure causes the subscription to be disabled.
$node_subscriber->poll_query_until('postgres',
"SELECT subenabled = false FROM pg_catalog.pg_subscription WHERE subname = 'sub'"
@@ -177,6 +182,10 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
+# Reset conflict resolver for 'insert_exist' conflict type to default.
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub CONFLICT RESOLVER (insert_exists=remote_apply)");
+
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
new file mode 100755
index 0000000000..58751c5c24
--- /dev/null
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -0,0 +1,576 @@
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+###############################
+# Setup
+###############################
+
+# Initialize publisher node
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf',
+ qq(max_prepared_transactions = 10));
+$node_publisher->start;
+
+# Create subscriber node with track_commit_timestamp_set
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ qq(track_commit_timestamp = on));
+$node_subscriber->start;
+
+# Create table on publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Create similar table on subscriber
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Setup logical replication
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE conf_tab");
+
+# Create the subscription
+my $appname = 'tap_sub';
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on);");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype"
+);
+is( $result, qq(delete_differ|remote_apply
+delete_missing|skip
+insert_exists|remote_apply
+update_differ|remote_apply
+update_exists|remote_apply
+update_missing|apply_or_skip),
+ "confirm that the default conflict resolvers are in place");
+
+############################################
+# Test 'remote_apply' for 'insert_exists'
+############################################
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
+
+# Create conflicting data on the publisher
+my $log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=1);");
+
+is($result, 'frompub', "remote data is kept");
+
+
+########################################
+# Test 'keep_local' for 'insert_exists'
+########################################
+
+# Change CONFLICT RESOLVER of insert_exists to keep_local
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'keep_local');"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'fromsub')");
+
+# Confirm that row is updated
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "data 2 from local is inserted");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that remote insert is ignored and the local row is kept
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "data from local is kept");
+
+###################################
+# Test 'error' for 'insert_exists'
+###################################
+
+# Change CONFLICT RESOLVER of insert_exists to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'error');"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'fromsub')");
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'frompub')");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Truncate table on subscriber to get rid of the error
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+###################################
+# Test 'skip' for 'delete_missing'
+###################################
+
+# Delete row on publisher that is not present on the subscriber and confirm that it is skipped
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=2);");
+
+# Confirm that the missing row is skipped because 'delete_missing' is set to 'skip'
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+####################################
+# Test 'error' for 'delete_missing'
+####################################
+
+# Change CONFLICT RESOLVER of delete_missing to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_missing = 'error');"
+);
+
+# Capture the log offset before performing the delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+
+# Perform the delete on the publisher
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=1);");
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate table on subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+
+#########################################
+# Test 'remote_apply' for 'delete_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote delete the local updated row
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);");
+
+is($result, '', "delete from remote is applied");
+
+#########################################
+# Test 'keep_local' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'keep_local');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'error');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);");
+
+is($result, 'frompubnew', "update from remote is kept");
+
+#########################################
+# Test 'keep_local' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_differ = 'keep_local');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_differ = 'error');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###########################################
+# Test 'apply_or_skip' for 'update_missing'
+###########################################
+
+#test the apply part
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=3);");
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the skip part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(4,'frompub',repeat('abcdefghij', 200));"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=4);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/DETAIL: Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update./,
+ $log_offset);
+
+###########################################
+# Test 'apply_or_error' for 'update_missing'
+###########################################
+
+# test the apply part
+
+#Change CONFLICT RESOLVER of update_missing to apply_or_error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_missing = 'apply_or_error');"
+);
+
+# Create new row on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (5,'frompub');");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=5);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=5);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=5);");
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the error part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(6,'frompub',repeat('abcdefghij', 200));"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=6);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=6);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###################################
+# Test 'skip' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to skip
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_missing = 'skip');"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the update does not change anything on the subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);");
+
+is($result, '', "update from remote is skipped on the subscriber");
+
+###################################
+# Test 'error' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_missing = 'error');"
+);
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnewer' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+done_testing();
--
2.34.1
[application/octet-stream] v7-0001-Detect-and-log-conflicts-in-logical-replication.patch (98.8K, ../../CAFPTHDbfEGJrL_063JC=qrxJLzN60CQZ=wv0yB8BYYBuiqa6BA@mail.gmail.com/5-v7-0001-Detect-and-log-conflicts-in-logical-replication.patch)
download | inline diff:
From 65db17b5a7857f8878d398275bbe3c14325ca5bd Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 24 Jul 2024 23:58:36 -0400
Subject: [PATCH v7 1/4] Detect and log conflicts in logical replication
This patch adds a new parameter detect_conflict for CREATE and ALTER
subscription commands. This new parameter will decide if subscription will
go for confict detection. By default, conflict detection will be off for a
subscription.
When conflict detection is enabled, additional logging is triggered in the
following conflict scenarios:
insert_exists: Inserting a row that violates a NOT DEFERRABLE unique constraint.
update_exists: The updated row value violates a NOT DEFERRABLE unique constraint.
update_differ: Updating a row that was previously modified by another origin.
update_missing: The tuple to be updated is missing.
delete_missing: The tuple to be deleted is missing.
delete_differ: Deleting a row that was previously modified by another origin.
For insert_exists conflict, the log can include origin and commit
timestamp details of the conflicting key with track_commit_timestamp
enabled.
update_differ and delete_differ conflicts can only be detected when
track_commit_timestamp is enabled.
---
doc/src/sgml/catalogs.sgml | 9 +
doc/src/sgml/logical-replication.sgml | 21 ++-
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 84 +++++++++
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 3 +-
src/backend/commands/subscriptioncmds.c | 31 +++-
src/backend/executor/execIndexing.c | 14 +-
src/backend/executor/execReplication.c | 142 +++++++++++++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/conflict.c | 194 ++++++++++++++++++++
src/backend/replication/logical/meson.build | 1 +
src/backend/replication/logical/worker.c | 84 +++++++--
src/bin/pg_dump/pg_dump.c | 17 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 6 +-
src/bin/psql/tab-complete.c | 14 +-
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/conflict.h | 50 +++++
src/test/regress/expected/subscription.out | 176 ++++++++++--------
src/test/regress/sql/subscription.sql | 15 ++
src/test/subscription/t/001_rep_changes.pl | 15 +-
src/test/subscription/t/013_partition.pl | 68 ++++---
src/test/subscription/t/029_on_error.pl | 5 +-
src/test/subscription/t/030_origin.pl | 43 +++++
src/tools/pgindent/typedefs.list | 1 +
26 files changed, 850 insertions(+), 155 deletions(-)
create mode 100644 src/backend/replication/logical/conflict.c
create mode 100644 src/include/replication/conflict.h
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b654fae1b2..b042a5a94a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8035,6 +8035,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subdetectconflict</structfield> <type>bool</type>
+ </para>
+ <para>
+ If true, the subscription is enabled for conflict detection.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subconninfo</structfield> <type>text</type>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index a23a3d57e2..d236d8530c 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1580,7 +1580,8 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
stop. This is referred to as a <firstterm>conflict</firstterm>. When
replicating <command>UPDATE</command> or <command>DELETE</command>
operations, missing data will not produce a conflict and such operations
- will simply be skipped.
+ will simply be skipped. Please refer to <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ for all the conflicts that will be logged when enabling <literal>detect_conflict</literal>.
</para>
<para>
@@ -1649,6 +1650,24 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
linkend="sql-altersubscription"><command>ALTER SUBSCRIPTION ...
SKIP</command></link>.
</para>
+
+ <para>
+ Enabling both <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ and <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ on the subscriber can provide additional details regarding conflicting
+ rows, such as their origin and commit timestamp, in case of a unique
+ constraint violation conflict:
+<screen>
+ERROR: conflict insert_exists detected on relation "public.t"
+DETAIL: Key (a)=(1) already exists in unique index "t_pkey", which was modified by origin 0 in transaction 740 at 2024-06-26 10:47:04.727375+08.
+CONTEXT: processing remote data for replication origin "pg_16389" during message type "INSERT" for replication target relation "public.t" in transaction 740, finished at 0/14F7EC0
+</screen>
+ Users can use these information to make decisions on whether to retain
+ the local change or adopt the remote alteration. For instance, the
+ origin in above log indicates that the existing row was modified by a
+ local change, users can manually perform a remote-change-win resolution
+ by deleting the local row.
+ </para>
</sect1>
<sect1 id="logical-replication-restrictions">
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index fdc648d007..dfbe25b59e 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -235,8 +235,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<link linkend="sql-createsubscription-params-with-password-required"><literal>password_required</literal></link>,
<link linkend="sql-createsubscription-params-with-run-as-owner"><literal>run_as_owner</literal></link>,
<link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>,
- <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>, and
- <link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>.
+ <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>,
+ <link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>, and
+ <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>.
Only a superuser can set <literal>password_required = false</literal>.
</para>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 740b7d9421..eb51805e81 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -428,6 +428,90 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createsubscription-params-with-detect-conflict">
+ <term><literal>detect_conflict</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the subscription is enabled for conflict detection.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ When conflict detection is enabled, additional logging is triggered
+ in the following scenarios:
+ <variablelist>
+ <varlistentry>
+ <term><literal>insert_exists</literal></term>
+ <listitem>
+ <para>
+ Inserting a row that violates a <literal>NOT DEFERRABLE</literal>
+ unique constraint. Note that to obtain the origin and commit
+ timestamp details of the conflicting key in the log, ensure that
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. In this scenario, an error will be raised until the
+ conflict is resolved manually.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_exists</literal></term>
+ <listitem>
+ <para>
+ The updated value of a row violates a <literal>NOT DEFERRABLE</literal>
+ unique constraint. Note that to obtain the origin and commit
+ timestamp details of the conflicting key in the log, ensure that
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. In this scenario, an error will be raised until the
+ conflict is resolved manually.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_differ</literal></term>
+ <listitem>
+ <para>
+ Updating a row that was previously modified by another origin.
+ Note that this conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. Currenly, the update is always applied regardless of
+ the origin of the local row.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be updated was not found. The update will simply be
+ skipped in this scenario.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be deleted was not found. The delete will simply be
+ skipped in this scenario.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_differ</literal></term>
+ <listitem>
+ <para>
+ Deleting a row that was previously modified by another origin. Note that this
+ conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. Currenly, the delete is always applied regardless of
+ the origin of the local row.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist></para>
</listitem>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..5a423f4fb0 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -72,6 +72,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->passwordrequired = subform->subpasswordrequired;
sub->runasowner = subform->subrunasowner;
sub->failover = subform->subfailover;
+ sub->detectconflict = subform->subdetectconflict;
/* Get conninfo */
datum = SysCacheGetAttrNotNull(SUBSCRIPTIONOID,
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 19cabc9a47..d084bfc48a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1356,7 +1356,8 @@ REVOKE ALL ON pg_subscription FROM public;
GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
subbinary, substream, subtwophasestate, subdisableonerr,
subpasswordrequired, subrunasowner, subfailover,
- subslotname, subsynccommit, subpublications, suborigin)
+ subdetectconflict, subslotname, subsynccommit,
+ subpublications, suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index d124bfe55c..b2bc095c71 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -71,8 +71,9 @@
#define SUBOPT_PASSWORD_REQUIRED 0x00000800
#define SUBOPT_RUN_AS_OWNER 0x00001000
#define SUBOPT_FAILOVER 0x00002000
-#define SUBOPT_LSN 0x00004000
-#define SUBOPT_ORIGIN 0x00008000
+#define SUBOPT_DETECT_CONFLICT 0x00004000
+#define SUBOPT_LSN 0x00008000
+#define SUBOPT_ORIGIN 0x00010000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -98,6 +99,7 @@ typedef struct SubOpts
bool passwordrequired;
bool runasowner;
bool failover;
+ bool detectconflict;
char *origin;
XLogRecPtr lsn;
} SubOpts;
@@ -162,6 +164,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->runasowner = false;
if (IsSet(supported_opts, SUBOPT_FAILOVER))
opts->failover = false;
+ if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT))
+ opts->detectconflict = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
@@ -307,6 +311,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_FAILOVER;
opts->failover = defGetBoolean(defel);
}
+ else if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT) &&
+ strcmp(defel->defname, "detect_conflict") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_DETECT_CONFLICT;
+ opts->detectconflict = defGetBoolean(defel);
+ }
else if (IsSet(supported_opts, SUBOPT_ORIGIN) &&
strcmp(defel->defname, "origin") == 0)
{
@@ -594,7 +607,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
SUBOPT_DISABLE_ON_ERR | SUBOPT_PASSWORD_REQUIRED |
- SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER | SUBOPT_ORIGIN);
+ SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -701,6 +715,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subpasswordrequired - 1] = BoolGetDatum(opts.passwordrequired);
values[Anum_pg_subscription_subrunasowner - 1] = BoolGetDatum(opts.runasowner);
values[Anum_pg_subscription_subfailover - 1] = BoolGetDatum(opts.failover);
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
values[Anum_pg_subscription_subconninfo - 1] =
CStringGetTextDatum(conninfo);
if (opts.slot_name)
@@ -1196,7 +1212,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
SUBOPT_DISABLE_ON_ERR |
SUBOPT_PASSWORD_REQUIRED |
SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
- SUBOPT_ORIGIN);
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1356,6 +1372,13 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
replaces[Anum_pg_subscription_subfailover - 1] = true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
+ replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9f05b3654c..ef522778a2 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -207,8 +207,9 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
ii = BuildIndexInfo(indexDesc);
/*
- * If the indexes are to be used for speculative insertion, add extra
- * information required by unique index entries.
+ * If the indexes are to be used for speculative insertion or conflict
+ * detection in logical replication, add extra information required by
+ * unique index entries.
*/
if (speculative && ii->ii_Unique)
BuildSpeculativeIndexInfo(indexDesc, ii);
@@ -521,6 +522,11 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* possible that a conflicting tuple is inserted immediately
* after this returns. But this can be used for a pre-check
* before insertion.
+ *
+ * If the 'slot' holds a tuple with valid tid, this tuple will
+ * be ignored when checking conflict. This can help in scenarios
+ * where we want to re-check for conflicts after inserting a
+ * tuple.
* ----------------------------------------------------------------
*/
bool
@@ -536,11 +542,9 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
ExprContext *econtext;
Datum values[INDEX_MAX_KEYS];
bool isnull[INDEX_MAX_KEYS];
- ItemPointerData invalidItemPtr;
bool checkedIndex = false;
ItemPointerSetInvalid(conflictTid);
- ItemPointerSetInvalid(&invalidItemPtr);
/*
* Get information from the result relation info structure.
@@ -629,7 +633,7 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
satisfiesConstraint =
check_exclusion_or_unique_constraint(heapRelation, indexRelation,
- indexInfo, &invalidItemPtr,
+ indexInfo, &slot->tts_tid,
values, isnull, estate, false,
CEOUC_WAIT, true,
conflictTid);
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index d0a89cd577..0680bc86fd 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -23,6 +23,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "replication/conflict.h"
#include "replication/logicalrelation.h"
#include "storage/lmgr.h"
#include "utils/builtins.h"
@@ -480,6 +481,121 @@ retry:
return found;
}
+/*
+ * Find the tuple that violates the passed in unique index constraint
+ * (conflictindex).
+ *
+ * Return false if there is no conflict and *conflictslot is set to NULL.
+ * Otherwise return true, and the conflicting tuple is locked and returned
+ * in *conflictslot.
+ */
+static bool
+FindConflictTuple(ResultRelInfo *resultRelInfo, EState *estate,
+ Oid conflictindex, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot)
+{
+ Relation rel = resultRelInfo->ri_RelationDesc;
+ ItemPointerData conflictTid;
+ TM_FailureData tmfd;
+ TM_Result res;
+
+ *conflictslot = NULL;
+
+retry:
+ if (ExecCheckIndexConstraints(resultRelInfo, slot, estate,
+ &conflictTid, list_make1_oid(conflictindex)))
+ {
+ if (*conflictslot)
+ ExecDropSingleTupleTableSlot(*conflictslot);
+
+ *conflictslot = NULL;
+ return false;
+ }
+
+ *conflictslot = table_slot_create(rel, NULL);
+
+ PushActiveSnapshot(GetLatestSnapshot());
+
+ res = table_tuple_lock(rel, &conflictTid, GetLatestSnapshot(),
+ *conflictslot,
+ GetCurrentCommandId(false),
+ LockTupleShare,
+ LockWaitBlock,
+ 0 /* don't follow updates */ ,
+ &tmfd);
+
+ PopActiveSnapshot();
+
+ switch (res)
+ {
+ case TM_Ok:
+ break;
+ case TM_Updated:
+ /* XXX: Improve handling here */
+ if (ItemPointerIndicatesMovedPartitions(&tmfd.ctid))
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("tuple to be locked was already moved to another partition due to concurrent update, retrying")));
+ else
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent update, retrying")));
+ goto retry;
+ case TM_Deleted:
+ /* XXX: Improve handling here */
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent delete, retrying")));
+ goto retry;
+ case TM_Invisible:
+ elog(ERROR, "attempted to lock invisible tuple");
+ break;
+ default:
+ elog(ERROR, "unexpected table_tuple_lock status: %u", res);
+ break;
+ }
+
+ return true;
+}
+
+/*
+ * Re-check all the unique indexes in 'recheckIndexes' to see if there are
+ * potential conflicts with the tuple in 'slot'.
+ *
+ * This function is invoked after inserting or updating a tuple that detected
+ * potential conflict tuples. It attempts to find the tuple that conflicts with
+ * the provided tuple. This operation may seem redundant with the unique
+ * violation check of indexam, but since we call this function only when we are
+ * detecting conflict in logical replication and encountering potential
+ * conflicts with any unique index constraints (which should not be frequent),
+ * so it's ok. Moreover, upon detecting a conflict, we will report an ERROR and
+ * restart the logical replication, so the additional cost of finding the tuple
+ * should be acceptable.
+ */
+static void
+ReCheckConflictIndexes(ResultRelInfo *resultRelInfo, EState *estate,
+ ConflictType type, List *recheckIndexes,
+ TupleTableSlot *slot)
+{
+ /* Re-check all the unique indexes for potential conflicts */
+ foreach_oid(uniqueidx, resultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ TupleTableSlot *conflictslot;
+
+ if (list_member_oid(recheckIndexes, uniqueidx) &&
+ FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &conflictslot))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+
+ GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
+ ReportApplyConflict(ERROR, type, resultRelInfo->ri_RelationDesc, uniqueidx,
+ xmin, origin, committs, conflictslot);
+ }
+ }
+}
+
/*
* Insert tuple represented in the slot to the relation, update the indexes,
* and execute any constraints and per-row triggers.
@@ -509,6 +625,8 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
+ List *conflictindexes;
+ bool conflict = false;
/* Compute stored generated columns */
if (rel->rd_att->constr &&
@@ -525,10 +643,17 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
- slot, estate, false, false,
- NULL, NIL, false);
+ slot, estate, false,
+ conflictindexes, &conflict,
+ conflictindexes, false);
+
+ if (conflict)
+ ReCheckConflictIndexes(resultRelInfo, estate, CT_INSERT_EXISTS,
+ recheckIndexes, slot);
/* AFTER ROW INSERT Triggers */
ExecARInsertTriggers(estate, resultRelInfo, slot,
@@ -577,6 +702,8 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
{
List *recheckIndexes = NIL;
TU_UpdateIndexes update_indexes;
+ List *conflictindexes;
+ bool conflict = false;
/* Compute stored generated columns */
if (rel->rd_att->constr &&
@@ -593,12 +720,19 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
&update_indexes);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
if (resultRelInfo->ri_NumIndices > 0 && (update_indexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
- slot, estate, true, false,
- NULL, NIL,
+ slot, estate, true,
+ conflictindexes,
+ &conflict, conflictindexes,
(update_indexes == TU_Summarizing));
+ if (conflict)
+ ReCheckConflictIndexes(resultRelInfo, estate, CT_UPDATE_EXISTS,
+ recheckIndexes, slot);
+
/* AFTER ROW UPDATE Triggers */
ExecARUpdateTriggers(estate, resultRelInfo,
NULL, NULL,
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index ba03eeff1c..1e08bbbd4e 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
applyparallelworker.o \
+ conflict.o \
decode.o \
launcher.o \
logical.o \
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
new file mode 100644
index 0000000000..b7dc73cfce
--- /dev/null
+++ b/src/backend/replication/logical/conflict.c
@@ -0,0 +1,194 @@
+/*-------------------------------------------------------------------------
+ * conflict.c
+ * Functionality for detecting and logging conflicts.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/conflict.c
+ *
+ * This file contains the code for detecting and logging conflicts on
+ * the subscriber during logical replication.
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/commit_ts.h"
+#include "replication/conflict.h"
+#include "replication/origin.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+
+const char *const ConflictTypeNames[] = {
+ [CT_INSERT_EXISTS] = "insert_exists",
+ [CT_UPDATE_EXISTS] = "update_exists",
+ [CT_UPDATE_DIFFER] = "update_differ",
+ [CT_UPDATE_MISSING] = "update_missing",
+ [CT_UPDATE_MISSING] = "update_missing",
+ [CT_DELETE_MISSING] = "delete_missing",
+ [CT_DELETE_DIFFER] = "delete_differ"
+};
+
+static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
+static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin,
+ RepOriginId localorigin,
+ TimestampTz localts,
+ TupleTableSlot *conflictslot);
+
+/*
+ * Get the xmin and commit timestamp data (origin and timestamp) associated
+ * with the provided local tuple.
+ *
+ * Return true if the commit timestamp data was found, false otherwise.
+ */
+bool
+GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts)
+{
+ Datum xminDatum;
+ bool isnull;
+
+ xminDatum = slot_getsysattr(localslot, MinTransactionIdAttributeNumber,
+ &isnull);
+ *xmin = DatumGetTransactionId(xminDatum);
+ Assert(!isnull);
+
+ /*
+ * The commit timestamp data is not available if track_commit_timestamp is
+ * disabled.
+ */
+ if (!track_commit_timestamp)
+ {
+ *localorigin = InvalidRepOriginId;
+ *localts = 0;
+ return false;
+ }
+
+ return TransactionIdGetCommitTsData(*xmin, localts, localorigin);
+}
+
+/*
+ * Report a conflict when applying remote changes.
+ */
+void
+ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot)
+{
+ ereport(elevel,
+ errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
+ errmsg("conflict %s detected on relation \"%s.%s\"",
+ ConflictTypeNames[type],
+ get_namespace_name(RelationGetNamespace(localrel)),
+ RelationGetRelationName(localrel)),
+ errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
+ localts, conflictslot));
+}
+
+/*
+ * Find all unique indexes to check for a conflict and store them into
+ * ResultRelInfo.
+ */
+void
+InitConflictIndexes(ResultRelInfo *relInfo)
+{
+ List *uniqueIndexes = NIL;
+
+ for (int i = 0; i < relInfo->ri_NumIndices; i++)
+ {
+ Relation indexRelation = relInfo->ri_IndexRelationDescs[i];
+
+ if (indexRelation == NULL)
+ continue;
+
+ /* Detect conflict only for unique indexes */
+ if (!relInfo->ri_IndexRelationInfo[i]->ii_Unique)
+ continue;
+
+ /* Don't support conflict detection for deferrable index */
+ if (!indexRelation->rd_index->indimmediate)
+ continue;
+
+ uniqueIndexes = lappend_oid(uniqueIndexes,
+ RelationGetRelid(indexRelation));
+ }
+
+ relInfo->ri_onConflictArbiterIndexes = uniqueIndexes;
+}
+
+/*
+ * Add an errdetail() line showing conflict detail.
+ */
+static int
+errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot)
+{
+ switch (type)
+ {
+ case CT_INSERT_EXISTS:
+ case CT_UPDATE_EXISTS:
+ {
+ /*
+ * Bulid the index value string. If the return value is NULL,
+ * it indicates that the current user lacks permissions to
+ * view all the columns involved.
+ */
+ char *index_value = build_index_value_desc(conflictidx,
+ conflictslot);
+
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
+ case CT_UPDATE_DIFFER:
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ case CT_UPDATE_MISSING:
+ return errdetail("Did not find the row to be updated.");
+ case CT_DELETE_MISSING:
+ return errdetail("Did not find the row to be deleted.");
+ case CT_DELETE_DIFFER:
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ }
+
+ return 0; /* silence compiler warning */
+}
+
+/*
+ * Helper functions to construct a string describing the contents of an index
+ * entry. See BuildIndexValueDescription for details.
+ */
+static char *
+build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
+{
+ char *conflict_row;
+ Relation indexDesc;
+
+ if (!conflictslot)
+ return NULL;
+
+ /* Assume the index has been locked */
+ indexDesc = index_open(indexoid, NoLock);
+
+ slot_getallattrs(conflictslot);
+
+ conflict_row = BuildIndexValueDescription(indexDesc,
+ conflictslot->tts_values,
+ conflictslot->tts_isnull);
+
+ index_close(indexDesc, NoLock);
+
+ return conflict_row;
+}
diff --git a/src/backend/replication/logical/meson.build b/src/backend/replication/logical/meson.build
index 3dec36a6de..3d36249d8a 100644
--- a/src/backend/replication/logical/meson.build
+++ b/src/backend/replication/logical/meson.build
@@ -2,6 +2,7 @@
backend_sources += files(
'applyparallelworker.c',
+ 'conflict.c',
'decode.c',
'launcher.c',
'logical.c',
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index ec96b5fe85..fc3f80e8b0 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -167,6 +167,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
+#include "replication/conflict.h"
#include "replication/logicallauncher.h"
#include "replication/logicalproto.h"
#include "replication/logicalrelation.h"
@@ -2458,7 +2459,10 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
EState *estate = edata->estate;
/* We must open indexes here. */
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, MySubscription->detectconflict);
+
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
@@ -2646,7 +2650,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
MemoryContext oldctx;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, MySubscription->detectconflict);
found = FindReplTupleInLocalRel(edata, localrel,
&relmapentry->remoterel,
@@ -2661,6 +2665,20 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
slot_modify_data(remoteslot, localslot, relmapentry, newtup);
@@ -2668,6 +2686,9 @@ apply_handle_update_internal(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot);
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
+
/* Do the actual update. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
@@ -2678,13 +2699,10 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2807,6 +2825,20 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/* If found delete it. */
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
EvalPlanQualSetSlot(&epqstate, localslot);
/* Do the actual delete. */
@@ -2818,13 +2850,10 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/*
* The tuple to be deleted could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be deleted "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2991,6 +3020,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ResultRelInfo *partrelinfo_new;
Relation partrel_new;
bool found;
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3002,16 +3034,28 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing
* except for emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation's partition \"%s\"",
- RelationGetRelationName(partrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING,
+ partrel, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL);
+
return;
}
+ /*
+ * If conflict detection is enabled, check whether the local
+ * tuple was modified by a different origin. If detected,
+ * report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
+ InvalidOid, localxmin, localorigin,
+ localts, NULL);
+
/*
* Apply the update to the local tuple, putting the result in
* remoteslot_part.
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b8b1888bd3..829f9b3e88 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4760,6 +4760,7 @@ getSubscriptions(Archive *fout)
int i_suboriginremotelsn;
int i_subenabled;
int i_subfailover;
+ int i_subdetectconflict;
int i,
ntups;
@@ -4832,11 +4833,17 @@ getSubscriptions(Archive *fout)
if (fout->remoteVersion >= 170000)
appendPQExpBufferStr(query,
- " s.subfailover\n");
+ " s.subfailover,\n");
else
appendPQExpBuffer(query,
- " false AS subfailover\n");
+ " false AS subfailover,\n");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ " s.subdetectconflict\n");
+ else
+ appendPQExpBuffer(query,
+ " false AS subdetectconflict\n");
appendPQExpBufferStr(query,
"FROM pg_subscription s\n");
@@ -4875,6 +4882,7 @@ getSubscriptions(Archive *fout)
i_suboriginremotelsn = PQfnumber(res, "suboriginremotelsn");
i_subenabled = PQfnumber(res, "subenabled");
i_subfailover = PQfnumber(res, "subfailover");
+ i_subdetectconflict = PQfnumber(res, "subdetectconflict");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4921,6 +4929,8 @@ getSubscriptions(Archive *fout)
pg_strdup(PQgetvalue(res, i, i_subenabled));
subinfo[i].subfailover =
pg_strdup(PQgetvalue(res, i, i_subfailover));
+ subinfo[i].subdetectconflict =
+ pg_strdup(PQgetvalue(res, i, i_subdetectconflict));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -5161,6 +5171,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subfailover, "t") == 0)
appendPQExpBufferStr(query, ", failover = true");
+ if (strcmp(subinfo->subdetectconflict, "t") == 0)
+ appendPQExpBufferStr(query, ", detect_conflict = true");
+
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 4b2e5870a9..bbd7cbeff6 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -671,6 +671,7 @@ typedef struct _SubscriptionInfo
char *suborigin;
char *suboriginremotelsn;
char *subfailover;
+ char *subdetectconflict;
} SubscriptionInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 7c9a1f234c..fef1ad0d70 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6539,7 +6539,7 @@ describeSubscriptions(const char *pattern, bool verbose)
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
false, false, false, false, false, false, false, false, false, false,
- false};
+ false, false};
if (pset.sversion < 100000)
{
@@ -6607,6 +6607,10 @@ describeSubscriptions(const char *pattern, bool verbose)
appendPQExpBuffer(&buf,
", subfailover AS \"%s\"\n",
gettext_noop("Failover"));
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ ", subdetectconflict AS \"%s\"\n",
+ gettext_noop("Detect conflict"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 891face1b6..086e135f65 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1946,9 +1946,10 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit", "two_phase");
+ COMPLETE_WITH("binary", "detect_conflict", "disable_on_error",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit", "two_phase");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
COMPLETE_WITH("lsn");
@@ -3363,9 +3364,10 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit", "two_phase");
+ "detect_conflict", "disable_on_error", "enabled",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..17daf11dc7 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -98,6 +98,9 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool subdetectconflict; /* True if replication should perform
+ * conflict detection */
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* Connection string to the publisher */
text subconninfo BKI_FORCE_NOT_NULL;
@@ -151,6 +154,7 @@ typedef struct Subscription
* (i.e. the main slot and the table sync
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool detectconflict; /* True if conflict detection is enabled */
char *conninfo; /* Connection string to the publisher */
char *slotname; /* Name of the replication slot */
char *synccommit; /* Synchronous commit setting for worker */
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
new file mode 100644
index 0000000000..3a7260d3c1
--- /dev/null
+++ b/src/include/replication/conflict.h
@@ -0,0 +1,50 @@
+/*-------------------------------------------------------------------------
+ * conflict.h
+ * Exports for conflict detection and log
+ *
+ * Copyright (c) 2012-2024, PostgreSQL Global Development Group
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef CONFLICT_H
+#define CONFLICT_H
+
+#include "access/xlogdefs.h"
+#include "executor/tuptable.h"
+#include "nodes/execnodes.h"
+#include "utils/relcache.h"
+#include "utils/timestamp.h"
+
+/*
+ * Conflict types that could be encountered when applying remote changes.
+ */
+typedef enum
+{
+ /* The row to be inserted violates unique constraint */
+ CT_INSERT_EXISTS,
+
+ /* The updated row value violates unique constraint */
+ CT_UPDATE_EXISTS,
+
+ /* The row to be updated was modified by a different origin */
+ CT_UPDATE_DIFFER,
+
+ /* The row to be updated is missing */
+ CT_UPDATE_MISSING,
+
+ /* The row to be deleted is missing */
+ CT_DELETE_MISSING,
+
+ /* The row to be deleted was modified by a different origin */
+ CT_DELETE_DIFFER,
+} ConflictType;
+
+extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts);
+extern void ReportApplyConflict(int elevel, ConflictType type,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot);
+extern void InitConflictIndexes(ResultRelInfo *relInfo);
+
+#endif
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 17d48b1685..cc9337ce73 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -116,18 +116,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -145,10 +145,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -157,10 +157,10 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = 'newname');
ALTER SUBSCRIPTION regress_testsub SET (password_required = false);
ALTER SUBSCRIPTION regress_testsub SET (run_as_owner = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (password_required = true);
@@ -176,10 +176,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -188,10 +188,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -223,10 +223,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -255,19 +255,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -279,27 +279,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -314,10 +314,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -332,10 +332,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -371,19 +371,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- we can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -393,10 +393,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -409,18 +409,42 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+ERROR: detect_conflict requires a Boolean value
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | t | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 007c9e7037..5c740fd263 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -287,6 +287,21 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
-- let's do some tests with pg_create_subscription rather than superuser
SET SESSION AUTHORIZATION regress_subscription_user3;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 471e981962..78c0307165 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -331,11 +331,12 @@ is( $result, qq(1|bar
2|baz),
'update works with REPLICA IDENTITY FULL and a primary key');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber->append_conf('postgresql.conf', "log_min_messages = debug1");
-$node_subscriber->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub SET (detect_conflict = true)"
+);
$node_subscriber->safe_psql('postgres', "DELETE FROM tab_full_pk");
@@ -352,10 +353,10 @@ $node_publisher->wait_for_catchup('tap_sub');
my $logfile = slurp_file($node_subscriber->logfile, $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation "tab_full_pk"/,
+ qr/conflict update_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be updated./m,
'update target row is missing');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab_full_pk"/,
+ qr/conflict delete_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be deleted./m,
'delete target row is missing');
$node_subscriber->append_conf('postgresql.conf',
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 29580525a9..7a66a06b51 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -343,12 +343,12 @@ $result =
$node_subscriber2->safe_psql('postgres', "SELECT a FROM tab1 ORDER BY 1");
is($result, qq(), 'truncate of tab1 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = true)"
+);
$node_publisher->safe_psql('postgres',
"INSERT INTO tab1 VALUES (1, 'foo'), (4, 'bar'), (10, 'baz')");
@@ -372,21 +372,21 @@ $node_publisher->wait_for_catchup('sub2');
my $logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab1_2_2"/,
+ qr/conflict update_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_1"/,
+ qr/conflict delete_missing detected on relation "public.tab1_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_2_2"/,
+ qr/conflict delete_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_def"/,
+ qr/conflict delete_missing detected on relation "public.tab1_def".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_def');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = false)"
+);
# Tests for replication using root table identity and schema
@@ -773,12 +773,12 @@ pub_tab2|3|yyy
pub_tab2|5|zzz
xxx_c|6|aaa), 'inserts into tab2 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = true)"
+);
$node_subscriber1->safe_psql('postgres', "DELETE FROM tab2");
@@ -796,15 +796,35 @@ $node_publisher->wait_for_catchup('sub2');
$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab2_1"/,
+ qr/conflict update_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab2_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab2_1"/,
+ qr/conflict delete_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab2_1');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+# Enable the track_commit_timestamp to detect the conflict when attempting
+# to update a row that was previously modified by a different origin.
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = on');
+$node_subscriber1->restart;
+
+$node_subscriber1->safe_psql('postgres', "INSERT INTO tab2 VALUES (3, 'yyy')");
+$node_publisher->safe_psql('postgres',
+ "UPDATE tab2 SET b = 'quux' WHERE a = 3");
+
+$node_publisher->wait_for_catchup('sub_viaroot');
+
+$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
+ok( $logfile =~
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/,
+ 'updating a tuple that was modified by a different origin');
+
+# The remaining tests no longer test conflict detection.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = false)"
+);
+
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_subscriber1->restart;
# Test that replication continues to work correctly after altering the
# partition of a partitioned target table.
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 0ab57a4b5b..496a3c6cd9 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -30,7 +30,7 @@ sub test_skip_lsn
# ERROR with its CONTEXT when retrieving this information.
my $contents = slurp_file($node_subscriber->logfile, $offset);
$contents =~
- qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
or die "could not get error-LSN";
my $lsn = $1;
@@ -83,6 +83,7 @@ $node_subscriber->append_conf(
'postgresql.conf',
qq[
max_prepared_transactions = 10
+track_commit_timestamp = on
]);
$node_subscriber->start;
@@ -109,7 +110,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
# Initial synchronization failure causes the subscription to be disabled.
diff --git a/src/test/subscription/t/030_origin.pl b/src/test/subscription/t/030_origin.pl
index 056561f008..8c929c07c7 100644
--- a/src/test/subscription/t/030_origin.pl
+++ b/src/test/subscription/t/030_origin.pl
@@ -27,9 +27,14 @@ my $stderr;
my $node_A = PostgreSQL::Test::Cluster->new('node_A');
$node_A->init(allows_streaming => 'logical');
$node_A->start;
+
# node_B
my $node_B = PostgreSQL::Test::Cluster->new('node_B');
$node_B->init(allows_streaming => 'logical');
+
+# Enable the track_commit_timestamp to detect the conflict when attempting to
+# update a row that was previously modified by a different origin.
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = on');
$node_B->start;
# Create table on node_A
@@ -139,6 +144,44 @@ is($result, qq(),
'Remote data originating from another node (not the publisher) is not replicated when origin parameter is none'
);
+###############################################################################
+# Check that the conflict can be detected when attempting to update or
+# delete a row that was previously modified by a different source.
+###############################################################################
+
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = true);
+ DELETE FROM tab;");
+
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (32);");
+
+# The update should update the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "UPDATE tab SET a = 33 WHERE a = 32;");
+
+$node_B->wait_for_log(
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_B->safe_psql('postgres', "DELETE FROM tab;");
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (33);");
+
+# The delete should remove the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "DELETE FROM tab WHERE a = 33;");
+
+$node_B->wait_for_log(
+ qr/Deleting a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_A->wait_for_catchup($subname_BA);
+$node_B->wait_for_catchup($subname_AB);
+
+# The remaining tests no longer test conflict detection.
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = false);");
+
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_B->restart;
+
###############################################################################
# Specifying origin = NONE indicates that the publisher should only replicate the
# changes that are generated locally from node_B, but in this case since the
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b4d7f9217c..2098ed7467 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -467,6 +467,7 @@ ConditionVariableMinimallyPadded
ConditionalStack
ConfigData
ConfigVariable
+ConflictType
ConnCacheEntry
ConnCacheKey
ConnParams
--
2.34.1
[application/octet-stream] v7-0004-Manage-Clock-skew-and-implement-last_update_wins.patch (49.2K, ../../CAFPTHDbfEGJrL_063JC=qrxJLzN60CQZ=wv0yB8BYYBuiqa6BA@mail.gmail.com/6-v7-0004-Manage-Clock-skew-and-implement-last_update_wins.patch)
download | inline diff:
From 576b65f0e775f048ae7afddf70fa77bec5bdb399 Mon Sep 17 00:00:00 2001
From: Nisha Moond <[email protected]>
Date: Thu, 25 Jul 2024 13:56:38 +0530
Subject: [PATCH v7 4/4] Manage Clock skew and implement last_update_wins
This patch attempts to manage clock skew between nodes by
introducing two new GUCs:
a) max_logical_rep_clock_skew
b) max_logical_rep_clock_skew_action
c) max_logical_rep_clock_skew_wait
If the timestamp of the currently replayed transaction is in the future
compared to the current time on the subscriber and the difference is
larger than 'max_logical_rep_clock_skew', then the action configured
in 'max_logical_rep_clock_skew_action' is performed by the apply worker.
If user configures 'wait' in 'max_logical_rep_clock_skew_action' and
actual clock skew is large while 'max_logical_rep_clock_skew' is small,
the apply worker may have to wait for a longer period to manage the clock
skew. To control this maximum wait time, a new GUC,
'max_logical_rep_clock_skew_wait', is provided. This allows the user to
set a cap on how long the apply worker should wait. If the computed wait
time exceeds this value, the apply worker will error out without waiting.
This patch also implements last_update_wins resolver.
Since conflict resolution for two phase commit transactions
using prepare-timestamp can result in data divergence, this patch
also restricts enabling two_phase and detect_conflict together
for a subscription.
---
src/backend/commands/subscriptioncmds.c | 45 +++++
src/backend/executor/execReplication.c | 2 +-
.../replication/logical/applyparallelworker.c | 26 ++-
src/backend/replication/logical/conflict.c | 146 +++++++++++++----
src/backend/replication/logical/origin.c | 1 +
src/backend/replication/logical/worker.c | 155 ++++++++++++++++--
.../utils/activity/wait_event_names.txt | 1 +
src/backend/utils/misc/guc_tables.c | 40 +++++
src/backend/utils/misc/postgresql.conf.sample | 9 +-
src/include/replication/conflict.h | 6 +-
src/include/replication/logicalworker.h | 18 ++
src/include/replication/origin.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/include/utils/timestamp.h | 1 +
src/test/regress/expected/subscription.out | 28 ++--
src/test/subscription/t/029_on_error.pl | 53 ++++--
.../subscription/t/034_conflict_resolver.pl | 116 +++++++++++--
src/tools/pgindent/typedefs.list | 1 +
18 files changed, 566 insertions(+), 85 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 0e40a2226d..8f9c474431 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -14,6 +14,7 @@
#include "postgres.h"
+#include "access/commit_ts.h"
#include "access/htup_details.h"
#include "access/table.h"
#include "access/twophase.h"
@@ -452,6 +453,22 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * Time based conflict resolution for two phase transactions can result in
+ * data divergence, so disallow enabling both together.
+ */
+ if (opts->detectconflict &&
+ IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ if (opts->twophase &&
+ IsSet(opts->specified_opts, SUBOPT_TWOPHASE_COMMIT))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ /*- translator: both %s are strings of the form "option = value" */
+ errmsg("%s and %s are mutually exclusive options",
+ "detect_conflict = true", "two_phase = true")));
+ }
}
/*
@@ -733,6 +750,13 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
elog(WARNING, "subscriptions created by regression test cases should have names starting with \"regress_\"");
#endif
+ /* Warn if detect_conflict is enabled and track_commit_timestamp is off */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
+
rel = table_open(SubscriptionRelationId, RowExclusiveLock);
/* Check if name is used */
@@ -1464,6 +1488,27 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
values[Anum_pg_subscription_subdetectconflict - 1] =
BoolGetDatum(opts.detectconflict);
replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+
+ /*
+ * Time based conflict resolution for two phase
+ * transactions can result in data divergence, so disallow
+ * enabling it when two_phase is enabled.
+ */
+ if (sub->twophasestate == LOGICALREP_TWOPHASE_STATE_ENABLED)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for a subscription that has two_phase enabled",
+ "detect_conflict")));
+
+ /*
+ * Warn if detect_conflict is enabled and
+ * track_commit_timestamp is off.
+ */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
}
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index fb96bb7005..93ba364319 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -669,7 +669,7 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
bool apply_remote = false;
GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
- resolver = GetConflictResolver(rel, CT_INSERT_EXISTS,
+ resolver = GetConflictResolver(*conflictslot, rel, CT_INSERT_EXISTS,
&apply_remote, NULL, subid);
ReportApplyConflict(CT_INSERT_EXISTS, resolver, rel,
diff --git a/src/backend/replication/logical/applyparallelworker.c b/src/backend/replication/logical/applyparallelworker.c
index e7f7d4c5e4..10c7ca99df 100644
--- a/src/backend/replication/logical/applyparallelworker.c
+++ b/src/backend/replication/logical/applyparallelworker.c
@@ -312,6 +312,20 @@ pa_can_start(void)
if (!AllTablesyncsReady())
return false;
+ /*
+ * Don't start a new parallel worker if user has either configured max
+ * clock skew or if conflict detection and resolution is ON. In both cases
+ * we need commit timestamp in the beginning.
+ *
+ * XXX: For conflict reolution case, see if we can reduce the scope of
+ * this restriction to only such cases where time-based resolvers are
+ * actually being used.
+ */
+ if ((max_logical_rep_clock_skew > LR_CLOCK_SKEW_DEFAULT) ||
+ MySubscription->detectconflict)
+ return false;
+
+
return true;
}
@@ -696,9 +710,19 @@ pa_process_spooled_messages_if_required(void)
}
else if (fileset_state == FS_READY)
{
+ /*
+ * Currently we do not support starting parallel apply worker when
+ * either clock skew is configured or conflict resolution is
+ * configured, thus it is okay to pass 0 as origin-timestamp here.
+ *
+ * XXX: If in future, we support starting pa worker even with conflict
+ * detection enabled, then here we need to pass remote's
+ * commit/prepare/abort timestamp; we can get that info from leader
+ * worker in shared memory.
+ */
apply_spooled_messages(&MyParallelShared->fileset,
MyParallelShared->xid,
- InvalidXLogRecPtr);
+ InvalidXLogRecPtr, 0);
pa_set_fileset_state(MyParallelShared, FS_EMPTY);
}
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 50f12a0efd..91252da578 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -51,6 +51,7 @@ const char *const ConflictTypeNames[] = {
const char *const ConflictResolverNames[] = {
[CR_REMOTE_APPLY] = "remote_apply",
[CR_KEEP_LOCAL] = "keep_local",
+ [CR_LAST_UPDATE_WINS] = "last_update_wins",
[CR_APPLY_OR_SKIP] = "apply_or_skip",
[CR_APPLY_OR_ERROR] = "apply_or_error",
[CR_SKIP] = "skip",
@@ -76,24 +77,24 @@ const char *const ConflictResolverNames[] = {
* friendly name for a resolver and thus has been added here.
*/
const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
- [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
- [CT_UPDATE_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
- [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
+ [CT_UPDATE_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
[CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
[CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
- [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR}
};
/*
* Default conflict resolver for each conflict type.
*/
const int ConflictTypeDefaultResolvers[] = {
- [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
- [CT_UPDATE_EXISTS] = CR_REMOTE_APPLY,
- [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_INSERT_EXISTS] = CR_LAST_UPDATE_WINS,
+ [CT_UPDATE_EXISTS] = CR_LAST_UPDATE_WINS,
+ [CT_UPDATE_DIFFER] = CR_LAST_UPDATE_WINS,
[CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
[CT_DELETE_MISSING] = CR_SKIP,
- [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+ [CT_DELETE_DIFFER] = CR_LAST_UPDATE_WINS
};
@@ -208,6 +209,12 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
TupleTableSlot *conflictslot, bool apply_remote)
{
char *applymsg;
+ int errdet = 0;
+ char *local_ts;
+ char *remote_ts;
+
+ local_ts = pstrdup(timestamptz_to_str(localts));
+ remote_ts = pstrdup(timestamptz_to_str(replorigin_session_origin_timestamp));
if (apply_remote)
applymsg = "applying the remote changes.";
@@ -230,43 +237,63 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
if (resolver == CR_ERROR)
{
if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
+ errdet = errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx),
+ localorigin, localxmin, local_ts);
else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ errdet = errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ errdet = errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
}
+ else if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Key already exists, %s. The local tuple : origin=%u, timestamp=%s; The remote tuple : origin=%u, timestamp=%s.",
+ applymsg, localorigin, local_ts,
+ replorigin_session_origin, remote_ts);
else
- return errdetail("Key already exists, %s", applymsg);
+ errdet = errdetail("Key already exists, %s", applymsg);
}
+ break;
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
- localorigin, localxmin,
- timestamptz_to_str(localts), applymsg);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, local_ts, applymsg,
+ replorigin_session_origin, remote_ts);
+ else
+ errdet = errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin, local_ts, applymsg);
+ break;
case CT_UPDATE_MISSING:
if (resolver == CR_APPLY_OR_SKIP && !apply_remote)
- return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
+ errdet = errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
else if (resolver == CR_APPLY_OR_ERROR && !apply_remote)
- return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
+ errdet = errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
else if (apply_remote)
- return errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
- applymsg);
+ errdet = errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
+ applymsg);
else
- return errdetail("Did not find the row to be updated, %s",
- applymsg);
+ errdet = errdetail("Did not find the row to be updated, %s",
+ applymsg);
+ break;
case CT_DELETE_MISSING:
- return errdetail("Did not find the row to be deleted.");
+ errdet = errdetail("Did not find the row to be deleted.");
+ break;
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
- localorigin, localxmin,
- timestamptz_to_str(localts), applymsg);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, local_ts, applymsg,
+ replorigin_session_origin, remote_ts);
+ else
+ errdet = errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin, local_ts, applymsg);
+ break;
}
- return 0; /* silence compiler warning */
+ pfree(local_ts);
+ pfree(remote_ts);
+
+ return errdet;
}
/*
@@ -377,6 +404,15 @@ validate_conflict_type_and_resolver(const char *conflict_type,
conflict_resolver,
conflict_type));
+ if ((resolver == CR_LAST_UPDATE_WINS) && !track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ conflict_resolver, "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+
+
}
/*
@@ -419,6 +455,42 @@ get_conflict_resolver_internal(ConflictType type, Oid subid)
return resolver;
}
+/*
+ * Compare the timestamps of given local tuple and the remote tuple to
+ * resolve the conflict.
+ *
+ * Returns true if remote tuple has the latest timestamp, false otherwise.
+ */
+static bool
+resolve_by_timestamp(TupleTableSlot *localslot)
+{
+ TransactionId local_xmin;
+ TimestampTz local_ts;
+ RepOriginId local_origin;
+ int ts_cmp;
+ uint64 local_system_identifier;
+
+ /* Get origin and timestamp info of the local tuple */
+ GetTupleCommitTs(localslot, &local_xmin, &local_origin, &local_ts);
+
+ /* Compare the timestamps of remote & local tuple to decide the winner */
+ ts_cmp = timestamptz_cmp_internal(replorigin_session_origin_timestamp,
+ local_ts);
+
+ if (ts_cmp == 0)
+ {
+ elog(LOG, "Timestamps of remote and local tuple are equal, comparing remote and local system identifiers");
+
+ /* Get current system's identifier */
+ local_system_identifier = GetSystemIdentifier();
+
+ return local_system_identifier <= replorigin_session_origin_sysid;
+ }
+ else
+ return (ts_cmp > 0);
+
+}
+
/*
* Check if a full tuple can be created from the new tuple.
* Return true if yes, false otherwise.
@@ -613,7 +685,8 @@ RemoveSubscriptionConflictBySubid(Oid subid)
* false otherwise.
*/
ConflictResolver
-GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
+GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
+ ConflictType type, bool *apply_remote,
LogicalRepTupleData *newtup, Oid subid)
{
ConflictResolver resolver;
@@ -622,6 +695,17 @@ GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
switch (resolver)
{
+ case CR_LAST_UPDATE_WINS:
+ if (!track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ ConflictResolverNames[resolver], "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+ else
+ *apply_remote = resolve_by_timestamp(localslot);
+ break;
case CR_REMOTE_APPLY:
*apply_remote = true;
break;
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 419e4814f0..bd8e6f0024 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -155,6 +155,7 @@ typedef struct ReplicationStateCtl
RepOriginId replorigin_session_origin = InvalidRepOriginId; /* assumed identity */
XLogRecPtr replorigin_session_origin_lsn = InvalidXLogRecPtr;
TimestampTz replorigin_session_origin_timestamp = 0;
+uint64 replorigin_session_origin_sysid = 0;
/*
* Base address into a shared memory array of replication states of size
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 36172b2460..5b3b89ebc4 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -318,6 +318,20 @@ static uint32 parallel_stream_nchanges = 0;
/* Are we initializing an apply worker? */
bool InitializingApplyWorker = false;
+/*
+ * GUC support
+ */
+const struct config_enum_entry logical_rep_clock_skew_action_options[] = {
+ {"error", LR_CLOCK_SKEW_ACTION_ERROR, false},
+ {"wait", LR_CLOCK_SKEW_ACTION_WAIT, false},
+ {NULL, 0, false}
+};
+
+/* GUCs */
+int max_logical_rep_clock_skew = LR_CLOCK_SKEW_DEFAULT;
+int max_logical_rep_clock_skew_action = LR_CLOCK_SKEW_ACTION_ERROR;
+int max_logical_rep_clock_skew_wait = 300; /* 5 mins */
+
/*
* We enable skipping all data modification changes (INSERT, UPDATE, etc.) for
* the subscription if the remote transaction's finish LSN matches the subskiplsn.
@@ -984,6 +998,95 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
ExecStoreVirtualTuple(slot);
}
+/*
+ * Manage clock skew between nodes.
+ *
+ * It checks if the remote timestamp is ahead of the local clock
+ * and if the difference exceeds max_logical_rep_clock_skew, it performs
+ * the action specified by the max_logical_rep_clock_skew_action.
+ */
+static void
+manage_clock_skew(TimestampTz origin_timestamp)
+{
+ TimestampTz current;
+ TimestampTz delayUntil;
+ long msecs;
+ int rc;
+
+ /* nothing to do if no max clock skew configured */
+ if (max_logical_rep_clock_skew == LR_CLOCK_SKEW_DEFAULT)
+ return;
+
+ current = GetCurrentTimestamp();
+
+ /*
+ * If the timestamp of the currently replayed transaction is in the future
+ * compared to the current time on the subscriber and the difference is
+ * larger than max_logical_rep_clock_skew, then perform the action
+ * specified by the max_logical_rep_clock_skew_action setting.
+ */
+ if (origin_timestamp > current &&
+ TimestampDifferenceExceeds(current, origin_timestamp,
+ max_logical_rep_clock_skew * 1000))
+ {
+ if (max_logical_rep_clock_skew_action == LR_CLOCK_SKEW_ACTION_ERROR)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg_internal("clock skew exceeds max_logical_rep_clock_skew (%d seconds)",
+ max_logical_rep_clock_skew)));
+
+ /* Perform the wait */
+ while (true)
+ {
+ delayUntil =
+ TimestampTzMinusSeconds(origin_timestamp,
+ max_logical_rep_clock_skew);
+
+ /* Exit without waiting if it's already past 'delayUntil' time */
+ msecs = TimestampDifferenceMilliseconds(GetCurrentTimestamp(),
+ delayUntil);
+ if (msecs <= 0)
+ break;
+
+ /* The wait time should not exceed max_logical_rep_clock_skew_wait */
+ if (msecs > (max_logical_rep_clock_skew_wait * 1000L))
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg_internal("clock skew wait time exceeds max_logical_rep_clock_skew_wait (%d seconds)",
+ max_logical_rep_clock_skew_wait)));
+
+ elog(DEBUG2, "delaying apply for %ld milliseconds to manage clock skew",
+ msecs);
+
+ /* Sleep until we are signaled or msecs have elapsed */
+ rc = WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ msecs,
+ WAIT_EVENT_LOGICAL_CLOCK_SKEW);
+
+ /* Exit the loop if msecs have elapsed */
+ if (rc & WL_TIMEOUT)
+ break;
+
+ if (rc & WL_LATCH_SET)
+ {
+ ResetLatch(MyLatch);
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ /*
+ * This might change max_logical_rep_clock_skew and
+ * max_logical_rep_clock_skew_wait.
+ */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+ }
+ }
+}
+
/*
* Handle BEGIN message.
*/
@@ -1005,6 +1108,15 @@ apply_handle_begin(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.committime);
+
+ /*
+ * Capture the commit timestamp of the remote transaction for time based
+ * conflict resolution purpose.
+ */
+ replorigin_session_origin_timestamp = begin_data.committime;
}
/*
@@ -1062,6 +1174,9 @@ apply_handle_begin_prepare(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.prepare_time);
}
/*
@@ -1302,7 +1417,8 @@ apply_handle_stream_prepare(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset,
- prepare_data.xid, prepare_data.prepare_lsn);
+ prepare_data.xid, prepare_data.prepare_lsn,
+ prepare_data.prepare_time);
/* Mark the transaction as prepared. */
apply_handle_prepare_internal(&prepare_data);
@@ -1999,7 +2115,8 @@ ensure_last_message(FileSet *stream_fileset, TransactionId xid, int fileno,
*/
void
apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn)
+ XLogRecPtr lsn,
+ TimestampTz origin_timestamp)
{
int nchanges;
char path[MAXPGPATH];
@@ -2052,6 +2169,16 @@ apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
end_replication_step();
+ /*
+ * If origin_timestamp is provided by caller, then check clock skew with
+ * respect to the passed time and take configured action.
+ */
+ if (origin_timestamp)
+ manage_clock_skew(origin_timestamp);
+
+ /* Capture the timestamp (prepare or commit) of the remote transaction */
+ replorigin_session_origin_timestamp = origin_timestamp;
+
/*
* Read the entries one by one and pass them through the same logic as in
* apply_dispatch.
@@ -2157,7 +2284,8 @@ apply_handle_stream_commit(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset, xid,
- commit_data.commit_lsn);
+ commit_data.commit_lsn,
+ commit_data.committime);
apply_handle_commit_internal(&commit_data);
@@ -2715,7 +2843,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(localrel, CT_UPDATE_DIFFER,
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_DIFFER,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -2755,7 +2883,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(localrel, CT_UPDATE_MISSING,
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_MISSING,
&apply_remote, newtup,
MyLogicalRepWorker->subid);
@@ -2909,7 +3037,7 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(localrel, CT_DELETE_DIFFER,
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_DIFFER,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -2936,7 +3064,7 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(localrel, CT_DELETE_MISSING,
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_MISSING,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -3135,7 +3263,7 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(partrel, CT_UPDATE_MISSING,
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_MISSING,
&apply_remote, newtup,
MyLogicalRepWorker->subid);
@@ -3168,7 +3296,7 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(partrel, CT_UPDATE_DIFFER,
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_DIFFER,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -4671,6 +4799,7 @@ run_apply_worker()
TimeLineID startpointTLI;
char *err;
bool must_use_password;
+ char *replorigin_sysid;
slotname = MySubscription->slotname;
@@ -4711,10 +4840,12 @@ run_apply_worker()
MySubscription->name, err)));
/*
- * We don't really use the output identify_system for anything but it does
- * some initializations on the upstream so let's still call it.
+ * Call identify_system to do some initializations on the upstream and
+ * store the output as system identifier of the replication origin node.
*/
- (void) walrcv_identify_system(LogRepWorkerWalRcvConn, &startpointTLI);
+ replorigin_sysid = walrcv_identify_system(LogRepWorkerWalRcvConn,
+ &startpointTLI);
+ replorigin_session_origin_sysid = strtoul(replorigin_sysid, NULL, 10);
set_apply_error_context_origin(originname);
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index db37beeaae..a51f82169e 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -59,6 +59,7 @@ CHECKPOINTER_MAIN "Waiting in main loop of checkpointer process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_CLOCK_SKEW "Waiting in apply-begin of logical replication apply process to bring clock skew in permissible range."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot sync worker."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f6fcdebb03..c8c8a991e2 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -68,6 +68,7 @@
#include "postmaster/walsummarizer.h"
#include "postmaster/walwriter.h"
#include "replication/logicallauncher.h"
+#include "replication/logicalworker.h"
#include "replication/slot.h"
#include "replication/slotsync.h"
#include "replication/syncrep.h"
@@ -482,6 +483,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry wal_sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry logical_rep_clock_skew_action_options[];
/*
* GUC option variables that are exported from this module
@@ -3639,6 +3641,33 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets maximum clock skew tolerance between logical "
+ "replication nodes beyond which action configured "
+ "in max_logical_rep_clock_skew_action is triggered."),
+ gettext_noop("-1 turns this check off."),
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew,
+ LR_CLOCK_SKEW_DEFAULT, LR_CLOCK_SKEW_DEFAULT, INT_MAX,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"max_logical_rep_clock_skew_wait", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets max limit on how long apply worker shall wait to "
+ "bring clock skew within permissible range of max_logical_rep_clock_skew. "
+ "If the computed wait time is more than this value, "
+ "apply worker will error out without waiting."),
+ gettext_noop("0 turns this limit off."),
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew_wait,
+ 300, 0, 3600,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
@@ -4905,6 +4934,17 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew_action", PGC_POSTMASTER, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the action to perform if a clock skew higher "
+ "than max_logical_rep_clock_skew is detected."),
+ NULL
+ },
+ &max_logical_rep_clock_skew_action,
+ LR_CLOCK_SKEW_ACTION_ERROR, logical_rep_clock_skew_action_options,
+ NULL, NULL, NULL
+ },
+
{
{"track_functions", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects function-level statistics on database activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 9ec9f97e92..f7a664a538 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -383,7 +383,14 @@
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
-
+#max_logical_rep_clock_skew = -1 # maximum clock skew tolerance between logical
+ # replication nodes beyond which action configured in
+ # 'max_logical_rep_clock_skew_action' is triggered.
+#max_logical_rep_clock_skew_action = error # error or wait
+ # (change requires restart)
+#max_logical_rep_clock_skew_wait = 300 # max limit on how long apply worker
+ # shall wait to bring clock skew within permissible
+ # range of max_logical_rep_clock_skew.
#------------------------------------------------------------------------------
# QUERY TUNING
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index 8ce20c5bb4..5aecefda8a 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -59,6 +59,9 @@ typedef enum ConflictResolver
/* Keep the local change */
CR_KEEP_LOCAL,
+ /* Apply the change with latest timestamp */
+ CR_LAST_UPDATE_WINS,
+
/* Apply the remote change; skip if it can not be applied */
CR_APPLY_OR_SKIP,
@@ -98,7 +101,8 @@ extern void UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid);
extern void validate_conflict_type_and_resolver(const char *conflict_type,
const char *conflict_resolver);
extern void SetDefaultResolvers(ConflictTypeResolver * conflictResolvers);
-extern ConflictResolver GetConflictResolver(Relation localrel,
+extern ConflictResolver GetConflictResolver(TupleTableSlot *localslot,
+ Relation localrel,
ConflictType type,
bool *apply_remote,
LogicalRepTupleData *newtup,
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index a18d79d1b2..7cb03062ac 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,7 +14,25 @@
#include <signal.h>
+/*
+ * The default for max_logical_rep_clock_skew is -1, which means ignore clock
+ * skew (the check is turned off).
+ */
+#define LR_CLOCK_SKEW_DEFAULT -1
+
+/*
+ * Worker Clock Skew Action.
+ */
+typedef enum
+{
+ LR_CLOCK_SKEW_ACTION_ERROR,
+ LR_CLOCK_SKEW_ACTION_WAIT,
+} LogicalRepClockSkewAction;
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
+extern PGDLLIMPORT int max_logical_rep_clock_skew;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_action;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_wait;
extern void ApplyWorkerMain(Datum main_arg);
extern void ParallelApplyWorkerMain(Datum main_arg);
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 7189ba9e76..dcbbbdf6ea 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -36,6 +36,7 @@ typedef struct xl_replorigin_drop
extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+extern PGDLLIMPORT uint64 replorigin_session_origin_sysid;
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 9646261d7e..95b2a5286d 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -268,7 +268,7 @@ extern void stream_stop_internal(TransactionId xid);
/* Common streaming function to apply all the spooled messages */
extern void apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn);
+ XLogRecPtr lsn, TimestampTz origin_timestamp);
extern void apply_dispatch(StringInfo s);
diff --git a/src/include/utils/timestamp.h b/src/include/utils/timestamp.h
index a6ce03ed46..53b828d89d 100644
--- a/src/include/utils/timestamp.h
+++ b/src/include/utils/timestamp.h
@@ -84,6 +84,7 @@ IntervalPGetDatum(const Interval *X)
/* Macros for doing timestamp arithmetic without assuming timestamp's units */
#define TimestampTzPlusMilliseconds(tz,ms) ((tz) + ((ms) * (int64) 1000))
#define TimestampTzPlusSeconds(tz,s) ((tz) + ((s) * (int64) 1000000))
+#define TimestampTzMinusSeconds(tz,s) ((tz) - ((s) * (int64) 1000000))
/* Set at postmaster start */
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 1bff69e1d9..ac15c30cdd 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -431,6 +431,8 @@ ERROR: detect_conflict requires a Boolean value
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
WARNING: Will use default resolvers configuration as detect_conflict is ON but resolvers are not given
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
@@ -442,13 +444,13 @@ HINT: To initiate replication, you must manually create the replication slot, e
-- confirm that the default conflict resolvers have been set
SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
- confrtype | confrres
-----------------+---------------
- delete_differ | remote_apply
+ confrtype | confrres
+----------------+------------------
+ delete_differ | last_update_wins
delete_missing | skip
- insert_exists | remote_apply
- update_differ | remote_apply
- update_exists | remote_apply
+ insert_exists | last_update_wins
+ update_differ | last_update_wins
+ update_exists | last_update_wins
update_missing | apply_or_skip
(6 rows)
@@ -499,14 +501,16 @@ SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
-- setting detect_conflict to true will set default conflict resolvers
ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = true);
WARNING: Using default conflict resolvers
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
- confrtype | confrres
-----------------+---------------
- delete_differ | remote_apply
+ confrtype | confrres
+----------------+------------------
+ delete_differ | last_update_wins
delete_missing | skip
- insert_exists | remote_apply
- update_differ | remote_apply
- update_exists | remote_apply
+ insert_exists | last_update_wins
+ update_differ | last_update_wins
+ update_exists | last_update_wins
update_missing | apply_or_skip
(6 rows)
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index e6f07fac03..9620a4bb1a 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -18,7 +18,8 @@ my $offset = 0;
# on the publisher.
sub test_skip_lsn
{
- my ($node_publisher, $node_subscriber, $nonconflict_data, $expected, $msg)
+ my ($node_publisher, $node_subscriber, $nonconflict_data, $expected,
+ $msg, $conflict_detection)
= @_;
# Wait until a conflict occurs on the subscriber.
@@ -26,13 +27,25 @@ sub test_skip_lsn
"SELECT subenabled = FALSE FROM pg_subscription WHERE subname = 'sub'"
);
+ my $lsn;
+ my $contents = slurp_file($node_subscriber->logfile, $offset);
+
# Get the finish LSN of the error transaction, mapping the expected
# ERROR with its CONTEXT when retrieving this information.
- my $contents = slurp_file($node_subscriber->logfile, $offset);
- $contents =~
- qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
- or die "could not get error-LSN";
- my $lsn = $1;
+ if ($conflict_detection)
+ {
+ $contents =~
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ or die "could not get error-LSN";
+ $lsn = $1;
+ }
+ else
+ {
+ $contents =~
+ qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ or die "could not get error-LSN";
+ $lsn = $1;
+ }
# Set skip lsn.
$node_subscriber->safe_psql('postgres',
@@ -110,7 +123,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, detect_conflict = on)"
);
# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
@@ -148,7 +161,22 @@ INSERT INTO tbl VALUES (1, NULL);
COMMIT;
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(2, NULL)", "2", "test skipping transaction");
+ "(2, NULL)", "2", "test skipping transaction", 1);
+
+# Cleanup before we start PREPARE AND COMMIT PREPARED tests
+$node_subscriber->safe_psql('postgres', "TRUNCATE tbl");
+$node_publisher->safe_psql('postgres', "TRUNCATE tbl");
+
+# Drop subscription and recreate with two_phase enabled
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+);
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'sub');
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tbl VALUES (1, NULL)");
# Test for PREPARE and COMMIT PREPARED. Insert the same data to tbl and
# PREPARE the transaction, raising an error. Then skip the transaction.
@@ -161,7 +189,7 @@ PREPARE TRANSACTION 'gtx';
COMMIT PREPARED 'gtx';
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(3, NULL)", "3", "test skipping prepare and commit prepared ");
+ "(2, NULL)", "2", "test skipping prepare and commit prepared ", 0);
# Test for STREAM COMMIT. Insert enough rows to tbl to exceed the 64kB
# limit, also raising an error on the subscriber during applying spooled
@@ -174,17 +202,14 @@ INSERT INTO tbl SELECT i, sha256(i::text::bytea) FROM generate_series(1, 10000)
COMMIT;
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(4, sha256(4::text::bytea))",
- "4", "test skipping stream-commit");
+ "(3, sha256(4::text::bytea))",
+ "3", "test skipping stream-commit", 0);
$result = $node_subscriber->safe_psql('postgres',
"SELECT COUNT(*) FROM pg_prepared_xacts");
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
-# Reset conflict resolver for 'insert_exist' conflict type to default.
-$node_subscriber->safe_psql('postgres',
- "ALTER SUBSCRIPTION sub CONFLICT RESOLVER (insert_exists=remote_apply)");
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
index 58751c5c24..e9618094d0 100755
--- a/src/test/subscription/t/034_conflict_resolver.pl
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -54,11 +54,11 @@ $node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
my $result = $node_subscriber->safe_psql('postgres',
"SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype"
);
-is( $result, qq(delete_differ|remote_apply
+is( $result, qq(delete_differ|last_update_wins
delete_missing|skip
-insert_exists|remote_apply
-update_differ|remote_apply
-update_exists|remote_apply
+insert_exists|last_update_wins
+update_differ|last_update_wins
+update_exists|last_update_wins
update_missing|apply_or_skip),
"confirm that the default conflict resolvers are in place");
@@ -66,6 +66,11 @@ update_missing|apply_or_skip),
# Test 'remote_apply' for 'insert_exists'
############################################
+# Change CONFLICT RESOLVER of insert_exists to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'remote_apply');"
+);
+
# Create local data on the subscriber
$node_subscriber->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
@@ -148,6 +153,34 @@ $node_subscriber->wait_for_log(
# Truncate table on subscriber to get rid of the error
$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+############################################
+# Test 'last_update_wins' for 'insert_exists'
+############################################
+
+# Change CONFLICT RESOLVER of insert_exists to last_update_wins
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'last_update_wins');"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (4,'fromsub')");
+
+# Create conflicting data on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (4,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=4);");
+
+is($result, 'frompub', "remote data wins");
+
###################################
# Test 'skip' for 'delete_missing'
###################################
@@ -202,16 +235,49 @@ $node_subscriber->safe_psql(
$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
-#########################################
-# Test 'remote_apply' for 'delete_differ'
-#########################################
+#############################################
+# Test 'last_update_wins' for 'delete_differ'
+#############################################
+
+# Change CONFLICT RESOLVER of delete_differ to last_update_wins
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'last_update_wins');"
+);
# Insert data in the publisher
$node_publisher->safe_psql(
'postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
- INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (4,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=4);");
+
+# Create a conflicting delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote delete the local updated row
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=4);");
+
+is($result, '', "delete from remote wins");
+
+#########################################
+# Test 'remote_apply' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'remote_apply');"
+);
# Modify data on the subscriber
$node_subscriber->safe_psql('postgres',
@@ -299,9 +365,9 @@ $node_subscriber->safe_psql(
# Wait for initial table sync to finish
$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
-#########################################
-# Test 'remote_apply' for 'update_differ'
-#########################################
+#############################################
+# Test 'last_update_wins' for 'update_differ'
+#############################################
# Insert data in the publisher
$node_publisher->safe_psql(
@@ -329,6 +395,34 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, 'frompubnew', "update from remote is kept");
+#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_differ = 'remote_apply');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub2' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew2' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);");
+
+is($result, 'frompubnew2', "update from remote is kept");
+
#########################################
# Test 'keep_local' for 'update_differ'
#########################################
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2098ed7467..d42f32dab4 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1565,6 +1565,7 @@ LogicalOutputPluginWriterPrepareWrite
LogicalOutputPluginWriterUpdateProgress
LogicalOutputPluginWriterWrite
LogicalRepBeginData
+LogicalRepClockSkewAction
LogicalRepCommitData
LogicalRepCommitPreparedTxnData
LogicalRepCtxStruct
--
2.34.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
@ 2024-07-30 04:19 ` shveta malik <[email protected]>
2024-07-31 11:24 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
1 sibling, 1 reply; 27+ messages in thread
From: shveta malik @ 2024-07-30 04:19 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: Nisha Moond <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Dilip Kumar <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>; shveta malik <[email protected]>
On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian <[email protected]> wrote:
>>
> Please find v7 patch-set, the changes are:
>
Thanks Ajin for working on this. Please find few comments:
1)
parse_subscription_conflict_resolvers():
Here we loop in this function to find the given conflict type in the
supported list and error out if conflict-type is not valid. Also we
call validate_conflict_type_and_resolver() which again validates
conflict-type. I would recommend to loop 'stmtresolvers' in parse
function and then read each type and resolver and pass that to
validate_conflict_type_and_resolver(). Avoid double validation.
2)
SetSubConflictResolver():
It works well, but it does not look apt that the 'resolvers' passed to
this function by the caller is an array and this function knows the
array range and traverse from CT_MIN to CT_MAX assuming this array
maps directly to ConflictType. I think it would be better to have it
passed as a list and then SetSubConflictResolver() traverse the list
without knowing the range of it. Similar to what we do in
alter-sub-flow in and around UpdateSubConflictResolvers().
3)
When I execute 'alter subscription ..(detect_conflict=on)' for a
subscription which *already* has detect_conflict as ON, it tries to
reset resolvers to default and ends up in error. It should actually be
no-op in this particular situation and should not reset resolvers to
default.
postgres=# alter subscription sub1 set (detect_conflict=on);
WARNING: Using default conflict resolvers
ERROR: duplicate key value violates unique constraint
"pg_subscription_conflict_sub_index"
4)
Do we need SUBSCRIPTIONCONFLICTOID cache? We are not using it
anywhere. Shall we remove this and the corresponding index?
5)
RemoveSubscriptionConflictBySubid().
--We can remove extra blank line before table_open.
--We can get rid of curly braces around CatalogTupleDelete() as it is
a single line in loop.
thanks
Shveta
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
2024-07-30 04:19 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
@ 2024-07-31 11:24 ` Ajin Cherian <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Ajin Cherian @ 2024-07-31 11:24 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Nisha Moond <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Dilip Kumar <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>
On Tue, Jul 30, 2024 at 2:19 PM shveta malik <[email protected]> wrote:
> On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian <[email protected]> wrote:
> >>
> > Please find v7 patch-set, the changes are:
> >
>
> Thanks Ajin for working on this. Please find few comments:
>
> 1)
> parse_subscription_conflict_resolvers():
> Here we loop in this function to find the given conflict type in the
> supported list and error out if conflict-type is not valid. Also we
> call validate_conflict_type_and_resolver() which again validates
> conflict-type. I would recommend to loop 'stmtresolvers' in parse
> function and then read each type and resolver and pass that to
> validate_conflict_type_and_resolver(). Avoid double validation.
>
>
I have modified this as per comment.
> 2)
> SetSubConflictResolver():
> It works well, but it does not look apt that the 'resolvers' passed to
> this function by the caller is an array and this function knows the
> array range and traverse from CT_MIN to CT_MAX assuming this array
> maps directly to ConflictType. I think it would be better to have it
> passed as a list and then SetSubConflictResolver() traverse the list
> without knowing the range of it. Similar to what we do in
> alter-sub-flow in and around UpdateSubConflictResolvers().
>
>
I have kept the array as it requires that all conflict resolvers be set, if
not provided by the user then default needs to be used. However, I have
modified SetSubConflictResolver such that it takes in the size of the array
and does not assume it.
3)
> When I execute 'alter subscription ..(detect_conflict=on)' for a
> subscription which *already* has detect_conflict as ON, it tries to
> reset resolvers to default and ends up in error. It should actually be
> no-op in this particular situation and should not reset resolvers to
> default.
>
> postgres=# alter subscription sub1 set (detect_conflict=on);
> WARNING: Using default conflict resolvers
> ERROR: duplicate key value violates unique constraint
> "pg_subscription_conflict_sub_index"
>
>
fixed
> 4)
> Do we need SUBSCRIPTIONCONFLICTOID cache? We are not using it
> anywhere. Shall we remove this and the corresponding index?
>
>
We are using the index but not the cache, so removing the cache.
> 5)
> RemoveSubscriptionConflictBySubid().
> --We can remove extra blank line before table_open.
> --We can get rid of curly braces around CatalogTupleDelete() as it is
> a single line in loop.
>
>
fixed.
On Tue, Jul 30, 2024 at 8:34 PM Dilip Kumar <[email protected]> wrote:
> On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian <[email protected]> wrote:
>
> Comment in 0002,
>
> 1) I do not see any test case that set a proper conflict type and
> conflict resolver, all tests either give incorrect conflict
> type/conflict resolver or the conflict resolver is ignored
>
fixed.
I've also fixed a cfbot error due to patch 0001. Rebase of table resolver
patch is still pending, will try and target that in the next patch-set.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v8-0004-Manage-Clock-skew-and-implement-last_update_wins.patch (51.3K, ../../CAFPTHDZvACoqRJzrdd8BtexuTD8y3xaSo__y5Ep4u6CnNBspkg@mail.gmail.com/3-v8-0004-Manage-Clock-skew-and-implement-last_update_wins.patch)
download | inline diff:
From cb9f46957122238076a28713ecfde1844e639e9c Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 31 Jul 2024 03:27:18 -0400
Subject: [PATCH v8 4/4] Manage Clock skew and implement last_update_wins
This patch attempts to manage clock skew between nodes by
introducing two new GUCs:
a) max_logical_rep_clock_skew
b) max_logical_rep_clock_skew_action
c) max_logical_rep_clock_skew_wait
If the timestamp of the currently replayed transaction is in the future
compared to the current time on the subscriber and the difference is
larger than 'max_logical_rep_clock_skew', then the action configured
in 'max_logical_rep_clock_skew_action' is performed by the apply worker.
If user configures 'wait' in 'max_logical_rep_clock_skew_action' and
actual clock skew is large while 'max_logical_rep_clock_skew' is small,
the apply worker may have to wait for a longer period to manage the clock
skew. To control this maximum wait time, a new GUC,
'max_logical_rep_clock_skew_wait', is provided. This allows the user to
set a cap on how long the apply worker should wait. If the computed wait
time exceeds this value, the apply worker will error out without waiting.
This patch also implements last_update_wins resolver.
Since conflict resolution for two phase commit transactions
using prepare-timestamp can result in data divergence, this patch
also restricts enabling two_phase and detect_conflict together
for a subscription.
---
src/backend/commands/subscriptioncmds.c | 45 ++++++
src/backend/executor/execReplication.c | 2 +-
.../replication/logical/applyparallelworker.c | 26 +++-
src/backend/replication/logical/conflict.c | 146 ++++++++++++++-----
src/backend/replication/logical/origin.c | 1 +
src/backend/replication/logical/worker.c | 155 +++++++++++++++++++--
src/backend/utils/activity/wait_event_names.txt | 1 +
src/backend/utils/misc/guc_tables.c | 40 ++++++
src/backend/utils/misc/postgresql.conf.sample | 9 +-
src/include/replication/conflict.h | 6 +-
src/include/replication/logicalworker.h | 18 +++
src/include/replication/origin.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/include/utils/timestamp.h | 1 +
src/test/regress/expected/subscription.out | 46 +++---
src/test/subscription/t/029_on_error.pl | 53 +++++--
src/test/subscription/t/034_conflict_resolver.pl | 116 +++++++++++++--
src/tools/pgindent/typedefs.list | 1 +
18 files changed, 576 insertions(+), 93 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 5c18c45..5a0c8b0 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -14,6 +14,7 @@
#include "postgres.h"
+#include "access/commit_ts.h"
#include "access/htup_details.h"
#include "access/table.h"
#include "access/twophase.h"
@@ -452,6 +453,22 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
"slot_name = NONE", "create_slot = false")));
}
}
+
+ /*
+ * Time based conflict resolution for two phase transactions can result in
+ * data divergence, so disallow enabling both together.
+ */
+ if (opts->detectconflict &&
+ IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ if (opts->twophase &&
+ IsSet(opts->specified_opts, SUBOPT_TWOPHASE_COMMIT))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ /*- translator: both %s are strings of the form "option = value" */
+ errmsg("%s and %s are mutually exclusive options",
+ "detect_conflict = true", "two_phase = true")));
+ }
}
/*
@@ -719,6 +736,13 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
elog(WARNING, "subscriptions created by regression test cases should have names starting with \"regress_\"");
#endif
+ /* Warn if detect_conflict is enabled and track_commit_timestamp is off */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
+
rel = table_open(SubscriptionRelationId, RowExclusiveLock);
/* Check if name is used */
@@ -1454,6 +1478,27 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
values[Anum_pg_subscription_subdetectconflict - 1] =
BoolGetDatum(opts.detectconflict);
replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+
+ /*
+ * Time based conflict resolution for two phase
+ * transactions can result in data divergence, so disallow
+ * enabling it when two_phase is enabled.
+ */
+ if (sub->twophasestate == LOGICALREP_TWOPHASE_STATE_ENABLED)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set %s for a subscription that has two_phase enabled",
+ "detect_conflict")));
+
+ /*
+ * Warn if detect_conflict is enabled and
+ * track_commit_timestamp is off.
+ */
+ if (opts.detectconflict && !track_commit_timestamp)
+ ereport(WARNING,
+ (errmsg("detect_conflict is enabled but \"%s\" is OFF, the last_update_wins resolution may not work",
+ "track_commit_timestamp"),
+ errhint("Enable \"%s\".", "track_commit_timestamp")));
}
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index fb96bb7..93ba364 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -669,7 +669,7 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
bool apply_remote = false;
GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
- resolver = GetConflictResolver(rel, CT_INSERT_EXISTS,
+ resolver = GetConflictResolver(*conflictslot, rel, CT_INSERT_EXISTS,
&apply_remote, NULL, subid);
ReportApplyConflict(CT_INSERT_EXISTS, resolver, rel,
diff --git a/src/backend/replication/logical/applyparallelworker.c b/src/backend/replication/logical/applyparallelworker.c
index e7f7d4c..10c7ca9 100644
--- a/src/backend/replication/logical/applyparallelworker.c
+++ b/src/backend/replication/logical/applyparallelworker.c
@@ -312,6 +312,20 @@ pa_can_start(void)
if (!AllTablesyncsReady())
return false;
+ /*
+ * Don't start a new parallel worker if user has either configured max
+ * clock skew or if conflict detection and resolution is ON. In both cases
+ * we need commit timestamp in the beginning.
+ *
+ * XXX: For conflict reolution case, see if we can reduce the scope of
+ * this restriction to only such cases where time-based resolvers are
+ * actually being used.
+ */
+ if ((max_logical_rep_clock_skew > LR_CLOCK_SKEW_DEFAULT) ||
+ MySubscription->detectconflict)
+ return false;
+
+
return true;
}
@@ -696,9 +710,19 @@ pa_process_spooled_messages_if_required(void)
}
else if (fileset_state == FS_READY)
{
+ /*
+ * Currently we do not support starting parallel apply worker when
+ * either clock skew is configured or conflict resolution is
+ * configured, thus it is okay to pass 0 as origin-timestamp here.
+ *
+ * XXX: If in future, we support starting pa worker even with conflict
+ * detection enabled, then here we need to pass remote's
+ * commit/prepare/abort timestamp; we can get that info from leader
+ * worker in shared memory.
+ */
apply_spooled_messages(&MyParallelShared->fileset,
MyParallelShared->xid,
- InvalidXLogRecPtr);
+ InvalidXLogRecPtr, 0);
pa_set_fileset_state(MyParallelShared, FS_EMPTY);
}
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 9ac707c..7af07b8 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -50,6 +50,7 @@ const char *const ConflictTypeNames[] = {
const char *const ConflictResolverNames[] = {
[CR_REMOTE_APPLY] = "remote_apply",
[CR_KEEP_LOCAL] = "keep_local",
+ [CR_LAST_UPDATE_WINS] = "last_update_wins",
[CR_APPLY_OR_SKIP] = "apply_or_skip",
[CR_APPLY_OR_ERROR] = "apply_or_error",
[CR_SKIP] = "skip",
@@ -75,24 +76,24 @@ const char *const ConflictResolverNames[] = {
* friendly name for a resolver and thus has been added here.
*/
const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
- [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
- [CT_UPDATE_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
- [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
+ [CT_UPDATE_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR},
[CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
[CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
- [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_LAST_UPDATE_WINS, CR_ERROR}
};
/*
* Default conflict resolver for each conflict type.
*/
const int ConflictTypeDefaultResolvers[] = {
- [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
- [CT_UPDATE_EXISTS] = CR_REMOTE_APPLY,
- [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_INSERT_EXISTS] = CR_LAST_UPDATE_WINS,
+ [CT_UPDATE_EXISTS] = CR_LAST_UPDATE_WINS,
+ [CT_UPDATE_DIFFER] = CR_LAST_UPDATE_WINS,
[CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
[CT_DELETE_MISSING] = CR_SKIP,
- [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+ [CT_DELETE_DIFFER] = CR_LAST_UPDATE_WINS
};
@@ -207,6 +208,12 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
TupleTableSlot *conflictslot, bool apply_remote)
{
char *applymsg;
+ int errdet = 0;
+ char *local_ts;
+ char *remote_ts;
+
+ local_ts = pstrdup(timestamptz_to_str(localts));
+ remote_ts = pstrdup(timestamptz_to_str(replorigin_session_origin_timestamp));
if (apply_remote)
applymsg = "applying the remote changes.";
@@ -229,43 +236,63 @@ errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
if (resolver == CR_ERROR)
{
if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
+ errdet = errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx),
+ localorigin, localxmin, local_ts);
else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ errdet = errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ errdet = errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
}
+ else if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Key already exists, %s. The local tuple : origin=%u, timestamp=%s; The remote tuple : origin=%u, timestamp=%s.",
+ applymsg, localorigin, local_ts,
+ replorigin_session_origin, remote_ts);
else
- return errdetail("Key already exists, %s", applymsg);
+ errdet = errdetail("Key already exists, %s", applymsg);
}
+ break;
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
- localorigin, localxmin,
- timestamptz_to_str(localts), applymsg);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, local_ts, applymsg,
+ replorigin_session_origin, remote_ts);
+ else
+ errdet = errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin, local_ts, applymsg);
+ break;
case CT_UPDATE_MISSING:
if (resolver == CR_APPLY_OR_SKIP && !apply_remote)
- return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
+ errdet = errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
else if (resolver == CR_APPLY_OR_ERROR && !apply_remote)
- return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
+ errdet = errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
else if (apply_remote)
- return errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
- applymsg);
+ errdet = errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
+ applymsg);
else
- return errdetail("Did not find the row to be updated, %s",
- applymsg);
+ errdet = errdetail("Did not find the row to be updated, %s",
+ applymsg);
+ break;
case CT_DELETE_MISSING:
- return errdetail("Did not find the row to be deleted.");
+ errdet = errdetail("Did not find the row to be deleted.");
+ break;
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
- localorigin, localxmin,
- timestamptz_to_str(localts), applymsg);
+ if (resolver == CR_LAST_UPDATE_WINS)
+ errdet = errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s The remote tuple : origin=%u, timestamp=%s.",
+ localorigin, localxmin, local_ts, applymsg,
+ replorigin_session_origin, remote_ts);
+ else
+ errdet = errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin, local_ts, applymsg);
+ break;
}
- return 0; /* silence compiler warning */
+ pfree(local_ts);
+ pfree(remote_ts);
+
+ return errdet;
}
/*
@@ -375,6 +402,15 @@ validate_conflict_type_and_resolver(const char *conflict_type,
errmsg("%s is not a valid conflict resolver for conflict type %s",
conflict_resolver,
conflict_type));
+
+ if ((resolver == CR_LAST_UPDATE_WINS) && !track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ conflict_resolver, "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+
return type;
}
@@ -420,6 +456,42 @@ get_conflict_resolver_internal(ConflictType type, Oid subid)
}
/*
+ * Compare the timestamps of given local tuple and the remote tuple to
+ * resolve the conflict.
+ *
+ * Returns true if remote tuple has the latest timestamp, false otherwise.
+ */
+static bool
+resolve_by_timestamp(TupleTableSlot *localslot)
+{
+ TransactionId local_xmin;
+ TimestampTz local_ts;
+ RepOriginId local_origin;
+ int ts_cmp;
+ uint64 local_system_identifier;
+
+ /* Get origin and timestamp info of the local tuple */
+ GetTupleCommitTs(localslot, &local_xmin, &local_origin, &local_ts);
+
+ /* Compare the timestamps of remote & local tuple to decide the winner */
+ ts_cmp = timestamptz_cmp_internal(replorigin_session_origin_timestamp,
+ local_ts);
+
+ if (ts_cmp == 0)
+ {
+ elog(LOG, "Timestamps of remote and local tuple are equal, comparing remote and local system identifiers");
+
+ /* Get current system's identifier */
+ local_system_identifier = GetSystemIdentifier();
+
+ return local_system_identifier <= replorigin_session_origin_sysid;
+ }
+ else
+ return (ts_cmp > 0);
+
+}
+
+/*
* Check if a full tuple can be created from the new tuple.
* Return true if yes, false otherwise.
*/
@@ -610,7 +682,8 @@ RemoveSubscriptionConflictBySubid(Oid subid)
* false otherwise.
*/
ConflictResolver
-GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
+GetConflictResolver(TupleTableSlot *localslot, Relation localrel,
+ ConflictType type, bool *apply_remote,
LogicalRepTupleData *newtup, Oid subid)
{
ConflictResolver resolver;
@@ -619,6 +692,17 @@ GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
switch (resolver)
{
+ case CR_LAST_UPDATE_WINS:
+ if (!track_commit_timestamp)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("resolver %s requires \"%s\" to be enabled",
+ ConflictResolverNames[resolver], "track_commit_timestamp"),
+ errhint("Make sure the configuration parameter \"%s\" is set.",
+ "track_commit_timestamp"));
+ else
+ *apply_remote = resolve_by_timestamp(localslot);
+ break;
case CR_REMOTE_APPLY:
*apply_remote = true;
break;
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 419e481..bd8e6f0 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -155,6 +155,7 @@ typedef struct ReplicationStateCtl
RepOriginId replorigin_session_origin = InvalidRepOriginId; /* assumed identity */
XLogRecPtr replorigin_session_origin_lsn = InvalidXLogRecPtr;
TimestampTz replorigin_session_origin_timestamp = 0;
+uint64 replorigin_session_origin_sysid = 0;
/*
* Base address into a shared memory array of replication states of size
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 36172b2..5b3b89e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -319,6 +319,20 @@ static uint32 parallel_stream_nchanges = 0;
bool InitializingApplyWorker = false;
/*
+ * GUC support
+ */
+const struct config_enum_entry logical_rep_clock_skew_action_options[] = {
+ {"error", LR_CLOCK_SKEW_ACTION_ERROR, false},
+ {"wait", LR_CLOCK_SKEW_ACTION_WAIT, false},
+ {NULL, 0, false}
+};
+
+/* GUCs */
+int max_logical_rep_clock_skew = LR_CLOCK_SKEW_DEFAULT;
+int max_logical_rep_clock_skew_action = LR_CLOCK_SKEW_ACTION_ERROR;
+int max_logical_rep_clock_skew_wait = 300; /* 5 mins */
+
+/*
* We enable skipping all data modification changes (INSERT, UPDATE, etc.) for
* the subscription if the remote transaction's finish LSN matches the subskiplsn.
* Once we start skipping changes, we don't stop it until we skip all changes of
@@ -985,6 +999,95 @@ slot_modify_data(TupleTableSlot *slot, TupleTableSlot *srcslot,
}
/*
+ * Manage clock skew between nodes.
+ *
+ * It checks if the remote timestamp is ahead of the local clock
+ * and if the difference exceeds max_logical_rep_clock_skew, it performs
+ * the action specified by the max_logical_rep_clock_skew_action.
+ */
+static void
+manage_clock_skew(TimestampTz origin_timestamp)
+{
+ TimestampTz current;
+ TimestampTz delayUntil;
+ long msecs;
+ int rc;
+
+ /* nothing to do if no max clock skew configured */
+ if (max_logical_rep_clock_skew == LR_CLOCK_SKEW_DEFAULT)
+ return;
+
+ current = GetCurrentTimestamp();
+
+ /*
+ * If the timestamp of the currently replayed transaction is in the future
+ * compared to the current time on the subscriber and the difference is
+ * larger than max_logical_rep_clock_skew, then perform the action
+ * specified by the max_logical_rep_clock_skew_action setting.
+ */
+ if (origin_timestamp > current &&
+ TimestampDifferenceExceeds(current, origin_timestamp,
+ max_logical_rep_clock_skew * 1000))
+ {
+ if (max_logical_rep_clock_skew_action == LR_CLOCK_SKEW_ACTION_ERROR)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg_internal("clock skew exceeds max_logical_rep_clock_skew (%d seconds)",
+ max_logical_rep_clock_skew)));
+
+ /* Perform the wait */
+ while (true)
+ {
+ delayUntil =
+ TimestampTzMinusSeconds(origin_timestamp,
+ max_logical_rep_clock_skew);
+
+ /* Exit without waiting if it's already past 'delayUntil' time */
+ msecs = TimestampDifferenceMilliseconds(GetCurrentTimestamp(),
+ delayUntil);
+ if (msecs <= 0)
+ break;
+
+ /* The wait time should not exceed max_logical_rep_clock_skew_wait */
+ if (msecs > (max_logical_rep_clock_skew_wait * 1000L))
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg_internal("clock skew wait time exceeds max_logical_rep_clock_skew_wait (%d seconds)",
+ max_logical_rep_clock_skew_wait)));
+
+ elog(DEBUG2, "delaying apply for %ld milliseconds to manage clock skew",
+ msecs);
+
+ /* Sleep until we are signaled or msecs have elapsed */
+ rc = WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ msecs,
+ WAIT_EVENT_LOGICAL_CLOCK_SKEW);
+
+ /* Exit the loop if msecs have elapsed */
+ if (rc & WL_TIMEOUT)
+ break;
+
+ if (rc & WL_LATCH_SET)
+ {
+ ResetLatch(MyLatch);
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ /*
+ * This might change max_logical_rep_clock_skew and
+ * max_logical_rep_clock_skew_wait.
+ */
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+ }
+ }
+}
+
+/*
* Handle BEGIN message.
*/
static void
@@ -1005,6 +1108,15 @@ apply_handle_begin(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.committime);
+
+ /*
+ * Capture the commit timestamp of the remote transaction for time based
+ * conflict resolution purpose.
+ */
+ replorigin_session_origin_timestamp = begin_data.committime;
}
/*
@@ -1062,6 +1174,9 @@ apply_handle_begin_prepare(StringInfo s)
in_remote_transaction = true;
pgstat_report_activity(STATE_RUNNING, NULL);
+
+ /* Check if there is any clock skew and perform configured action */
+ manage_clock_skew(begin_data.prepare_time);
}
/*
@@ -1302,7 +1417,8 @@ apply_handle_stream_prepare(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset,
- prepare_data.xid, prepare_data.prepare_lsn);
+ prepare_data.xid, prepare_data.prepare_lsn,
+ prepare_data.prepare_time);
/* Mark the transaction as prepared. */
apply_handle_prepare_internal(&prepare_data);
@@ -1999,7 +2115,8 @@ ensure_last_message(FileSet *stream_fileset, TransactionId xid, int fileno,
*/
void
apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn)
+ XLogRecPtr lsn,
+ TimestampTz origin_timestamp)
{
int nchanges;
char path[MAXPGPATH];
@@ -2053,6 +2170,16 @@ apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
end_replication_step();
/*
+ * If origin_timestamp is provided by caller, then check clock skew with
+ * respect to the passed time and take configured action.
+ */
+ if (origin_timestamp)
+ manage_clock_skew(origin_timestamp);
+
+ /* Capture the timestamp (prepare or commit) of the remote transaction */
+ replorigin_session_origin_timestamp = origin_timestamp;
+
+ /*
* Read the entries one by one and pass them through the same logic as in
* apply_dispatch.
*/
@@ -2157,7 +2284,8 @@ apply_handle_stream_commit(StringInfo s)
* spooled operations.
*/
apply_spooled_messages(MyLogicalRepWorker->stream_fileset, xid,
- commit_data.commit_lsn);
+ commit_data.commit_lsn,
+ commit_data.committime);
apply_handle_commit_internal(&commit_data);
@@ -2715,7 +2843,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(localrel, CT_UPDATE_DIFFER,
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_DIFFER,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -2755,7 +2883,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(localrel, CT_UPDATE_MISSING,
+ resolver = GetConflictResolver(localslot, localrel, CT_UPDATE_MISSING,
&apply_remote, newtup,
MyLogicalRepWorker->subid);
@@ -2909,7 +3037,7 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(localrel, CT_DELETE_DIFFER,
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_DIFFER,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -2936,7 +3064,7 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(localrel, CT_DELETE_MISSING,
+ resolver = GetConflictResolver(localslot, localrel, CT_DELETE_MISSING,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -3135,7 +3263,7 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
*/
if (MySubscription->detectconflict)
{
- resolver = GetConflictResolver(partrel, CT_UPDATE_MISSING,
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_MISSING,
&apply_remote, newtup,
MyLogicalRepWorker->subid);
@@ -3168,7 +3296,7 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
{
- resolver = GetConflictResolver(partrel, CT_UPDATE_DIFFER,
+ resolver = GetConflictResolver(localslot, partrel, CT_UPDATE_DIFFER,
&apply_remote, NULL,
MyLogicalRepWorker->subid);
@@ -4671,6 +4799,7 @@ run_apply_worker()
TimeLineID startpointTLI;
char *err;
bool must_use_password;
+ char *replorigin_sysid;
slotname = MySubscription->slotname;
@@ -4711,10 +4840,12 @@ run_apply_worker()
MySubscription->name, err)));
/*
- * We don't really use the output identify_system for anything but it does
- * some initializations on the upstream so let's still call it.
+ * Call identify_system to do some initializations on the upstream and
+ * store the output as system identifier of the replication origin node.
*/
- (void) walrcv_identify_system(LogRepWorkerWalRcvConn, &startpointTLI);
+ replorigin_sysid = walrcv_identify_system(LogRepWorkerWalRcvConn,
+ &startpointTLI);
+ replorigin_session_origin_sysid = strtoul(replorigin_sysid, NULL, 10);
set_apply_error_context_origin(originname);
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index db37bee..a51f821 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -59,6 +59,7 @@ CHECKPOINTER_MAIN "Waiting in main loop of checkpointer process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_CLOCK_SKEW "Waiting in apply-begin of logical replication apply process to bring clock skew in permissible range."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot sync worker."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 6a623f5..c9fdfa1 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -68,6 +68,7 @@
#include "postmaster/walsummarizer.h"
#include "postmaster/walwriter.h"
#include "replication/logicallauncher.h"
+#include "replication/logicalworker.h"
#include "replication/slot.h"
#include "replication/slotsync.h"
#include "replication/syncrep.h"
@@ -482,6 +483,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry wal_sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry logical_rep_clock_skew_action_options[];
/*
* GUC option variables that are exported from this module
@@ -3651,6 +3653,33 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_logical_rep_clock_skew", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets maximum clock skew tolerance between logical "
+ "replication nodes beyond which action configured "
+ "in max_logical_rep_clock_skew_action is triggered."),
+ gettext_noop("-1 turns this check off."),
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew,
+ LR_CLOCK_SKEW_DEFAULT, LR_CLOCK_SKEW_DEFAULT, INT_MAX,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"max_logical_rep_clock_skew_wait", PGC_SIGHUP, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets max limit on how long apply worker shall wait to "
+ "bring clock skew within permissible range of max_logical_rep_clock_skew. "
+ "If the computed wait time is more than this value, "
+ "apply worker will error out without waiting."),
+ gettext_noop("0 turns this limit off."),
+ GUC_UNIT_S
+ },
+ &max_logical_rep_clock_skew_wait,
+ 300, 0, 3600,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
@@ -4918,6 +4947,17 @@ struct config_enum ConfigureNamesEnum[] =
},
{
+ {"max_logical_rep_clock_skew_action", PGC_POSTMASTER, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the action to perform if a clock skew higher "
+ "than max_logical_rep_clock_skew is detected."),
+ NULL
+ },
+ &max_logical_rep_clock_skew_action,
+ LR_CLOCK_SKEW_ACTION_ERROR, logical_rep_clock_skew_action_options,
+ NULL, NULL, NULL
+ },
+
+ {
{"track_functions", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects function-level statistics on database activity."),
NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 9ec9f97..f7a664a 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -383,7 +383,14 @@
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
-
+#max_logical_rep_clock_skew = -1 # maximum clock skew tolerance between logical
+ # replication nodes beyond which action configured in
+ # 'max_logical_rep_clock_skew_action' is triggered.
+#max_logical_rep_clock_skew_action = error # error or wait
+ # (change requires restart)
+#max_logical_rep_clock_skew_wait = 300 # max limit on how long apply worker
+ # shall wait to bring clock skew within permissible
+ # range of max_logical_rep_clock_skew.
#------------------------------------------------------------------------------
# QUERY TUNING
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index 2793ad6..8791f1b 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -59,6 +59,9 @@ typedef enum ConflictResolver
/* Keep the local change */
CR_KEEP_LOCAL,
+ /* Apply the change with latest timestamp */
+ CR_LAST_UPDATE_WINS,
+
/* Apply the remote change; skip if it can not be applied */
CR_APPLY_OR_SKIP,
@@ -98,7 +101,8 @@ extern void UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid);
extern ConflictType validate_conflict_type_and_resolver(const char *conflict_type,
const char *conflict_resolver);
extern void SetDefaultResolvers(ConflictTypeResolver * conflictResolvers);
-extern ConflictResolver GetConflictResolver(Relation localrel,
+extern ConflictResolver GetConflictResolver(TupleTableSlot *localslot,
+ Relation localrel,
ConflictType type,
bool *apply_remote,
LogicalRepTupleData *newtup,
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index a18d79d..7cb0306 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,7 +14,25 @@
#include <signal.h>
+/*
+ * The default for max_logical_rep_clock_skew is -1, which means ignore clock
+ * skew (the check is turned off).
+ */
+#define LR_CLOCK_SKEW_DEFAULT -1
+
+/*
+ * Worker Clock Skew Action.
+ */
+typedef enum
+{
+ LR_CLOCK_SKEW_ACTION_ERROR,
+ LR_CLOCK_SKEW_ACTION_WAIT,
+} LogicalRepClockSkewAction;
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
+extern PGDLLIMPORT int max_logical_rep_clock_skew;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_action;
+extern PGDLLIMPORT int max_logical_rep_clock_skew_wait;
extern void ApplyWorkerMain(Datum main_arg);
extern void ParallelApplyWorkerMain(Datum main_arg);
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 7189ba9..dcbbbdf 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -36,6 +36,7 @@ typedef struct xl_replorigin_drop
extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+extern PGDLLIMPORT uint64 replorigin_session_origin_sysid;
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 9646261..95b2a52 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -268,7 +268,7 @@ extern void stream_stop_internal(TransactionId xid);
/* Common streaming function to apply all the spooled messages */
extern void apply_spooled_messages(FileSet *stream_fileset, TransactionId xid,
- XLogRecPtr lsn);
+ XLogRecPtr lsn, TimestampTz origin_timestamp);
extern void apply_dispatch(StringInfo s);
diff --git a/src/include/utils/timestamp.h b/src/include/utils/timestamp.h
index a6ce03e..53b828d 100644
--- a/src/include/utils/timestamp.h
+++ b/src/include/utils/timestamp.h
@@ -84,6 +84,7 @@ IntervalPGetDatum(const Interval *X)
/* Macros for doing timestamp arithmetic without assuming timestamp's units */
#define TimestampTzPlusMilliseconds(tz,ms) ((tz) + ((ms) * (int64) 1000))
#define TimestampTzPlusSeconds(tz,s) ((tz) + ((s) * (int64) 1000000))
+#define TimestampTzMinusSeconds(tz,s) ((tz) - ((s) * (int64) 1000000))
/* Set at postmaster start */
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 7bd4e82..f46e922 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -431,6 +431,8 @@ ERROR: detect_conflict requires a Boolean value
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
WARNING: Will use default resolvers configuration as detect_conflict is ON but resolvers are not given
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
@@ -442,13 +444,13 @@ HINT: To initiate replication, you must manually create the replication slot, e
-- confirm that the default conflict resolvers have been set
SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
- confrtype | confrres
-----------------+---------------
- delete_differ | remote_apply
+ confrtype | confrres
+----------------+------------------
+ delete_differ | last_update_wins
delete_missing | skip
- insert_exists | remote_apply
- update_differ | remote_apply
- update_exists | remote_apply
+ insert_exists | last_update_wins
+ update_differ | last_update_wins
+ update_exists | last_update_wins
update_missing | apply_or_skip
(6 rows)
@@ -487,18 +489,20 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
--try setting resolvers for few types
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = 'keep_local', update_missing = 'skip', delete_differ = 'keep_local' );
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
--check if above are configured; for non specified conflict types, default resolvers should be seen
SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
- confrtype | confrres
-----------------+--------------
+ confrtype | confrres
+----------------+------------------
delete_differ | keep_local
delete_missing | skip
insert_exists | keep_local
- update_differ | remote_apply
- update_exists | remote_apply
+ update_differ | last_update_wins
+ update_exists | last_update_wins
update_missing | skip
(6 rows)
@@ -518,14 +522,16 @@ SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
-- setting detect_conflict to true will set default conflict resolvers
ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = true);
WARNING: Using default conflict resolvers
+WARNING: detect_conflict is enabled but "track_commit_timestamp" is OFF, the last_update_wins resolution may not work
+HINT: Enable "track_commit_timestamp".
SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
- confrtype | confrres
-----------------+---------------
- delete_differ | remote_apply
+ confrtype | confrres
+----------------+------------------
+ delete_differ | last_update_wins
delete_missing | skip
- insert_exists | remote_apply
- update_differ | remote_apply
- update_exists | remote_apply
+ insert_exists | last_update_wins
+ update_differ | last_update_wins
+ update_exists | last_update_wins
update_missing | apply_or_skip
(6 rows)
@@ -538,13 +544,13 @@ ERROR: foo is not a valid conflict resolver
-- ok - valid conflict type and resolver
ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'keep_local', update_missing = 'skip', delete_differ = 'keep_local' );
SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
- confrtype | confrres
-----------------+--------------
+ confrtype | confrres
+----------------+------------------
delete_differ | keep_local
delete_missing | skip
insert_exists | keep_local
- update_differ | remote_apply
- update_exists | remote_apply
+ update_differ | last_update_wins
+ update_exists | last_update_wins
update_missing | skip
(6 rows)
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index e6f07fa..9620a4b 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -18,7 +18,8 @@ my $offset = 0;
# on the publisher.
sub test_skip_lsn
{
- my ($node_publisher, $node_subscriber, $nonconflict_data, $expected, $msg)
+ my ($node_publisher, $node_subscriber, $nonconflict_data, $expected,
+ $msg, $conflict_detection)
= @_;
# Wait until a conflict occurs on the subscriber.
@@ -26,13 +27,25 @@ sub test_skip_lsn
"SELECT subenabled = FALSE FROM pg_subscription WHERE subname = 'sub'"
);
+ my $lsn;
+ my $contents = slurp_file($node_subscriber->logfile, $offset);
+
# Get the finish LSN of the error transaction, mapping the expected
# ERROR with its CONTEXT when retrieving this information.
- my $contents = slurp_file($node_subscriber->logfile, $offset);
- $contents =~
- qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
- or die "could not get error-LSN";
- my $lsn = $1;
+ if ($conflict_detection)
+ {
+ $contents =~
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ or die "could not get error-LSN";
+ $lsn = $1;
+ }
+ else
+ {
+ $contents =~
+ qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ or die "could not get error-LSN";
+ $lsn = $1;
+ }
# Set skip lsn.
$node_subscriber->safe_psql('postgres',
@@ -110,7 +123,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, detect_conflict = on)"
);
# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
@@ -148,7 +161,22 @@ INSERT INTO tbl VALUES (1, NULL);
COMMIT;
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(2, NULL)", "2", "test skipping transaction");
+ "(2, NULL)", "2", "test skipping transaction", 1);
+
+# Cleanup before we start PREPARE AND COMMIT PREPARED tests
+$node_subscriber->safe_psql('postgres', "TRUNCATE tbl");
+$node_publisher->safe_psql('postgres', "TRUNCATE tbl");
+
+# Drop subscription and recreate with two_phase enabled
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+);
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'sub');
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tbl VALUES (1, NULL)");
# Test for PREPARE and COMMIT PREPARED. Insert the same data to tbl and
# PREPARE the transaction, raising an error. Then skip the transaction.
@@ -161,7 +189,7 @@ PREPARE TRANSACTION 'gtx';
COMMIT PREPARED 'gtx';
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(3, NULL)", "3", "test skipping prepare and commit prepared ");
+ "(2, NULL)", "2", "test skipping prepare and commit prepared ", 0);
# Test for STREAM COMMIT. Insert enough rows to tbl to exceed the 64kB
# limit, also raising an error on the subscriber during applying spooled
@@ -174,17 +202,14 @@ INSERT INTO tbl SELECT i, sha256(i::text::bytea) FROM generate_series(1, 10000)
COMMIT;
]);
test_skip_lsn($node_publisher, $node_subscriber,
- "(4, sha256(4::text::bytea))",
- "4", "test skipping stream-commit");
+ "(3, sha256(4::text::bytea))",
+ "3", "test skipping stream-commit", 0);
$result = $node_subscriber->safe_psql('postgres',
"SELECT COUNT(*) FROM pg_prepared_xacts");
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
-# Reset conflict resolver for 'insert_exist' conflict type to default.
-$node_subscriber->safe_psql('postgres',
- "ALTER SUBSCRIPTION sub CONFLICT RESOLVER (insert_exists=remote_apply)");
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
index 58751c5..e961809 100755
--- a/src/test/subscription/t/034_conflict_resolver.pl
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -54,11 +54,11 @@ $node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
my $result = $node_subscriber->safe_psql('postgres',
"SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype"
);
-is( $result, qq(delete_differ|remote_apply
+is( $result, qq(delete_differ|last_update_wins
delete_missing|skip
-insert_exists|remote_apply
-update_differ|remote_apply
-update_exists|remote_apply
+insert_exists|last_update_wins
+update_differ|last_update_wins
+update_exists|last_update_wins
update_missing|apply_or_skip),
"confirm that the default conflict resolvers are in place");
@@ -66,6 +66,11 @@ update_missing|apply_or_skip),
# Test 'remote_apply' for 'insert_exists'
############################################
+# Change CONFLICT RESOLVER of insert_exists to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'remote_apply');"
+);
+
# Create local data on the subscriber
$node_subscriber->safe_psql('postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
@@ -148,6 +153,34 @@ $node_subscriber->wait_for_log(
# Truncate table on subscriber to get rid of the error
$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+############################################
+# Test 'last_update_wins' for 'insert_exists'
+############################################
+
+# Change CONFLICT RESOLVER of insert_exists to last_update_wins
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'last_update_wins');"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (4,'fromsub')");
+
+# Create conflicting data on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (4,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=4);");
+
+is($result, 'frompub', "remote data wins");
+
###################################
# Test 'skip' for 'delete_missing'
###################################
@@ -202,16 +235,49 @@ $node_subscriber->safe_psql(
$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
-#########################################
-# Test 'remote_apply' for 'delete_differ'
-#########################################
+#############################################
+# Test 'last_update_wins' for 'delete_differ'
+#############################################
+
+# Change CONFLICT RESOLVER of delete_differ to last_update_wins
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'last_update_wins');"
+);
# Insert data in the publisher
$node_publisher->safe_psql(
'postgres',
"INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
- INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (4,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=4);");
+
+# Create a conflicting delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote delete the local updated row
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=4);");
+
+is($result, '', "delete from remote wins");
+
+#########################################
+# Test 'remote_apply' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'remote_apply');"
+);
# Modify data on the subscriber
$node_subscriber->safe_psql('postgres',
@@ -299,9 +365,9 @@ $node_subscriber->safe_psql(
# Wait for initial table sync to finish
$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
-#########################################
-# Test 'remote_apply' for 'update_differ'
-#########################################
+#############################################
+# Test 'last_update_wins' for 'update_differ'
+#############################################
# Insert data in the publisher
$node_publisher->safe_psql(
@@ -330,6 +396,34 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, 'frompubnew', "update from remote is kept");
#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to remote_apply
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_differ = 'remote_apply');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub2' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew2' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);");
+
+is($result, 'frompubnew2', "update from remote is kept");
+
+#########################################
# Test 'keep_local' for 'update_differ'
#########################################
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ae1db1a..77c6469 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1565,6 +1565,7 @@ LogicalOutputPluginWriterPrepareWrite
LogicalOutputPluginWriterUpdateProgress
LogicalOutputPluginWriterWrite
LogicalRepBeginData
+LogicalRepClockSkewAction
LogicalRepCommitData
LogicalRepCommitPreparedTxnData
LogicalRepCtxStruct
--
1.8.3.1
[application/octet-stream] v8-0002-Add-CONFLICT-RESOLVERS-into-the-syntax-for-CREATE.patch (37.6K, ../../CAFPTHDZvACoqRJzrdd8BtexuTD8y3xaSo__y5Ep4u6CnNBspkg@mail.gmail.com/4-v8-0002-Add-CONFLICT-RESOLVERS-into-the-syntax-for-CREATE.patch)
download | inline diff:
From d5cabc6ea1b2c680efcf6012ec89b6da5525c8ea Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 31 Jul 2024 03:08:50 -0400
Subject: [PATCH v8 2/4] Add CONFLICT RESOLVERS into the syntax for CREATE and
ALTER SUBSCRIPTION
This patch provides support for configuring subscriptions with conflict resolvers
Syntax for CREATE SUBSCRIPTION:
CREATE SUBSCRIPTION <subname> CONNECTION <conninfo> PUBLICATION <pubname> CONFLICT RESOLVER
(conflict_type1 = resolver1, conflict_type2 = resolver2, conflict_type3 = resolver3,...);
Syntax for ALTER SUBSCRIPTION:
ALTER SUBSCRIPTION <subname> CONFLICT RESOLVER
(conflict_type1 = resolver1, conflict_type2 = resolver2, conflict_type3 = resolver3,...);
---
src/backend/commands/subscriptioncmds.c | 100 ++++++++
src/backend/parser/gram.y | 25 +-
src/backend/replication/logical/conflict.c | 308 +++++++++++++++++++++++++
src/include/catalog/Makefile | 3 +-
src/include/catalog/meson.build | 1 +
src/include/catalog/pg_subscription_conflict.h | 55 +++++
src/include/nodes/parsenodes.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/replication/conflict.h | 49 ++++
src/test/regress/expected/oidjoins.out | 1 +
src/test/regress/expected/subscription.out | 101 ++++++++
src/test/regress/sql/subscription.sql | 48 ++++
12 files changed, 691 insertions(+), 4 deletions(-)
create mode 100644 src/include/catalog/pg_subscription_conflict.h
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index b2bc095..5c18c45 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -28,6 +28,7 @@
#include "catalog/pg_database_d.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
+#include "catalog/pg_subscription_conflict.h"
#include "catalog/pg_type.h"
#include "commands/dbcommands.h"
#include "commands/defrem.h"
@@ -37,6 +38,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "pgstat.h"
+#include "replication/conflict.h"
#include "replication/logicallauncher.h"
#include "replication/logicalworker.h"
#include "replication/origin.h"
@@ -453,6 +455,32 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
/*
+ * Parsing function for conflict resolvers in CREATE SUBSCRIPTION command
+ */
+static void
+parse_subscription_conflict_resolvers(List *stmtresolvers,
+ ConflictTypeResolver * resolvers)
+{
+ ListCell *lc;
+
+ if (!stmtresolvers)
+ return;
+
+ foreach(lc, stmtresolvers)
+ {
+ DefElem *defel = (DefElem *) lfirst(lc);
+ ConflictType type;
+
+ /* validate the conflict type and resolver */
+ type = validate_conflict_type_and_resolver(defel->defname,
+ defGetString(defel));
+
+ /* update the corresponding resolver for the given conflict type */
+ resolvers[type].resolver = defGetString(defel);
+ }
+}
+
+/*
* Add publication names from the list to a string.
*/
static void
@@ -596,6 +624,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
bits32 supported_opts;
SubOpts opts = {0};
AclResult aclresult;
+ ConflictTypeResolver conflictResolvers[CT_MAX + 1];
+ bool skip_conflict_resolvers = false;
/*
* Parse and check options.
@@ -611,6 +641,33 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
+ if (!opts.detectconflict)
+ {
+ /*
+ * If conflict resolvers are set but detect_conflict is not enabled,
+ * issue warning and ignore resolvers.
+ */
+ if (stmt->resolvers)
+ ereport(WARNING,
+ (errmsg("Ignoring given CONFLICT RESOLVERS as detect_conflict is not enabled.")));
+ skip_conflict_resolvers = true;
+ }
+ else
+ {
+ /* If conflict resolvers are not set, use default values */
+ if (!stmt->resolvers)
+ {
+ ereport(WARNING,
+ (errmsg("Will use default resolvers configuration as detect_conflict is ON but resolvers are not given")));
+ }
+
+ /*
+ * Parse and check conflict resolvers. Initialize with default values
+ */
+ SetDefaultResolvers(conflictResolvers);
+ parse_subscription_conflict_resolvers(stmt->resolvers, conflictResolvers);
+ }
+
/*
* Since creating a replication slot is not transactional, rolling back
* the transaction leaves the created replication slot. So we cannot run
@@ -742,6 +799,10 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_create(originname);
+ /* Update the Conflict Resolvers in pg_subscription_conflict */
+ if (!skip_conflict_resolvers)
+ SetSubConflictResolver(subid, conflictResolvers, CT_MAX + 1);
+
/*
* Connect to remote side to execute requested commands and fetch table
* info.
@@ -1374,6 +1435,22 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
if (IsSet(opts.specified_opts, SUBOPT_DETECT_CONFLICT))
{
+ ConflictTypeResolver conflictResolvers[CT_MAX + 1];
+
+ if (!opts.detectconflict)
+ RemoveSubscriptionConflictBySubid(subid);
+ else
+ {
+ /* if detect_conflict is already set, then ignore */
+ if (!sub->detectconflict)
+ {
+ ereport(WARNING,
+ (errmsg("Using default conflict resolvers")));
+ SetDefaultResolvers(conflictResolvers);
+ SetSubConflictResolver(subid, conflictResolvers, CT_MAX + 1);
+ }
+ }
+
values[Anum_pg_subscription_subdetectconflict - 1] =
BoolGetDatum(opts.detectconflict);
replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
@@ -1604,6 +1681,26 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
update_tuple = true;
break;
}
+ case ALTER_SUBSCRIPTION_CONFLICT_RESOLVERS:
+ {
+ List *conflict_resolvers = NIL;
+
+ /* make sure that detect_conflict is enabled, else throw error */
+ if (!sub->detectconflict)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set conflict resolvers when detect_conflict is not enabled")));
+
+ /* get list of conflict types and resolvers and validate them */
+ conflict_resolvers = GetAndValidateSubsConflictResolverList(stmt->resolvers);
+
+ /*
+ * Update the conflict resolvers for the corresponding
+ * conflict types in the pg_subscription_conflict catalog
+ */
+ UpdateSubConflictResolvers(conflict_resolvers, subid);
+ break;
+ }
default:
elog(ERROR, "unrecognized ALTER SUBSCRIPTION kind %d",
@@ -1855,6 +1952,9 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
/* Remove any associated relation synchronization states. */
RemoveSubscriptionRel(subid, InvalidOid);
+ /* Remove any associated conflict resolvers */
+ RemoveSubscriptionConflictBySubid(subid);
+
/* Remove the origin tracking if exists. */
ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_drop_by_name(originname, true, false);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a043fd4..a690b46 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -426,7 +426,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
OptTableElementList TableElementList OptInherit definition
OptTypedTableElementList TypedTableElementList
reloptions opt_reloptions
- OptWith opt_definition func_args func_args_list
+ OptWith opt_definition opt_resolver_definition func_args func_args_list
func_args_with_defaults func_args_with_defaults_list
aggr_args aggr_args_list
func_as createfunc_opt_list opt_createfunc_opt_list alterfunc_opt_list
@@ -772,7 +772,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
RANGE READ REAL REASSIGN RECHECK RECURSIVE REF_P REFERENCES REFERENCING
REFRESH REINDEX RELATIVE_P RELEASE RENAME REPEATABLE REPLACE REPLICA
- RESET RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
+ RESET RESOLVER RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
ROUTINE ROUTINES ROW ROWS RULE
SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT
@@ -8824,6 +8824,11 @@ opt_definition:
| /*EMPTY*/ { $$ = NIL; }
;
+opt_resolver_definition:
+ CONFLICT RESOLVER definition { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
table_func_column: param_name func_type
{
FunctionParameter *n = makeNode(FunctionParameter);
@@ -10758,7 +10763,7 @@ AlterPublicationStmt:
*****************************************************************************/
CreateSubscriptionStmt:
- CREATE SUBSCRIPTION name CONNECTION Sconst PUBLICATION name_list opt_definition
+ CREATE SUBSCRIPTION name CONNECTION Sconst PUBLICATION name_list opt_definition opt_resolver_definition
{
CreateSubscriptionStmt *n =
makeNode(CreateSubscriptionStmt);
@@ -10766,6 +10771,7 @@ CreateSubscriptionStmt:
n->conninfo = $5;
n->publication = $7;
n->options = $8;
+ n->resolvers = $9;
$$ = (Node *) n;
}
;
@@ -10872,6 +10878,17 @@ AlterSubscriptionStmt:
n->options = $5;
$$ = (Node *) n;
}
+ | ALTER SUBSCRIPTION name opt_resolver_definition
+ {
+ AlterSubscriptionStmt *n =
+ makeNode(AlterSubscriptionStmt);
+
+ n->kind = ALTER_SUBSCRIPTION_CONFLICT_RESOLVERS;
+ n->subname = $3;
+ n->resolvers = $4;
+ $$ = (Node *) n;
+ }
+
;
/*****************************************************************************
@@ -17797,6 +17814,7 @@ unreserved_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
@@ -18428,6 +18446,7 @@ bare_label_keyword:
| REPLACE
| REPLICA
| RESET
+ | RESOLVER
| RESTART
| RESTRICT
| RETURN
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 4918011..720e923 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -15,9 +15,23 @@
#include "postgres.h"
#include "access/commit_ts.h"
+#include "access/heaptoast.h"
+#include "access/heapam.h"
+#include "access/table.h"
+#include "access/tableam.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_subscription.h"
+#include "catalog/pg_subscription_conflict.h"
+#include "catalog/pg_subscription_conflict_d.h"
+#include "catalog/pg_inherits.h"
+#include "commands/defrem.h"
#include "replication/conflict.h"
#include "replication/origin.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
#include "utils/rel.h"
const char *const ConflictTypeNames[] = {
@@ -29,6 +43,55 @@ const char *const ConflictTypeNames[] = {
[CT_DELETE_DIFFER] = "delete_differ"
};
+const char *const ConflictResolverNames[] = {
+ [CR_REMOTE_APPLY] = "remote_apply",
+ [CR_KEEP_LOCAL] = "keep_local",
+ [CR_APPLY_OR_SKIP] = "apply_or_skip",
+ [CR_APPLY_OR_ERROR] = "apply_or_error",
+ [CR_SKIP] = "skip",
+ [CR_ERROR] = "error"
+};
+
+#define CONFLICT_TYPE_MAX_RESOLVERS 4
+
+/*
+ * Valid conflict resolvers for each conflict type.
+ *
+ * XXX: If we do not want to maintain different resolvers such as
+ * apply_or_skip and apply_or_error for update_missing conflict,
+ * then we can retain remote_apply and keep_local only. Then these
+ * resolvers in context of update_missing will mean:
+ *
+ * keep_local: do not apply the update as INSERT.
+ * remote_apply: apply the update as INSERT. If we could not apply,
+ * then log and skip.
+ *
+ * Similarly SKIP can be replaced with KEEP_LOCAL for both update_missing
+ * and delete_missing conflicts. For missing rows, 'SKIP' sounds more user
+ * friendly name for a resolver and thus has been added here.
+ */
+const int ConflictTypeResolverMap[][CONFLICT_TYPE_MAX_RESOLVERS] = {
+ [CT_INSERT_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_EXISTS] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR},
+ [CT_UPDATE_MISSING] = {CR_APPLY_OR_SKIP, CR_APPLY_OR_ERROR, CR_SKIP, CR_ERROR},
+ [CT_DELETE_MISSING] = {CR_SKIP, CR_ERROR},
+ [CT_DELETE_DIFFER] = {CR_REMOTE_APPLY, CR_KEEP_LOCAL, CR_ERROR}
+};
+
+/*
+ * Default conflict resolver for each conflict type.
+ */
+const int ConflictTypeDefaultResolvers[] = {
+ [CT_INSERT_EXISTS] = CR_REMOTE_APPLY,
+ [CT_UPDATE_EXISTS] = CR_REMOTE_APPLY,
+ [CT_UPDATE_DIFFER] = CR_REMOTE_APPLY,
+ [CT_UPDATE_MISSING] = CR_APPLY_OR_SKIP,
+ [CT_DELETE_MISSING] = CR_SKIP,
+ [CT_DELETE_DIFFER] = CR_REMOTE_APPLY
+
+};
+
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
TransactionId localxmin,
@@ -191,3 +254,248 @@ build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
return conflict_row;
}
+
+/*
+ * Set default values for CONFLICT RESOLVERS for each conflict type
+ */
+void
+SetDefaultResolvers(ConflictTypeResolver * conflictResolvers)
+{
+ ConflictType type;
+
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ conflictResolvers[type].conflict_type = ConflictTypeNames[type];
+ conflictResolvers[type].resolver =
+ ConflictResolverNames[ConflictTypeDefaultResolvers[type]];
+ }
+}
+
+/*
+ * Validate the conflict type and resolver.
+ */
+ConflictType
+validate_conflict_type_and_resolver(const char *conflict_type,
+ const char *conflict_resolver)
+{
+ ConflictType type;
+ ConflictResolver resolver;
+ bool valid = false;
+ int i;
+
+ /* Check conflict type validity */
+ for (type = CT_MIN; type <= CT_MAX; type++)
+ {
+ if (strcmp(ConflictTypeNames[type], conflict_type) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict type", conflict_type));
+
+ /* Reset */
+ valid = false;
+
+ /* Check conflict resolver validity. */
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_resolver) == 0)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver", conflict_resolver));
+
+ /* Reset */
+ valid = false;
+
+ /* Check if conflict resolver is a valid one for the given conflict type */
+ for (i = 0; i < CONFLICT_TYPE_MAX_RESOLVERS; i++)
+ {
+ if (ConflictTypeResolverMap[type][i] == resolver)
+ {
+ valid = true;
+ break;
+ }
+ }
+
+ if (!valid)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("%s is not a valid conflict resolver for conflict type %s",
+ conflict_resolver,
+ conflict_type));
+ return type;
+
+}
+
+/*
+ * Extract the conflict type and conflict resolvers from the
+ * ALTER SUBSCRIPTION command and return a list of ConflictTypeResolver nodes.
+ */
+List *
+GetAndValidateSubsConflictResolverList(List *stmtresolvers)
+{
+ ListCell *lc;
+ ConflictTypeResolver *CTR = NULL;
+ List *res = NIL;
+
+ foreach(lc, stmtresolvers)
+ {
+ DefElem *defel = (DefElem *) lfirst(lc);
+
+ CTR = palloc(sizeof(ConflictTypeResolver));
+ CTR->conflict_type = defel->defname;
+ CTR->resolver = defGetString(defel);
+
+ /*
+ * Validate the conflict type and that the resolver is valid for that
+ * conflict type
+ */
+ validate_conflict_type_and_resolver(CTR->conflict_type, CTR->resolver);
+
+ res = lappend(res, CTR);
+ }
+
+ return res;
+}
+
+/*
+ * Update the Subscription's conflict resolver for a conflict type in
+ * pg_subscription_conflict system catalog
+ */
+void
+UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid)
+{
+ ListCell *lc;
+ Datum values[Natts_pg_subscription_conflict];
+ bool nulls[Natts_pg_subscription_conflict];
+ bool replaces[Natts_pg_subscription_conflict];
+ HeapTuple oldtup;
+ HeapTuple newtup = NULL;
+ Relation pg_subscription_conflict;
+
+ /* Prepare to update a tuple. */
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+ memset(values, 0, sizeof(values));
+
+ pg_subscription_conflict = table_open(SubscriptionConflictId, RowExclusiveLock);
+
+ foreach(lc, conflict_resolvers)
+ {
+ ConflictTypeResolver *CTR = (ConflictTypeResolver *) lfirst(lc);
+
+ /* set up subid and conflict_type to search in cache */
+ values[Anum_pg_subscription_conflict_confsubid - 1] = ObjectIdGetDatum(subid);
+ values[Anum_pg_subscription_conflict_confrtype - 1] = CStringGetTextDatum(CTR->conflict_type);
+
+ oldtup = SearchSysCache2(SUBSCRIPTIONCONFLICTSUBOID,
+ values[Anum_pg_subscription_conflict_confsubid - 1],
+ values[Anum_pg_subscription_conflict_confrtype - 1]);
+
+ if (HeapTupleIsValid(oldtup))
+ {
+ /* Update the new resolver */
+ values[Anum_pg_subscription_conflict_confrres - 1] = CStringGetTextDatum(CTR->resolver);
+ replaces[Anum_pg_subscription_conflict_confrres - 1] = true;
+
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(pg_subscription_conflict),
+ values, nulls, replaces);
+ CatalogTupleUpdate(pg_subscription_conflict, &oldtup->t_self, newtup);
+ ReleaseSysCache(oldtup);
+ heap_freetuple(newtup);
+ }
+ else
+ elog(ERROR, "cache lookup failed for table conflict %s for subid %u",
+ CTR->conflict_type, subid);
+
+ }
+
+ table_close(pg_subscription_conflict, RowExclusiveLock);
+}
+
+/*
+ * Set Conflict Resolvers on the subscription
+ */
+void
+SetSubConflictResolver(Oid subId, ConflictTypeResolver * resolvers, int resolvers_cnt)
+{
+ Relation pg_subscription_conflict;
+ Datum values[Natts_pg_subscription_conflict];
+ bool nulls[Natts_pg_subscription_conflict];
+ bool replaces[Natts_pg_subscription_conflict];
+ HeapTuple newtup = NULL;
+ int type;
+ Oid conflict_oid;
+
+ pg_subscription_conflict = table_open(SubscriptionConflictId, RowExclusiveLock);
+
+ for (type = 0; type < resolvers_cnt; type++)
+ {
+ /* Prepare to update a tuple. */
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_subscription_conflict_confsubid - 1] = ObjectIdGetDatum(subId);
+ values[Anum_pg_subscription_conflict_confrtype - 1] =
+ CStringGetTextDatum(resolvers[type].conflict_type);
+ values[Anum_pg_subscription_conflict_confrres - 1] =
+ CStringGetTextDatum(resolvers[type].resolver);
+
+ /* Get a new oid and update the tuple into catalog */
+ conflict_oid = GetNewOidWithIndex(pg_subscription_conflict, SubscriptionConflictOidIndexId,
+ Anum_pg_subscription_conflict_oid);
+ values[Anum_pg_subscription_conflict_oid - 1] = ObjectIdGetDatum(conflict_oid);
+ newtup = heap_form_tuple(RelationGetDescr(pg_subscription_conflict),
+ values, nulls);
+ CatalogTupleInsert(pg_subscription_conflict, newtup);
+ heap_freetuple(newtup);
+ }
+
+ table_close(pg_subscription_conflict, RowExclusiveLock);
+}
+
+/*
+ * Remove the subscription conflict resolvers for the subscription id
+ */
+void
+RemoveSubscriptionConflictBySubid(Oid subid)
+{
+ Relation rel;
+ HeapTuple tup;
+ TableScanDesc scan;
+ ScanKeyData skey[1];
+ int nkeys = 0;
+
+ rel = table_open(SubscriptionConflictId, RowExclusiveLock);
+
+ /*
+ * Search using the subid, this should return all conflict resolvers for
+ * this sub
+ */
+ ScanKeyInit(&skey[nkeys++],
+ Anum_pg_subscription_conflict_confsubid,
+ BTEqualStrategyNumber,
+ F_OIDEQ,
+ ObjectIdGetDatum(subid));
+
+ scan = table_beginscan_catalog(rel, nkeys, skey);
+
+ /* Iterate through the tuples and delete them */
+ while (HeapTupleIsValid(tup = heap_getnext(scan, ForwardScanDirection)))
+ CatalogTupleDelete(rel, &tup->t_self);
+
+ table_endscan(scan);
+ table_close(rel, RowExclusiveLock);
+}
diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile
index 167f91a..f2611c1 100644
--- a/src/include/catalog/Makefile
+++ b/src/include/catalog/Makefile
@@ -81,7 +81,8 @@ CATALOG_HEADERS := \
pg_publication_namespace.h \
pg_publication_rel.h \
pg_subscription.h \
- pg_subscription_rel.h
+ pg_subscription_rel.h \
+ pg_subscription_conflict.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h)
diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index f70d1da..959e1d9 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -69,6 +69,7 @@ catalog_headers = [
'pg_publication_rel.h',
'pg_subscription.h',
'pg_subscription_rel.h',
+ 'pg_subscription_conflict.h',
]
# The .dat files we need can just be listed alphabetically.
diff --git a/src/include/catalog/pg_subscription_conflict.h b/src/include/catalog/pg_subscription_conflict.h
new file mode 100644
index 0000000..c8b37c2
--- /dev/null
+++ b/src/include/catalog/pg_subscription_conflict.h
@@ -0,0 +1,55 @@
+/* -------------------------------------------------------------------------
+ *
+ * pg_subscription_conflict.h
+ * definition of the "subscription conflict resolver" system
+ * catalog (pg_subscription_conflict)
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_subscription_conflict.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ * -------------------------------------------------------------------------
+ */
+#ifndef PG_SUBSCRIPTION_CONFLICT_H
+#define PG_SUBSCRIPTION_CONFLICT_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_subscription_conflict_d.h"
+
+/* ----------------
+ * pg_subscription_conflict definition. cpp turns this into
+ * typedef struct FormData_pg_subscription_conflict
+ * ----------------
+ */
+CATALOG(pg_subscription_conflict,8881,SubscriptionConflictId)
+{
+ Oid oid; /* OID of the object itself */
+ Oid confsubid BKI_LOOKUP(pg_subscription); /* OID of subscription */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text confrtype BKI_FORCE_NOT_NULL; /* conflict type */
+ text confrres BKI_FORCE_NOT_NULL; /* conflict resolver */
+#endif
+} FormData_pg_subscription_conflict;
+
+/* ----------------
+ * Form_pg_subscription_conflict corresponds to a pointer to a row with
+ * the format of pg_subscription_conflict relation.
+ * ----------------
+ */
+typedef FormData_pg_subscription_conflict * Form_pg_subscription_conflict;
+
+DECLARE_TOAST(pg_subscription_conflict, 8882, 8883);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_subscription_conflict_oid_index, 8884, SubscriptionConflictOidIndexId, pg_subscription_conflict, btree(oid oid_ops));
+
+DECLARE_UNIQUE_INDEX(pg_subscription_conflict_sub_index, 8885, SubscriptionConflictSubIndexId, pg_subscription_conflict, btree(confsubid oid_ops, confrtype text_ops));
+
+MAKE_SYSCACHE(SUBSCRIPTIONCONFLICTSUBOID, pg_subscription_conflict_sub_index, 256);
+
+#endif /* PG_SUBSCRIPTION_CONFLICT_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 85a62b5..2e1eefc3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -4203,6 +4203,7 @@ typedef struct CreateSubscriptionStmt
char *conninfo; /* Connection string to publisher */
List *publication; /* One or more publication to subscribe to */
List *options; /* List of DefElem nodes */
+ List *resolvers; /* List of conflict resolvers */
} CreateSubscriptionStmt;
typedef enum AlterSubscriptionType
@@ -4215,6 +4216,7 @@ typedef enum AlterSubscriptionType
ALTER_SUBSCRIPTION_REFRESH,
ALTER_SUBSCRIPTION_ENABLED,
ALTER_SUBSCRIPTION_SKIP,
+ ALTER_SUBSCRIPTION_CONFLICT_RESOLVERS,
} AlterSubscriptionType;
typedef struct AlterSubscriptionStmt
@@ -4225,6 +4227,7 @@ typedef struct AlterSubscriptionStmt
char *conninfo; /* Connection string to publisher */
List *publication; /* One or more publication to subscribe to */
List *options; /* List of DefElem nodes */
+ List *resolvers; /* List of conflict resolvers */
} AlterSubscriptionStmt;
typedef struct DropSubscriptionStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f7fe834..21e5cb1 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -377,6 +377,7 @@ PG_KEYWORD("repeatable", REPEATABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replace", REPLACE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replica", REPLICA, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("reset", RESET, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("resolver", RESOLVER, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restart", RESTART, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("restrict", RESTRICT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("return", RETURN, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index 3a7260d..c102f74 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -39,6 +39,47 @@ typedef enum
CT_DELETE_DIFFER,
} ConflictType;
+/* Min and max conflict type */
+#define CT_MIN CT_INSERT_EXISTS
+#define CT_MAX CT_DELETE_DIFFER
+
+/*
+ * Conflict resolvers that can be used to resolve various conflicts.
+ *
+ * See ConflictTypeResolverMap in conflcit.c to find out which all
+ * resolvers are supported for each conflict type.
+ */
+typedef enum ConflictResolver
+{
+ /* Apply the remote change */
+ CR_REMOTE_APPLY = 1,
+
+ /* Keep the local change */
+ CR_KEEP_LOCAL,
+
+ /* Apply the remote change; skip if it can not be applied */
+ CR_APPLY_OR_SKIP,
+
+ /* Apply the remote change; emit error if it can not be applied */
+ CR_APPLY_OR_ERROR,
+
+ /* Skip applying the change */
+ CR_SKIP,
+
+ /* Error out */
+ CR_ERROR,
+} ConflictResolver;
+
+/* Min and max conflict resolver */
+#define CR_MIN CR_REMOTE_APPLY
+#define CR_MAX CR_ERROR
+
+typedef struct ConflictTypeResolver
+{
+ const char *conflict_type;
+ const char *resolver;
+} ConflictTypeResolver;
+
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
extern void ReportApplyConflict(int elevel, ConflictType type,
@@ -46,5 +87,13 @@ extern void ReportApplyConflict(int elevel, ConflictType type,
TransactionId localxmin, RepOriginId localorigin,
TimestampTz localts, TupleTableSlot *conflictslot);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
+extern void SetSubConflictResolver(Oid subId, ConflictTypeResolver * resolvers, int max_types);
+extern void RemoveSubscriptionConflictById(Oid confid);
+extern void RemoveSubscriptionConflictBySubid(Oid confid);
+extern List *GetAndValidateSubsConflictResolverList(List *stmtresolvers);
+extern void UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid);
+extern ConflictType validate_conflict_type_and_resolver(const char *conflict_type,
+ const char *conflict_resolver);
+extern void SetDefaultResolvers(ConflictTypeResolver * conflictResolvers);
#endif
diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out
index 215eb89..42bf2cc 100644
--- a/src/test/regress/expected/oidjoins.out
+++ b/src/test/regress/expected/oidjoins.out
@@ -266,3 +266,4 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid}
NOTICE: checking pg_subscription {subowner} => pg_authid {oid}
NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid}
NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid}
+NOTICE: checking pg_subscription_conflict {confsubid} => pg_subscription {oid}
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index cc9337c..7bd4e82 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -430,6 +430,7 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
ERROR: detect_conflict requires a Boolean value
-- now it works
CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: Will use default resolvers configuration as detect_conflict is ON but resolvers are not given
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
@@ -439,6 +440,18 @@ HINT: To initiate replication, you must manually create the replication slot, e
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | t | off | dbname=regress_doesnotexist | 0/0
(1 row)
+-- confirm that the default conflict resolvers have been set
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+----------------+---------------
+ delete_differ | remote_apply
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | remote_apply
+ update_exists | remote_apply
+ update_missing | apply_or_skip
+(6 rows)
+
ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
\dRs+
List of subscriptions
@@ -447,6 +460,94 @@ ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
+-- confirm that the conflict resolvers have been dropped
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+-----------+----------
+(0 rows)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail - invalid conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = foo);
+ERROR: foo is not a valid conflict resolver
+-- fail - invalid conflict types
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (foo = 'keep_local');
+ERROR: foo is not a valid conflict type
+-- creating subscription with detect_conflict = false should not create conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+-----------+----------
+(0 rows)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+--try setting resolvers for few types
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = 'keep_local', update_missing = 'skip', delete_differ = 'keep_local' );
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+
+--check if above are configured; for non specified conflict types, default resolvers should be seen
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+----------------+--------------
+ delete_differ | keep_local
+ delete_missing | skip
+ insert_exists | keep_local
+ update_differ | remote_apply
+ update_exists | remote_apply
+ update_missing | skip
+(6 rows)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- creating subscription with detect_conflict = false but conflict resolvers specified
+-- will result in conflict resolvers being ignored
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false) CONFLICT RESOLVER (insert_exists = 'keep_local');
+WARNING: Ignoring given CONFLICT RESOLVERS as detect_conflict is not enabled.
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+-----------+----------
+(0 rows)
+
+-- setting detect_conflict to true will set default conflict resolvers
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = true);
+WARNING: Using default conflict resolvers
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+----------------+---------------
+ delete_differ | remote_apply
+ delete_missing | skip
+ insert_exists | remote_apply
+ update_differ | remote_apply
+ update_exists | remote_apply
+ update_missing | apply_or_skip
+(6 rows)
+
+-- fail - altering with invalid conflict type
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (foo = 'keep_local');
+ERROR: foo is not a valid conflict type
+-- fail - altering with invalid conflict resolver
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'foo');
+ERROR: foo is not a valid conflict resolver
+-- ok - valid conflict type and resolver
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'keep_local', update_missing = 'skip', delete_differ = 'keep_local' );
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ confrtype | confrres
+----------------+--------------
+ delete_differ | keep_local
+ delete_missing | skip
+ insert_exists | keep_local
+ update_differ | remote_apply
+ update_exists | remote_apply
+ update_missing | skip
+(6 rows)
+
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
-- let's do some tests with pg_create_subscription rather than superuser
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 5c740fd..bdd2118 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -295,10 +295,58 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
\dRs+
+-- confirm that the default conflict resolvers have been set
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
\dRs+
+-- confirm that the conflict resolvers have been dropped
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
+-- fail - invalid conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = foo);
+
+-- fail - invalid conflict types
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (foo = 'keep_local');
+
+-- creating subscription with detect_conflict = false should not create conflict resolvers
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false);
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
+--try setting resolvers for few types
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true) CONFLICT RESOLVER (insert_exists = 'keep_local', update_missing = 'skip', delete_differ = 'keep_local' );
+
+--check if above are configured; for non specified conflict types, default resolvers should be seen
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
+-- creating subscription with detect_conflict = false but conflict resolvers specified
+-- will result in conflict resolvers being ignored
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = false) CONFLICT RESOLVER (insert_exists = 'keep_local');
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
+-- setting detect_conflict to true will set default conflict resolvers
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = true);
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
+-- fail - altering with invalid conflict type
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (foo = 'keep_local');
+
+-- fail - altering with invalid conflict resolver
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'foo');
+
+-- ok - valid conflict type and resolver
+ALTER SUBSCRIPTION regress_testsub CONFLICT RESOLVER (insert_exists = 'keep_local', update_missing = 'skip', delete_differ = 'keep_local' );
+SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype;
+
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
--
1.8.3.1
[application/octet-stream] v8-0001-Detect-and-log-conflicts-in-logical-replication.patch (98.6K, ../../CAFPTHDZvACoqRJzrdd8BtexuTD8y3xaSo__y5Ep4u6CnNBspkg@mail.gmail.com/5-v8-0001-Detect-and-log-conflicts-in-logical-replication.patch)
download | inline diff:
From 65e60fa35df180ccf781363d6c2d363c0be36637 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 31 Jul 2024 03:01:17 -0400
Subject: [PATCH v8 1/4] Detect and log conflicts in logical replication
This patch adds a new parameter detect_conflict for CREATE and ALTER
subscription commands. This new parameter will decide if subscription will
go for confict detection. By default, conflict detection will be off for a
subscription.
When conflict detection is enabled, additional logging is triggered in the
following conflict scenarios:
insert_exists: Inserting a row that violates a NOT DEFERRABLE unique constraint.
update_exists: The updated row value violates a NOT DEFERRABLE unique constraint.
update_differ: Updating a row that was previously modified by another origin.
update_missing: The tuple to be updated is missing.
delete_missing: The tuple to be deleted is missing.
delete_differ: Deleting a row that was previously modified by another origin.
For insert_exists conflict, the log can include origin and commit
timestamp details of the conflicting key with track_commit_timestamp
enabled.
update_differ and delete_differ conflicts can only be detected when
track_commit_timestamp is enabled.
---
doc/src/sgml/catalogs.sgml | 9 ++
doc/src/sgml/logical-replication.sgml | 21 ++-
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 84 ++++++++++++
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 3 +-
src/backend/commands/subscriptioncmds.c | 31 ++++-
src/backend/executor/execIndexing.c | 14 +-
src/backend/executor/execReplication.c | 142 +++++++++++++++++++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/conflict.c | 193 ++++++++++++++++++++++++++++
src/backend/replication/logical/meson.build | 1 +
src/backend/replication/logical/worker.c | 84 +++++++++---
src/bin/pg_dump/pg_dump.c | 17 ++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 6 +-
src/bin/psql/tab-complete.c | 14 +-
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/conflict.h | 50 +++++++
src/test/regress/expected/subscription.out | 176 ++++++++++++++-----------
src/test/regress/sql/subscription.sql | 15 +++
src/test/subscription/t/001_rep_changes.pl | 15 ++-
src/test/subscription/t/013_partition.pl | 68 ++++++----
src/test/subscription/t/029_on_error.pl | 5 +-
src/test/subscription/t/030_origin.pl | 43 +++++++
src/tools/pgindent/typedefs.list | 1 +
26 files changed, 849 insertions(+), 155 deletions(-)
create mode 100644 src/backend/replication/logical/conflict.c
create mode 100644 src/include/replication/conflict.h
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b654fae..b042a5a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8037,6 +8037,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subdetectconflict</structfield> <type>bool</type>
+ </para>
+ <para>
+ If true, the subscription is enabled for conflict detection.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>subconninfo</structfield> <type>text</type>
</para>
<para>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index a23a3d5..d236d85 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1580,7 +1580,8 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
stop. This is referred to as a <firstterm>conflict</firstterm>. When
replicating <command>UPDATE</command> or <command>DELETE</command>
operations, missing data will not produce a conflict and such operations
- will simply be skipped.
+ will simply be skipped. Please refer to <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ for all the conflicts that will be logged when enabling <literal>detect_conflict</literal>.
</para>
<para>
@@ -1649,6 +1650,24 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
linkend="sql-altersubscription"><command>ALTER SUBSCRIPTION ...
SKIP</command></link>.
</para>
+
+ <para>
+ Enabling both <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>
+ and <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ on the subscriber can provide additional details regarding conflicting
+ rows, such as their origin and commit timestamp, in case of a unique
+ constraint violation conflict:
+<screen>
+ERROR: conflict insert_exists detected on relation "public.t"
+DETAIL: Key (a)=(1) already exists in unique index "t_pkey", which was modified by origin 0 in transaction 740 at 2024-06-26 10:47:04.727375+08.
+CONTEXT: processing remote data for replication origin "pg_16389" during message type "INSERT" for replication target relation "public.t" in transaction 740, finished at 0/14F7EC0
+</screen>
+ Users can use these information to make decisions on whether to retain
+ the local change or adopt the remote alteration. For instance, the
+ origin in above log indicates that the existing row was modified by a
+ local change, users can manually perform a remote-change-win resolution
+ by deleting the local row.
+ </para>
</sect1>
<sect1 id="logical-replication-restrictions">
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index fdc648d..dfbe25b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -235,8 +235,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<link linkend="sql-createsubscription-params-with-password-required"><literal>password_required</literal></link>,
<link linkend="sql-createsubscription-params-with-run-as-owner"><literal>run_as_owner</literal></link>,
<link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>,
- <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>, and
- <link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>.
+ <link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>,
+ <link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>, and
+ <link linkend="sql-createsubscription-params-with-detect-conflict"><literal>detect_conflict</literal></link>.
Only a superuser can set <literal>password_required = false</literal>.
</para>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 740b7d94..eb51805 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -428,6 +428,90 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createsubscription-params-with-detect-conflict">
+ <term><literal>detect_conflict</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the subscription is enabled for conflict detection.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ When conflict detection is enabled, additional logging is triggered
+ in the following scenarios:
+ <variablelist>
+ <varlistentry>
+ <term><literal>insert_exists</literal></term>
+ <listitem>
+ <para>
+ Inserting a row that violates a <literal>NOT DEFERRABLE</literal>
+ unique constraint. Note that to obtain the origin and commit
+ timestamp details of the conflicting key in the log, ensure that
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. In this scenario, an error will be raised until the
+ conflict is resolved manually.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_exists</literal></term>
+ <listitem>
+ <para>
+ The updated value of a row violates a <literal>NOT DEFERRABLE</literal>
+ unique constraint. Note that to obtain the origin and commit
+ timestamp details of the conflicting key in the log, ensure that
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. In this scenario, an error will be raised until the
+ conflict is resolved manually.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_differ</literal></term>
+ <listitem>
+ <para>
+ Updating a row that was previously modified by another origin.
+ Note that this conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. Currenly, the update is always applied regardless of
+ the origin of the local row.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>update_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be updated was not found. The update will simply be
+ skipped in this scenario.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_missing</literal></term>
+ <listitem>
+ <para>
+ The tuple to be deleted was not found. The delete will simply be
+ skipped in this scenario.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>delete_differ</literal></term>
+ <listitem>
+ <para>
+ Deleting a row that was previously modified by another origin. Note that this
+ conflict can only be detected when
+ <link linkend="guc-track-commit-timestamp"><varname>track_commit_timestamp</varname></link>
+ is enabled. Currenly, the delete is always applied regardless of
+ the origin of the local row.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist></para>
</listitem>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc915..5a423f4 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -72,6 +72,7 @@ GetSubscription(Oid subid, bool missing_ok)
sub->passwordrequired = subform->subpasswordrequired;
sub->runasowner = subform->subrunasowner;
sub->failover = subform->subfailover;
+ sub->detectconflict = subform->subdetectconflict;
/* Get conninfo */
datum = SysCacheGetAttrNotNull(SUBSCRIPTIONOID,
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 19cabc9..d084bfc 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1356,7 +1356,8 @@ REVOKE ALL ON pg_subscription FROM public;
GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
subbinary, substream, subtwophasestate, subdisableonerr,
subpasswordrequired, subrunasowner, subfailover,
- subslotname, subsynccommit, subpublications, suborigin)
+ subdetectconflict, subslotname, subsynccommit,
+ subpublications, suborigin)
ON pg_subscription TO public;
CREATE VIEW pg_stat_subscription_stats AS
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index d124bfe..b2bc095 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -71,8 +71,9 @@
#define SUBOPT_PASSWORD_REQUIRED 0x00000800
#define SUBOPT_RUN_AS_OWNER 0x00001000
#define SUBOPT_FAILOVER 0x00002000
-#define SUBOPT_LSN 0x00004000
-#define SUBOPT_ORIGIN 0x00008000
+#define SUBOPT_DETECT_CONFLICT 0x00004000
+#define SUBOPT_LSN 0x00008000
+#define SUBOPT_ORIGIN 0x00010000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -98,6 +99,7 @@ typedef struct SubOpts
bool passwordrequired;
bool runasowner;
bool failover;
+ bool detectconflict;
char *origin;
XLogRecPtr lsn;
} SubOpts;
@@ -162,6 +164,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->runasowner = false;
if (IsSet(supported_opts, SUBOPT_FAILOVER))
opts->failover = false;
+ if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT))
+ opts->detectconflict = false;
if (IsSet(supported_opts, SUBOPT_ORIGIN))
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
@@ -307,6 +311,15 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->specified_opts |= SUBOPT_FAILOVER;
opts->failover = defGetBoolean(defel);
}
+ else if (IsSet(supported_opts, SUBOPT_DETECT_CONFLICT) &&
+ strcmp(defel->defname, "detect_conflict") == 0)
+ {
+ if (IsSet(opts->specified_opts, SUBOPT_DETECT_CONFLICT))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_DETECT_CONFLICT;
+ opts->detectconflict = defGetBoolean(defel);
+ }
else if (IsSet(supported_opts, SUBOPT_ORIGIN) &&
strcmp(defel->defname, "origin") == 0)
{
@@ -594,7 +607,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_SYNCHRONOUS_COMMIT | SUBOPT_BINARY |
SUBOPT_STREAMING | SUBOPT_TWOPHASE_COMMIT |
SUBOPT_DISABLE_ON_ERR | SUBOPT_PASSWORD_REQUIRED |
- SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER | SUBOPT_ORIGIN);
+ SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -701,6 +715,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
values[Anum_pg_subscription_subpasswordrequired - 1] = BoolGetDatum(opts.passwordrequired);
values[Anum_pg_subscription_subrunasowner - 1] = BoolGetDatum(opts.runasowner);
values[Anum_pg_subscription_subfailover - 1] = BoolGetDatum(opts.failover);
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
values[Anum_pg_subscription_subconninfo - 1] =
CStringGetTextDatum(conninfo);
if (opts.slot_name)
@@ -1196,7 +1212,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
SUBOPT_DISABLE_ON_ERR |
SUBOPT_PASSWORD_REQUIRED |
SUBOPT_RUN_AS_OWNER | SUBOPT_FAILOVER |
- SUBOPT_ORIGIN);
+ SUBOPT_DETECT_CONFLICT | SUBOPT_ORIGIN);
parse_subscription_options(pstate, stmt->options,
supported_opts, &opts);
@@ -1356,6 +1372,13 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
replaces[Anum_pg_subscription_subfailover - 1] = true;
}
+ if (IsSet(opts.specified_opts, SUBOPT_DETECT_CONFLICT))
+ {
+ values[Anum_pg_subscription_subdetectconflict - 1] =
+ BoolGetDatum(opts.detectconflict);
+ replaces[Anum_pg_subscription_subdetectconflict - 1] = true;
+ }
+
if (IsSet(opts.specified_opts, SUBOPT_ORIGIN))
{
values[Anum_pg_subscription_suborigin - 1] =
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9f05b36..ef52277 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -207,8 +207,9 @@ ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative)
ii = BuildIndexInfo(indexDesc);
/*
- * If the indexes are to be used for speculative insertion, add extra
- * information required by unique index entries.
+ * If the indexes are to be used for speculative insertion or conflict
+ * detection in logical replication, add extra information required by
+ * unique index entries.
*/
if (speculative && ii->ii_Unique)
BuildSpeculativeIndexInfo(indexDesc, ii);
@@ -521,6 +522,11 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
* possible that a conflicting tuple is inserted immediately
* after this returns. But this can be used for a pre-check
* before insertion.
+ *
+ * If the 'slot' holds a tuple with valid tid, this tuple will
+ * be ignored when checking conflict. This can help in scenarios
+ * where we want to re-check for conflicts after inserting a
+ * tuple.
* ----------------------------------------------------------------
*/
bool
@@ -536,11 +542,9 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
ExprContext *econtext;
Datum values[INDEX_MAX_KEYS];
bool isnull[INDEX_MAX_KEYS];
- ItemPointerData invalidItemPtr;
bool checkedIndex = false;
ItemPointerSetInvalid(conflictTid);
- ItemPointerSetInvalid(&invalidItemPtr);
/*
* Get information from the result relation info structure.
@@ -629,7 +633,7 @@ ExecCheckIndexConstraints(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
satisfiesConstraint =
check_exclusion_or_unique_constraint(heapRelation, indexRelation,
- indexInfo, &invalidItemPtr,
+ indexInfo, &slot->tts_tid,
values, isnull, estate, false,
CEOUC_WAIT, true,
conflictTid);
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index d0a89cd..0680bc8 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -23,6 +23,7 @@
#include "commands/trigger.h"
#include "executor/executor.h"
#include "executor/nodeModifyTable.h"
+#include "replication/conflict.h"
#include "replication/logicalrelation.h"
#include "storage/lmgr.h"
#include "utils/builtins.h"
@@ -481,6 +482,121 @@ retry:
}
/*
+ * Find the tuple that violates the passed in unique index constraint
+ * (conflictindex).
+ *
+ * Return false if there is no conflict and *conflictslot is set to NULL.
+ * Otherwise return true, and the conflicting tuple is locked and returned
+ * in *conflictslot.
+ */
+static bool
+FindConflictTuple(ResultRelInfo *resultRelInfo, EState *estate,
+ Oid conflictindex, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot)
+{
+ Relation rel = resultRelInfo->ri_RelationDesc;
+ ItemPointerData conflictTid;
+ TM_FailureData tmfd;
+ TM_Result res;
+
+ *conflictslot = NULL;
+
+retry:
+ if (ExecCheckIndexConstraints(resultRelInfo, slot, estate,
+ &conflictTid, list_make1_oid(conflictindex)))
+ {
+ if (*conflictslot)
+ ExecDropSingleTupleTableSlot(*conflictslot);
+
+ *conflictslot = NULL;
+ return false;
+ }
+
+ *conflictslot = table_slot_create(rel, NULL);
+
+ PushActiveSnapshot(GetLatestSnapshot());
+
+ res = table_tuple_lock(rel, &conflictTid, GetLatestSnapshot(),
+ *conflictslot,
+ GetCurrentCommandId(false),
+ LockTupleShare,
+ LockWaitBlock,
+ 0 /* don't follow updates */ ,
+ &tmfd);
+
+ PopActiveSnapshot();
+
+ switch (res)
+ {
+ case TM_Ok:
+ break;
+ case TM_Updated:
+ /* XXX: Improve handling here */
+ if (ItemPointerIndicatesMovedPartitions(&tmfd.ctid))
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("tuple to be locked was already moved to another partition due to concurrent update, retrying")));
+ else
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent update, retrying")));
+ goto retry;
+ case TM_Deleted:
+ /* XXX: Improve handling here */
+ ereport(LOG,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("concurrent delete, retrying")));
+ goto retry;
+ case TM_Invisible:
+ elog(ERROR, "attempted to lock invisible tuple");
+ break;
+ default:
+ elog(ERROR, "unexpected table_tuple_lock status: %u", res);
+ break;
+ }
+
+ return true;
+}
+
+/*
+ * Re-check all the unique indexes in 'recheckIndexes' to see if there are
+ * potential conflicts with the tuple in 'slot'.
+ *
+ * This function is invoked after inserting or updating a tuple that detected
+ * potential conflict tuples. It attempts to find the tuple that conflicts with
+ * the provided tuple. This operation may seem redundant with the unique
+ * violation check of indexam, but since we call this function only when we are
+ * detecting conflict in logical replication and encountering potential
+ * conflicts with any unique index constraints (which should not be frequent),
+ * so it's ok. Moreover, upon detecting a conflict, we will report an ERROR and
+ * restart the logical replication, so the additional cost of finding the tuple
+ * should be acceptable.
+ */
+static void
+ReCheckConflictIndexes(ResultRelInfo *resultRelInfo, EState *estate,
+ ConflictType type, List *recheckIndexes,
+ TupleTableSlot *slot)
+{
+ /* Re-check all the unique indexes for potential conflicts */
+ foreach_oid(uniqueidx, resultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ TupleTableSlot *conflictslot;
+
+ if (list_member_oid(recheckIndexes, uniqueidx) &&
+ FindConflictTuple(resultRelInfo, estate, uniqueidx, slot, &conflictslot))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+
+ GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
+ ReportApplyConflict(ERROR, type, resultRelInfo->ri_RelationDesc, uniqueidx,
+ xmin, origin, committs, conflictslot);
+ }
+ }
+}
+
+/*
* Insert tuple represented in the slot to the relation, update the indexes,
* and execute any constraints and per-row triggers.
*
@@ -509,6 +625,8 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
+ List *conflictindexes;
+ bool conflict = false;
/* Compute stored generated columns */
if (rel->rd_att->constr &&
@@ -525,10 +643,17 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
- slot, estate, false, false,
- NULL, NIL, false);
+ slot, estate, false,
+ conflictindexes, &conflict,
+ conflictindexes, false);
+
+ if (conflict)
+ ReCheckConflictIndexes(resultRelInfo, estate, CT_INSERT_EXISTS,
+ recheckIndexes, slot);
/* AFTER ROW INSERT Triggers */
ExecARInsertTriggers(estate, resultRelInfo, slot,
@@ -577,6 +702,8 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
{
List *recheckIndexes = NIL;
TU_UpdateIndexes update_indexes;
+ List *conflictindexes;
+ bool conflict = false;
/* Compute stored generated columns */
if (rel->rd_att->constr &&
@@ -593,12 +720,19 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
&update_indexes);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
if (resultRelInfo->ri_NumIndices > 0 && (update_indexes != TU_None))
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
- slot, estate, true, false,
- NULL, NIL,
+ slot, estate, true,
+ conflictindexes,
+ &conflict, conflictindexes,
(update_indexes == TU_Summarizing));
+ if (conflict)
+ ReCheckConflictIndexes(resultRelInfo, estate, CT_UPDATE_EXISTS,
+ recheckIndexes, slot);
+
/* AFTER ROW UPDATE Triggers */
ExecARUpdateTriggers(estate, resultRelInfo,
NULL, NULL,
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index ba03eef..1e08bbb 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
applyparallelworker.o \
+ conflict.o \
decode.o \
launcher.o \
logical.o \
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
new file mode 100644
index 0000000..4918011
--- /dev/null
+++ b/src/backend/replication/logical/conflict.c
@@ -0,0 +1,193 @@
+/*-------------------------------------------------------------------------
+ * conflict.c
+ * Functionality for detecting and logging conflicts.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/conflict.c
+ *
+ * This file contains the code for detecting and logging conflicts on
+ * the subscriber during logical replication.
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/commit_ts.h"
+#include "replication/conflict.h"
+#include "replication/origin.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+
+const char *const ConflictTypeNames[] = {
+ [CT_INSERT_EXISTS] = "insert_exists",
+ [CT_UPDATE_EXISTS] = "update_exists",
+ [CT_UPDATE_DIFFER] = "update_differ",
+ [CT_UPDATE_MISSING] = "update_missing",
+ [CT_DELETE_MISSING] = "delete_missing",
+ [CT_DELETE_DIFFER] = "delete_differ"
+};
+
+static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
+static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin,
+ RepOriginId localorigin,
+ TimestampTz localts,
+ TupleTableSlot *conflictslot);
+
+/*
+ * Get the xmin and commit timestamp data (origin and timestamp) associated
+ * with the provided local tuple.
+ *
+ * Return true if the commit timestamp data was found, false otherwise.
+ */
+bool
+GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts)
+{
+ Datum xminDatum;
+ bool isnull;
+
+ xminDatum = slot_getsysattr(localslot, MinTransactionIdAttributeNumber,
+ &isnull);
+ *xmin = DatumGetTransactionId(xminDatum);
+ Assert(!isnull);
+
+ /*
+ * The commit timestamp data is not available if track_commit_timestamp is
+ * disabled.
+ */
+ if (!track_commit_timestamp)
+ {
+ *localorigin = InvalidRepOriginId;
+ *localts = 0;
+ return false;
+ }
+
+ return TransactionIdGetCommitTsData(*xmin, localts, localorigin);
+}
+
+/*
+ * Report a conflict when applying remote changes.
+ */
+void
+ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot)
+{
+ ereport(elevel,
+ errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
+ errmsg("conflict %s detected on relation \"%s.%s\"",
+ ConflictTypeNames[type],
+ get_namespace_name(RelationGetNamespace(localrel)),
+ RelationGetRelationName(localrel)),
+ errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
+ localts, conflictslot));
+}
+
+/*
+ * Find all unique indexes to check for a conflict and store them into
+ * ResultRelInfo.
+ */
+void
+InitConflictIndexes(ResultRelInfo *relInfo)
+{
+ List *uniqueIndexes = NIL;
+
+ for (int i = 0; i < relInfo->ri_NumIndices; i++)
+ {
+ Relation indexRelation = relInfo->ri_IndexRelationDescs[i];
+
+ if (indexRelation == NULL)
+ continue;
+
+ /* Detect conflict only for unique indexes */
+ if (!relInfo->ri_IndexRelationInfo[i]->ii_Unique)
+ continue;
+
+ /* Don't support conflict detection for deferrable index */
+ if (!indexRelation->rd_index->indimmediate)
+ continue;
+
+ uniqueIndexes = lappend_oid(uniqueIndexes,
+ RelationGetRelid(indexRelation));
+ }
+
+ relInfo->ri_onConflictArbiterIndexes = uniqueIndexes;
+}
+
+/*
+ * Add an errdetail() line showing conflict detail.
+ */
+static int
+errdetail_apply_conflict(ConflictType type, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot)
+{
+ switch (type)
+ {
+ case CT_INSERT_EXISTS:
+ case CT_UPDATE_EXISTS:
+ {
+ /*
+ * Bulid the index value string. If the return value is NULL,
+ * it indicates that the current user lacks permissions to
+ * view all the columns involved.
+ */
+ char *index_value = build_index_value_desc(conflictidx,
+ conflictslot);
+
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
+ case CT_UPDATE_DIFFER:
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ case CT_UPDATE_MISSING:
+ return errdetail("Did not find the row to be updated.");
+ case CT_DELETE_MISSING:
+ return errdetail("Did not find the row to be deleted.");
+ case CT_DELETE_DIFFER:
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
+ localorigin, localxmin, timestamptz_to_str(localts));
+ }
+
+ return 0; /* silence compiler warning */
+}
+
+/*
+ * Helper functions to construct a string describing the contents of an index
+ * entry. See BuildIndexValueDescription for details.
+ */
+static char *
+build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot)
+{
+ char *conflict_row;
+ Relation indexDesc;
+
+ if (!conflictslot)
+ return NULL;
+
+ /* Assume the index has been locked */
+ indexDesc = index_open(indexoid, NoLock);
+
+ slot_getallattrs(conflictslot);
+
+ conflict_row = BuildIndexValueDescription(indexDesc,
+ conflictslot->tts_values,
+ conflictslot->tts_isnull);
+
+ index_close(indexDesc, NoLock);
+
+ return conflict_row;
+}
diff --git a/src/backend/replication/logical/meson.build b/src/backend/replication/logical/meson.build
index 3dec36a..3d36249 100644
--- a/src/backend/replication/logical/meson.build
+++ b/src/backend/replication/logical/meson.build
@@ -2,6 +2,7 @@
backend_sources += files(
'applyparallelworker.c',
+ 'conflict.c',
'decode.c',
'launcher.c',
'logical.c',
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index ec96b5f..fc3f80e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -167,6 +167,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
+#include "replication/conflict.h"
#include "replication/logicallauncher.h"
#include "replication/logicalproto.h"
#include "replication/logicalrelation.h"
@@ -2458,7 +2459,10 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
EState *estate = edata->estate;
/* We must open indexes here. */
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, MySubscription->detectconflict);
+
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
@@ -2646,7 +2650,7 @@ apply_handle_update_internal(ApplyExecutionData *edata,
MemoryContext oldctx;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(relinfo, false);
+ ExecOpenIndices(relinfo, MySubscription->detectconflict);
found = FindReplTupleInLocalRel(edata, localrel,
&relmapentry->remoterel,
@@ -2661,6 +2665,20 @@ apply_handle_update_internal(ApplyExecutionData *edata,
*/
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
/* Process and store remote tuple in the slot */
oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
slot_modify_data(remoteslot, localslot, relmapentry, newtup);
@@ -2668,6 +2686,9 @@ apply_handle_update_internal(ApplyExecutionData *edata,
EvalPlanQualSetSlot(&epqstate, remoteslot);
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
+
/* Do the actual update. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
@@ -2678,13 +2699,10 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2807,6 +2825,20 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/* If found delete it. */
if (found)
{
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
+
+ /*
+ * If conflict detection is enabled, check whether the local tuple was
+ * modified by a different origin. If detected, report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
+ localxmin, localorigin, localts, NULL);
+
EvalPlanQualSetSlot(&epqstate, localslot);
/* Do the actual delete. */
@@ -2818,13 +2850,10 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
/*
* The tuple to be deleted could not be found. Do nothing except for
* emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be deleted "
- "in replication target relation \"%s\"",
- RelationGetRelationName(localrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
+ InvalidTransactionId, InvalidRepOriginId, 0, NULL);
}
/* Cleanup. */
@@ -2991,6 +3020,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ResultRelInfo *partrelinfo_new;
Relation partrel_new;
bool found;
+ RepOriginId localorigin;
+ TransactionId localxmin;
+ TimestampTz localts;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3002,17 +3034,29 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
/*
* The tuple to be updated could not be found. Do nothing
* except for emitting a log message.
- *
- * XXX should this be promoted to ereport(LOG) perhaps?
*/
- elog(DEBUG1,
- "logical replication did not find row to be updated "
- "in replication target relation's partition \"%s\"",
- RelationGetRelationName(partrel));
+ if (MySubscription->detectconflict)
+ ReportApplyConflict(LOG, CT_UPDATE_MISSING,
+ partrel, InvalidOid,
+ InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL);
+
return;
}
/*
+ * If conflict detection is enabled, check whether the local
+ * tuple was modified by a different origin. If detected,
+ * report the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
+ InvalidOid, localxmin, localorigin,
+ localts, NULL);
+
+ /*
* Apply the update to the local tuple, putting the result in
* remoteslot_part.
*/
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2b02148..0f72d28 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4760,6 +4760,7 @@ getSubscriptions(Archive *fout)
int i_suboriginremotelsn;
int i_subenabled;
int i_subfailover;
+ int i_subdetectconflict;
int i,
ntups;
@@ -4832,11 +4833,17 @@ getSubscriptions(Archive *fout)
if (fout->remoteVersion >= 170000)
appendPQExpBufferStr(query,
- " s.subfailover\n");
+ " s.subfailover,\n");
else
appendPQExpBuffer(query,
- " false AS subfailover\n");
+ " false AS subfailover,\n");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ " s.subdetectconflict\n");
+ else
+ appendPQExpBuffer(query,
+ " false AS subdetectconflict\n");
appendPQExpBufferStr(query,
"FROM pg_subscription s\n");
@@ -4875,6 +4882,7 @@ getSubscriptions(Archive *fout)
i_suboriginremotelsn = PQfnumber(res, "suboriginremotelsn");
i_subenabled = PQfnumber(res, "subenabled");
i_subfailover = PQfnumber(res, "subfailover");
+ i_subdetectconflict = PQfnumber(res, "subdetectconflict");
subinfo = pg_malloc(ntups * sizeof(SubscriptionInfo));
@@ -4921,6 +4929,8 @@ getSubscriptions(Archive *fout)
pg_strdup(PQgetvalue(res, i, i_subenabled));
subinfo[i].subfailover =
pg_strdup(PQgetvalue(res, i, i_subfailover));
+ subinfo[i].subdetectconflict =
+ pg_strdup(PQgetvalue(res, i, i_subdetectconflict));
/* Decide whether we want to dump it */
selectDumpableObject(&(subinfo[i].dobj), fout);
@@ -5161,6 +5171,9 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (strcmp(subinfo->subfailover, "t") == 0)
appendPQExpBufferStr(query, ", failover = true");
+ if (strcmp(subinfo->subdetectconflict, "t") == 0)
+ appendPQExpBufferStr(query, ", detect_conflict = true");
+
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 4b2e587..bbd7cbe 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -671,6 +671,7 @@ typedef struct _SubscriptionInfo
char *suborigin;
char *suboriginremotelsn;
char *subfailover;
+ char *subdetectconflict;
} SubscriptionInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 7c9a1f2..fef1ad0 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6539,7 +6539,7 @@ describeSubscriptions(const char *pattern, bool verbose)
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
false, false, false, false, false, false, false, false, false, false,
- false};
+ false, false};
if (pset.sversion < 100000)
{
@@ -6607,6 +6607,10 @@ describeSubscriptions(const char *pattern, bool verbose)
appendPQExpBuffer(&buf,
", subfailover AS \"%s\"\n",
gettext_noop("Failover"));
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ ", subdetectconflict AS \"%s\"\n",
+ gettext_noop("Detect conflict"));
appendPQExpBuffer(&buf,
", subsynccommit AS \"%s\"\n"
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 0244694..1c416cf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1946,9 +1946,10 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("(", "PUBLICATION");
/* ALTER SUBSCRIPTION <name> SET ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SET", "("))
- COMPLETE_WITH("binary", "disable_on_error", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit", "two_phase");
+ COMPLETE_WITH("binary", "detect_conflict", "disable_on_error",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit", "two_phase");
/* ALTER SUBSCRIPTION <name> SKIP ( */
else if (HeadMatches("ALTER", "SUBSCRIPTION", MatchAny) && TailMatches("SKIP", "("))
COMPLETE_WITH("lsn");
@@ -3357,9 +3358,10 @@ psql_completion(const char *text, int start, int end)
/* Complete "CREATE SUBSCRIPTION <name> ... WITH ( <opt>" */
else if (HeadMatches("CREATE", "SUBSCRIPTION") && TailMatches("WITH", "("))
COMPLETE_WITH("binary", "connect", "copy_data", "create_slot",
- "disable_on_error", "enabled", "failover", "origin",
- "password_required", "run_as_owner", "slot_name",
- "streaming", "synchronous_commit", "two_phase");
+ "detect_conflict", "disable_on_error", "enabled",
+ "failover", "origin", "password_required",
+ "run_as_owner", "slot_name", "streaming",
+ "synchronous_commit", "two_phase");
/* CREATE TRIGGER --- is allowed inside CREATE SCHEMA, so use TailMatches */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec..17daf11 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -98,6 +98,9 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool subdetectconflict; /* True if replication should perform
+ * conflict detection */
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* Connection string to the publisher */
text subconninfo BKI_FORCE_NOT_NULL;
@@ -151,6 +154,7 @@ typedef struct Subscription
* (i.e. the main slot and the table sync
* slots) in the upstream database are enabled
* to be synchronized to the standbys. */
+ bool detectconflict; /* True if conflict detection is enabled */
char *conninfo; /* Connection string to the publisher */
char *slotname; /* Name of the replication slot */
char *synccommit; /* Synchronous commit setting for worker */
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
new file mode 100644
index 0000000..3a7260d
--- /dev/null
+++ b/src/include/replication/conflict.h
@@ -0,0 +1,50 @@
+/*-------------------------------------------------------------------------
+ * conflict.h
+ * Exports for conflict detection and log
+ *
+ * Copyright (c) 2012-2024, PostgreSQL Global Development Group
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef CONFLICT_H
+#define CONFLICT_H
+
+#include "access/xlogdefs.h"
+#include "executor/tuptable.h"
+#include "nodes/execnodes.h"
+#include "utils/relcache.h"
+#include "utils/timestamp.h"
+
+/*
+ * Conflict types that could be encountered when applying remote changes.
+ */
+typedef enum
+{
+ /* The row to be inserted violates unique constraint */
+ CT_INSERT_EXISTS,
+
+ /* The updated row value violates unique constraint */
+ CT_UPDATE_EXISTS,
+
+ /* The row to be updated was modified by a different origin */
+ CT_UPDATE_DIFFER,
+
+ /* The row to be updated is missing */
+ CT_UPDATE_MISSING,
+
+ /* The row to be deleted is missing */
+ CT_DELETE_MISSING,
+
+ /* The row to be deleted was modified by a different origin */
+ CT_DELETE_DIFFER,
+} ConflictType;
+
+extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
+ RepOriginId *localorigin, TimestampTz *localts);
+extern void ReportApplyConflict(int elevel, ConflictType type,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot);
+extern void InitConflictIndexes(ResultRelInfo *relInfo);
+
+#endif
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index 17d48b1..cc9337c 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -116,18 +116,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | none | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -145,10 +145,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -157,10 +157,10 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = 'newname');
ALTER SUBSCRIPTION regress_testsub SET (password_required = false);
ALTER SUBSCRIPTION regress_testsub SET (run_as_owner = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | f | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (password_required = true);
@@ -176,10 +176,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/12345
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/12345
(1 row)
-- ok - with lsn = NONE
@@ -188,10 +188,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist2 | 0/0
(1 row)
BEGIN;
@@ -223,10 +223,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (synchronous_commit = foobar);
ERROR: invalid value for parameter "synchronous_commit": "foobar"
HINT: Available values: local, remote_write, remote_apply, on, off.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+------------------------------+----------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | local | dbname=regress_doesnotexist2 | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+------------------------------+----------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | off | d | f | any | t | f | f | f | local | dbname=regress_doesnotexist2 | 0/0
(1 row)
-- rename back to keep the rest simple
@@ -255,19 +255,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -279,27 +279,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication already exists
@@ -314,10 +314,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- fail - publication used more than once
@@ -332,10 +332,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -371,19 +371,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
-- we can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -393,10 +393,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -409,18 +409,42 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Synchronous commit | Conninfo | Skip LSN
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------------------+-----------------------------+----------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | off | dbname=regress_doesnotexist | 0/0
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | t | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+ERROR: detect_conflict requires a Boolean value
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+WARNING: subscription was created, but is not connected
+HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and refresh the subscription.
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | t | off | dbname=regress_doesnotexist | 0/0
+(1 row)
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+\dRs+
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Detect conflict | Synchronous commit | Conninfo | Skip LSN
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+-----------------+--------------------+-----------------------------+----------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | f | off | dbname=regress_doesnotexist | 0/0
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql
index 007c9e7..5c740fd 100644
--- a/src/test/regress/sql/subscription.sql
+++ b/src/test/regress/sql/subscription.sql
@@ -287,6 +287,21 @@ ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
DROP SUBSCRIPTION regress_testsub;
+-- fail - detect_conflict must be boolean
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = foo);
+
+-- now it works
+CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, detect_conflict = true);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (detect_conflict = false);
+
+\dRs+
+
+ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
+DROP SUBSCRIPTION regress_testsub;
+
-- let's do some tests with pg_create_subscription rather than superuser
SET SESSION AUTHORIZATION regress_subscription_user3;
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 471e981..78c0307 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -331,11 +331,12 @@ is( $result, qq(1|bar
2|baz),
'update works with REPLICA IDENTITY FULL and a primary key');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber->append_conf('postgresql.conf', "log_min_messages = debug1");
-$node_subscriber->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub SET (detect_conflict = true)"
+);
$node_subscriber->safe_psql('postgres', "DELETE FROM tab_full_pk");
@@ -352,10 +353,10 @@ $node_publisher->wait_for_catchup('tap_sub');
my $logfile = slurp_file($node_subscriber->logfile, $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation "tab_full_pk"/,
+ qr/conflict update_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be updated./m,
'update target row is missing');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab_full_pk"/,
+ qr/conflict delete_missing detected on relation "public.tab_full_pk".*\n.*DETAIL:.* Did not find the row to be deleted./m,
'delete target row is missing');
$node_subscriber->append_conf('postgresql.conf',
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 2958052..7a66a06 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -343,12 +343,12 @@ $result =
$node_subscriber2->safe_psql('postgres', "SELECT a FROM tab1 ORDER BY 1");
is($result, qq(), 'truncate of tab1 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = true)"
+);
$node_publisher->safe_psql('postgres',
"INSERT INTO tab1 VALUES (1, 'foo'), (4, 'bar'), (10, 'baz')");
@@ -372,21 +372,21 @@ $node_publisher->wait_for_catchup('sub2');
my $logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab1_2_2"/,
+ qr/conflict update_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_1"/,
+ qr/conflict delete_missing detected on relation "public.tab1_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_2_2"/,
+ qr/conflict delete_missing detected on relation "public.tab1_2_2".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_2_2');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab1_def"/,
+ qr/conflict delete_missing detected on relation "public.tab1_def".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab1_def');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub1 SET (detect_conflict = false)"
+);
# Tests for replication using root table identity and schema
@@ -773,12 +773,12 @@ pub_tab2|3|yyy
pub_tab2|5|zzz
xxx_c|6|aaa), 'inserts into tab2 replicated');
-# Check that subscriber handles cases where update/delete target tuple
-# is missing. We have to look for the DEBUG1 log messages about that,
-# so temporarily bump up the log verbosity.
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = debug1");
-$node_subscriber1->reload;
+# To check that subscriber handles cases where update/delete target tuple
+# is missing, detect_conflict is temporarily enabled to log conflicts
+# related to missing tuples.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = true)"
+);
$node_subscriber1->safe_psql('postgres', "DELETE FROM tab2");
@@ -796,15 +796,35 @@ $node_publisher->wait_for_catchup('sub2');
$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
ok( $logfile =~
- qr/logical replication did not find row to be updated in replication target relation's partition "tab2_1"/,
+ qr/conflict update_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be updated./,
'update target row is missing in tab2_1');
ok( $logfile =~
- qr/logical replication did not find row to be deleted in replication target relation "tab2_1"/,
+ qr/conflict delete_missing detected on relation "public.tab2_1".*\n.*DETAIL:.* Did not find the row to be deleted./,
'delete target row is missing in tab2_1');
-$node_subscriber1->append_conf('postgresql.conf',
- "log_min_messages = warning");
-$node_subscriber1->reload;
+# Enable the track_commit_timestamp to detect the conflict when attempting
+# to update a row that was previously modified by a different origin.
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = on');
+$node_subscriber1->restart;
+
+$node_subscriber1->safe_psql('postgres', "INSERT INTO tab2 VALUES (3, 'yyy')");
+$node_publisher->safe_psql('postgres',
+ "UPDATE tab2 SET b = 'quux' WHERE a = 3");
+
+$node_publisher->wait_for_catchup('sub_viaroot');
+
+$logfile = slurp_file($node_subscriber1->logfile(), $log_location);
+ok( $logfile =~
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/,
+ 'updating a tuple that was modified by a different origin');
+
+# The remaining tests no longer test conflict detection.
+$node_subscriber1->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub_viaroot SET (detect_conflict = false)"
+);
+
+$node_subscriber1->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_subscriber1->restart;
# Test that replication continues to work correctly after altering the
# partition of a partitioned target table.
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 0ab57a4..496a3c6 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -30,7 +30,7 @@ sub test_skip_lsn
# ERROR with its CONTEXT when retrieving this information.
my $contents = slurp_file($node_subscriber->logfile, $offset);
$contents =~
- qr/duplicate key value violates unique constraint "tbl_pkey".*\n.*DETAIL:.*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
+ qr/conflict insert_exists detected on relation "public.tbl".*\n.*DETAIL:.* Key \(i\)=\(1\) already exists in unique index "tbl_pkey", which was modified by origin \d+ in transaction \d+ at .*\n.*CONTEXT:.* for replication target relation "public.tbl" in transaction \d+, finished at ([[:xdigit:]]+\/[[:xdigit:]]+)/m
or die "could not get error-LSN";
my $lsn = $1;
@@ -83,6 +83,7 @@ $node_subscriber->append_conf(
'postgresql.conf',
qq[
max_prepared_transactions = 10
+track_commit_timestamp = on
]);
$node_subscriber->start;
@@ -109,7 +110,7 @@ my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION pub FOR TABLE tbl");
$node_subscriber->safe_psql('postgres',
- "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on)"
+ "CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
# Initial synchronization failure causes the subscription to be disabled.
diff --git a/src/test/subscription/t/030_origin.pl b/src/test/subscription/t/030_origin.pl
index 056561f..8c929c0 100644
--- a/src/test/subscription/t/030_origin.pl
+++ b/src/test/subscription/t/030_origin.pl
@@ -27,9 +27,14 @@ my $stderr;
my $node_A = PostgreSQL::Test::Cluster->new('node_A');
$node_A->init(allows_streaming => 'logical');
$node_A->start;
+
# node_B
my $node_B = PostgreSQL::Test::Cluster->new('node_B');
$node_B->init(allows_streaming => 'logical');
+
+# Enable the track_commit_timestamp to detect the conflict when attempting to
+# update a row that was previously modified by a different origin.
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = on');
$node_B->start;
# Create table on node_A
@@ -140,6 +145,44 @@ is($result, qq(),
);
###############################################################################
+# Check that the conflict can be detected when attempting to update or
+# delete a row that was previously modified by a different source.
+###############################################################################
+
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = true);
+ DELETE FROM tab;");
+
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (32);");
+
+# The update should update the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "UPDATE tab SET a = 33 WHERE a = 32;");
+
+$node_B->wait_for_log(
+ qr/Updating a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_B->safe_psql('postgres', "DELETE FROM tab;");
+$node_A->safe_psql('postgres', "INSERT INTO tab VALUES (33);");
+
+# The delete should remove the row on node B that was inserted by node A.
+$node_C->safe_psql('postgres', "DELETE FROM tab WHERE a = 33;");
+
+$node_B->wait_for_log(
+ qr/Deleting a row that was modified by a different origin [0-9]+ in transaction [0-9]+ at .*/
+);
+
+$node_A->wait_for_catchup($subname_BA);
+$node_B->wait_for_catchup($subname_AB);
+
+# The remaining tests no longer test conflict detection.
+$node_B->safe_psql('postgres',
+ "ALTER SUBSCRIPTION $subname_BC SET (detect_conflict = false);");
+
+$node_B->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$node_B->restart;
+
+###############################################################################
# Specifying origin = NONE indicates that the publisher should only replicate the
# changes that are generated locally from node_B, but in this case since the
# node_B is also subscribing data from node_A, node_B can have remotely
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3deb611..ae1db1a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -467,6 +467,7 @@ ConditionVariableMinimallyPadded
ConditionalStack
ConfigData
ConfigVariable
+ConflictType
ConnCacheEntry
ConnCacheKey
ConnParams
--
1.8.3.1
[application/octet-stream] v8-0003-Conflict-resolvers-for-insert-update-and-delete.patch (55.8K, ../../CAFPTHDZvACoqRJzrdd8BtexuTD8y3xaSo__y5Ep4u6CnNBspkg@mail.gmail.com/6-v8-0003-Conflict-resolvers-for-insert-update-and-delete.patch)
download | inline diff:
From 9bf4d33a053b906b8a3329805d4482e2a7fdd548 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 31 Jul 2024 03:22:24 -0400
Subject: [PATCH v8 3/4] Conflict resolvers for insert, update, and delete
This patch introduces support for handling conflicts with the following resolutions:
- For INSERT conflicts:
- insert_exists: remote_apply, keep_local, error
- For UPDATE conflicts:
- update_differ: remote_apply, keep_local, error
- update_missing: apply_or_skip, apply_or_error, skip, error
- For DELETE conflicts:
- delete_missing: skip, error
- delete_differ: remote_apply, keep_local, error
---
src/backend/executor/execReplication.c | 56 ++-
src/backend/replication/logical/conflict.c | 207 ++++++--
src/backend/replication/logical/worker.c | 366 ++++++++++----
src/include/executor/executor.h | 5 +-
src/include/replication/conflict.h | 13 +-
src/test/subscription/meson.build | 1 +
src/test/subscription/t/029_on_error.pl | 9 +
src/test/subscription/t/034_conflict_resolver.pl | 576 +++++++++++++++++++++++
8 files changed, 1104 insertions(+), 129 deletions(-)
create mode 100755 src/test/subscription/t/034_conflict_resolver.pl
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 0680bc8..fb96bb7 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -590,8 +590,9 @@ ReCheckConflictIndexes(ResultRelInfo *resultRelInfo, EState *estate,
TransactionId xmin;
GetTupleCommitTs(conflictslot, &xmin, &origin, &committs);
- ReportApplyConflict(ERROR, type, resultRelInfo->ri_RelationDesc, uniqueidx,
- xmin, origin, committs, conflictslot);
+ ReportApplyConflict(type, CR_ERROR, resultRelInfo->ri_RelationDesc,
+ uniqueidx, xmin, origin, committs,
+ conflictslot, false);
}
}
}
@@ -604,7 +605,8 @@ ReCheckConflictIndexes(ResultRelInfo *resultRelInfo, EState *estate,
*/
void
ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot)
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictslot, Oid subid)
{
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
@@ -640,11 +642,55 @@ ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
+
+ /*
+ * If caller has passed non null conflictslot, check all the unique
+ * indexes for potential conflicts. If the configured resolver is in
+ * favour of apply, give the conflicted tuple information in
+ * conflictslot.
+ */
+ if (conflictslot)
+ {
+ foreach_oid(uniqueidx, conflictindexes)
+ {
+ /*
+ * Return if any conflict is found other than one with 'ERROR'
+ * resolver configured. In case of 'ERROR' resolver, emit
+ * error here; otherwise return to caller for resolutions.
+ */
+ if (FindConflictTuple(resultRelInfo, estate, uniqueidx,
+ slot, &(*conflictslot)))
+ {
+ RepOriginId origin;
+ TimestampTz committs;
+ TransactionId xmin;
+ ConflictResolver resolver;
+ bool apply_remote = false;
+
+ GetTupleCommitTs(*conflictslot, &xmin, &origin, &committs);
+ resolver = GetConflictResolver(rel, CT_INSERT_EXISTS,
+ &apply_remote, NULL, subid);
+
+ ReportApplyConflict(CT_INSERT_EXISTS, resolver, rel,
+ uniqueidx, xmin, origin, committs,
+ *conflictslot, apply_remote);
+
+ /* Nothing to apply, free the resources */
+ if (!apply_remote)
+ {
+ ExecDropSingleTupleTableSlot(*conflictslot);
+ *conflictslot = NULL;
+ }
+
+ return;
+ }
+ }
+ }
+
/* OK, store the tuple and create index entries for it */
simple_table_tuple_insert(resultRelInfo->ri_RelationDesc, slot);
- conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
-
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
slot, estate, false,
diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c
index 720e923..9ac707c 100644
--- a/src/backend/replication/logical/conflict.c
+++ b/src/backend/replication/logical/conflict.c
@@ -26,13 +26,17 @@
#include "catalog/pg_subscription_conflict_d.h"
#include "catalog/pg_inherits.h"
#include "commands/defrem.h"
+#include "executor/executor.h"
#include "replication/conflict.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "replication/origin.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/syscache.h"
#include "utils/rel.h"
+#include "utils/syscache.h"
const char *const ConflictTypeNames[] = {
[CT_INSERT_EXISTS] = "insert_exists",
@@ -93,11 +97,13 @@ const int ConflictTypeDefaultResolvers[] = {
};
static char *build_index_value_desc(Oid indexoid, TupleTableSlot *conflictslot);
-static int errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin,
+static int errdetail_apply_conflict(ConflictType type,
+ ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
RepOriginId localorigin,
TimestampTz localts,
- TupleTableSlot *conflictslot);
+ TupleTableSlot *conflictslot,
+ bool apply_remote);
/*
* Get the xmin and commit timestamp data (origin and timestamp) associated
@@ -132,22 +138,32 @@ GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
}
/*
- * Report a conflict when applying remote changes.
+ * Report conflict and resolution applied while applying remote changes.
*/
void
-ReportApplyConflict(int elevel, ConflictType type, Relation localrel,
- Oid conflictidx, TransactionId localxmin,
- RepOriginId localorigin, TimestampTz localts,
- TupleTableSlot *conflictslot)
+ReportApplyConflict(ConflictType type, ConflictResolver resolver,
+ Relation localrel, Oid conflictidx,
+ TransactionId localxmin, RepOriginId localorigin,
+ TimestampTz localts, TupleTableSlot *conflictslot,
+ bool apply_remote)
{
+ int elevel;
+
+ if (resolver == CR_ERROR ||
+ (resolver == CR_APPLY_OR_ERROR && !apply_remote))
+ elevel = ERROR;
+ else
+ elevel = LOG;
+
ereport(elevel,
errcode(ERRCODE_INTEGRITY_CONSTRAINT_VIOLATION),
- errmsg("conflict %s detected on relation \"%s.%s\"",
+ errmsg("conflict %s detected on relation \"%s.%s\". Resolution: %s",
ConflictTypeNames[type],
get_namespace_name(RelationGetNamespace(localrel)),
- RelationGetRelationName(localrel)),
- errdetail_apply_conflict(type, conflictidx, localxmin, localorigin,
- localts, conflictslot));
+ RelationGetRelationName(localrel),
+ ConflictResolverNames[resolver]),
+ errdetail_apply_conflict(type, resolver, conflictidx, localxmin, localorigin,
+ localts, conflictslot, apply_remote));
}
/*
@@ -182,13 +198,21 @@ InitConflictIndexes(ResultRelInfo *relInfo)
}
/*
- * Add an errdetail() line showing conflict detail.
+ * Add an errdetail() line showing conflict and resolution details.
*/
static int
-errdetail_apply_conflict(ConflictType type, Oid conflictidx,
- TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot)
+errdetail_apply_conflict(ConflictType type, ConflictResolver resolver,
+ Oid conflictidx, TransactionId localxmin,
+ RepOriginId localorigin, TimestampTz localts,
+ TupleTableSlot *conflictslot, bool apply_remote)
{
+ char *applymsg;
+
+ if (apply_remote)
+ applymsg = "applying the remote changes.";
+ else
+ applymsg = "ignoring the remote changes.";
+
switch (type)
{
case CT_INSERT_EXISTS:
@@ -202,27 +226,43 @@ errdetail_apply_conflict(ConflictType type, Oid conflictidx,
char *index_value = build_index_value_desc(conflictidx,
conflictslot);
- if (index_value && localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
- index_value, get_rel_name(conflictidx), localorigin,
- localxmin, timestamptz_to_str(localts));
- else if (index_value && !localts)
- return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
- index_value, get_rel_name(conflictidx), localxmin);
+ if (resolver == CR_ERROR)
+ {
+ if (index_value && localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified by origin %u in transaction %u at %s.",
+ index_value, get_rel_name(conflictidx), localorigin,
+ localxmin, timestamptz_to_str(localts));
+ else if (index_value && !localts)
+ return errdetail("Key %s already exists in unique index \"%s\", which was modified in transaction %u.",
+ index_value, get_rel_name(conflictidx), localxmin);
+ else
+ return errdetail("Key already exists in unique index \"%s\".",
+ get_rel_name(conflictidx));
+ }
else
- return errdetail("Key already exists in unique index \"%s\".",
- get_rel_name(conflictidx));
+ return errdetail("Key already exists, %s", applymsg);
}
case CT_UPDATE_DIFFER:
- return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Updating a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin,
+ timestamptz_to_str(localts), applymsg);
case CT_UPDATE_MISSING:
- return errdetail("Did not find the row to be updated.");
+ if (resolver == CR_APPLY_OR_SKIP && !apply_remote)
+ return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update.");
+ else if (resolver == CR_APPLY_OR_ERROR && !apply_remote)
+ return errdetail("Did not find the row to be updated. UPDATE can not be converted to INSERT, hence ERROR out.");
+ else if (apply_remote)
+ return errdetail("Did not find the row to be updated. Convert UPDATE to INSERT and %s",
+ applymsg);
+ else
+ return errdetail("Did not find the row to be updated, %s",
+ applymsg);
case CT_DELETE_MISSING:
return errdetail("Did not find the row to be deleted.");
case CT_DELETE_DIFFER:
- return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s.",
- localorigin, localxmin, timestamptz_to_str(localts));
+ return errdetail("Deleting a row that was modified by a different origin %u in transaction %u at %s, %s",
+ localorigin, localxmin,
+ timestamptz_to_str(localts), applymsg);
}
return 0; /* silence compiler warning */
@@ -340,6 +380,69 @@ validate_conflict_type_and_resolver(const char *conflict_type,
}
/*
+ * Get the conflict resolver configured at subscription level for
+ * for the given conflict type.
+ */
+static ConflictResolver
+get_conflict_resolver_internal(ConflictType type, Oid subid)
+{
+
+ ConflictResolver resolver;
+ HeapTuple tuple;
+ Datum datum;
+ char *conflict_res;
+
+ /*
+ * XXX: Currently, we fetch the conflict resolver from cache for each
+ * conflict detection. If needed, we can keep the info in global variable
+ * and fetch from cache only once after cache invalidation.
+ */
+ tuple = SearchSysCache2(SUBSCRIPTIONCONFLICTSUBOID,
+ ObjectIdGetDatum(subid),
+ CStringGetTextDatum(ConflictTypeNames[type]));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for conflict type %u", type);
+
+ datum = SysCacheGetAttrNotNull(SUBSCRIPTIONCONFLICTSUBOID,
+ tuple, Anum_pg_subscription_conflict_confrres);
+
+ conflict_res = TextDatumGetCString(datum);
+
+ for (resolver = CR_MIN; resolver <= CR_MAX; resolver++)
+ {
+ if (strcmp(ConflictResolverNames[resolver], conflict_res) == 0)
+ break;
+ }
+
+ ReleaseSysCache(tuple);
+ return resolver;
+}
+
+/*
+ * Check if a full tuple can be created from the new tuple.
+ * Return true if yes, false otherwise.
+ */
+static bool
+can_create_full_tuple(Relation localrel,
+ LogicalRepTupleData *newtup)
+{
+ int i;
+ int local_att = RelationGetNumberOfAttributes(localrel);
+
+ if (newtup->ncols != local_att)
+ return false;
+
+ for (i = 0; i < newtup->ncols; i++)
+ {
+ if (newtup->colstatus[i] == LOGICALREP_COLUMN_UNCHANGED)
+ return false;
+ }
+
+ return true;
+}
+
+/*
* Extract the conflict type and conflict resolvers from the
* ALTER SUBSCRIPTION command and return a list of ConflictTypeResolver nodes.
*/
@@ -499,3 +602,47 @@ RemoveSubscriptionConflictBySubid(Oid subid)
table_endscan(scan);
table_close(rel, RowExclusiveLock);
}
+
+/*
+ * Find the resolver of the conflict type set under the given subscription.
+ *
+ * Set 'apply_remote' to true if remote tuple should be applied,
+ * false otherwise.
+ */
+ConflictResolver
+GetConflictResolver(Relation localrel, ConflictType type, bool *apply_remote,
+ LogicalRepTupleData *newtup, Oid subid)
+{
+ ConflictResolver resolver;
+
+ resolver = get_conflict_resolver_internal(type, subid);
+
+ switch (resolver)
+ {
+ case CR_REMOTE_APPLY:
+ *apply_remote = true;
+ break;
+ case CR_APPLY_OR_SKIP:
+ if (can_create_full_tuple(localrel, newtup))
+ *apply_remote = true;
+ else
+ *apply_remote = false;
+ break;
+ case CR_APPLY_OR_ERROR:
+ if (can_create_full_tuple(localrel, newtup))
+ *apply_remote = true;
+ else
+ *apply_remote = false;
+ break;
+ case CR_KEEP_LOCAL:
+ case CR_SKIP:
+ case CR_ERROR:
+ *apply_remote = false;
+ break;
+ default:
+ elog(ERROR, "Conflict %s is detected! Unrecogonized conflict resolution method",
+ ConflictTypeNames[type]);
+ }
+
+ return resolver;
+}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc3f80e..36172b2 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -382,7 +382,9 @@ static void send_feedback(XLogRecPtr recvpos, bool force, bool requestReply);
static void apply_handle_commit_internal(LogicalRepCommitData *commit_data);
static void apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot);
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry);
static void apply_handle_update_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
TupleTableSlot *remoteslot,
@@ -2428,10 +2430,10 @@ apply_handle_insert(StringInfo s)
/* For a partitioned table, insert the tuple into a partition. */
if (rel->localrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
apply_handle_tuple_routing(edata,
- remoteslot, NULL, CMD_INSERT);
+ remoteslot, &newtup, CMD_INSERT);
else
apply_handle_insert_internal(edata, edata->targetRelInfo,
- remoteslot);
+ remoteslot, &newtup, rel);
finish_edata(edata);
@@ -2454,9 +2456,13 @@ apply_handle_insert(StringInfo s)
static void
apply_handle_insert_internal(ApplyExecutionData *edata,
ResultRelInfo *relinfo,
- TupleTableSlot *remoteslot)
+ TupleTableSlot *remoteslot,
+ LogicalRepTupleData *newtup,
+ LogicalRepRelMapEntry *rel_entry)
{
EState *estate = edata->estate;
+ MemoryContext oldctx;
+ TupleTableSlot *conflictslot = NULL;
/* We must open indexes here. */
ExecOpenIndices(relinfo, MySubscription->detectconflict);
@@ -2466,7 +2472,36 @@ apply_handle_insert_internal(ApplyExecutionData *edata,
/* Do the insert. */
TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_INSERT);
- ExecSimpleRelationInsert(relinfo, estate, remoteslot);
+
+ if (MySubscription->detectconflict)
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, &conflictslot,
+ MyLogicalRepWorker->subid);
+ else
+ ExecSimpleRelationInsert(relinfo, estate, remoteslot, NULL, InvalidOid);
+
+
+ /* Apply remote tuple by converting INSERT to UPDATE */
+ if (conflictslot)
+ {
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, conflictslot, rel_entry, newtup);
+ MemoryContextSwitchTo(oldctx);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
+
+ /* Do the update */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, conflictslot,
+ remoteslot);
+
+ EvalPlanQualEnd(&epqstate);
+ ExecDropSingleTupleTableSlot(conflictslot);
+ }
/* Cleanup. */
ExecCloseIndices(relinfo);
@@ -2648,6 +2683,8 @@ apply_handle_update_internal(ApplyExecutionData *edata,
TupleTableSlot *localslot;
bool found;
MemoryContext oldctx;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, MySubscription->detectconflict);
@@ -2671,38 +2708,71 @@ apply_handle_update_internal(ApplyExecutionData *edata,
/*
* If conflict detection is enabled, check whether the local tuple was
- * modified by a different origin. If detected, report the conflict.
+ * modified by a different origin. If detected, report the conflict
+ * and configured resolver.
*/
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
- /* Process and store remote tuple in the slot */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot, localslot, relmapentry, newtup);
- MemoryContextSwitchTo(oldctx);
+ ReportApplyConflict(CT_UPDATE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- EvalPlanQualSetSlot(&epqstate, remoteslot);
+ /*
+ * Apply the change if configured resolver is in favor of that, else
+ * ignore the remote update.
+ */
+ if (apply_remote)
+ {
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot, localslot, relmapentry, newtup);
+ MemoryContextSwitchTo(oldctx);
- if (MySubscription->detectconflict)
- InitConflictIndexes(relinfo);
+ EvalPlanQualSetSlot(&epqstate, remoteslot);
- /* Do the actual update. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
- ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
- remoteslot);
+ if (MySubscription->detectconflict)
+ InitConflictIndexes(relinfo);
+
+ /* Do the actual update. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_UPDATE);
+ ExecSimpleRelationUpdate(relinfo, estate, &epqstate, localslot,
+ remoteslot);
+ }
}
else
{
/*
- * The tuple to be updated could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be updated could not be found. Report the conflict. If
+ * the configured resolver is in favor of applying the change, convert
+ * UPDATE to INSERT and apply the change.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup,
+ MyLogicalRepWorker->subid);
+
+ ReportApplyConflict(CT_UPDATE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+
+ /* Process and store remote tuple in the slot */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot, relmapentry, newtup);
+ slot_fill_defaults(relmapentry, estate, remoteslot);
+ MemoryContextSwitchTo(oldctx);
+
+ if (apply_remote)
+ apply_handle_insert_internal(edata, relinfo, remoteslot,
+ newtup, relmapentry);
+ }
}
/* Cleanup. */
@@ -2815,6 +2885,8 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
EPQState epqstate;
TupleTableSlot *localslot;
bool found;
+ bool apply_remote = true;
+ ConflictResolver resolver;
EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
ExecOpenIndices(relinfo, false);
@@ -2836,24 +2908,44 @@ apply_handle_delete_internal(ApplyExecutionData *edata,
if (MySubscription->detectconflict &&
GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_DELETE_DIFFER, localrel, InvalidOid,
- localxmin, localorigin, localts, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_DELETE_DIFFER,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
+
+ ReportApplyConflict(CT_DELETE_DIFFER, resolver, localrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- EvalPlanQualSetSlot(&epqstate, localslot);
+ if (apply_remote)
+ {
+ EvalPlanQualSetSlot(&epqstate, localslot);
+
+ /* Do the actual delete. */
+ TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
+ ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
+ }
- /* Do the actual delete. */
- TargetPrivilegesCheck(relinfo->ri_RelationDesc, ACL_DELETE);
- ExecSimpleRelationDelete(relinfo, estate, &epqstate, localslot);
}
else
{
/*
- * The tuple to be deleted could not be found. Do nothing except for
- * emitting a log message.
+ * The tuple to be deleted could not be found. Based on resolver
+ * configured, either skip and log a message or emit an error.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_DELETE_MISSING, localrel, InvalidOid,
- InvalidTransactionId, InvalidRepOriginId, 0, NULL);
+ {
+ resolver = GetConflictResolver(localrel, CT_DELETE_MISSING,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
+
+ /* Resolver is set to skip, thus report the conflict and skip */
+ if (!apply_remote)
+ ReportApplyConflict(CT_DELETE_MISSING, resolver, localrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+ }
}
/* Cleanup. */
@@ -2986,19 +3078,21 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
}
MemoryContextSwitchTo(oldctx);
- /* Check if we can do the update or delete on the leaf partition. */
- if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ if (operation == CMD_INSERT || operation == CMD_UPDATE ||
+ operation == CMD_DELETE)
{
- part_entry = logicalrep_partition_open(relmapentry, partrel,
- attrmap);
- check_relation_updatable(part_entry);
+ part_entry = logicalrep_partition_open(relmapentry, partrel, attrmap);
+
+ /* Check if we can do the update or delete on the leaf partition */
+ if (operation != CMD_INSERT)
+ check_relation_updatable(part_entry);
}
switch (operation)
{
case CMD_INSERT:
apply_handle_insert_internal(edata, partrelinfo,
- remoteslot_part);
+ remoteslot_part, newtup, part_entry);
break;
case CMD_DELETE:
@@ -3023,6 +3117,9 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
RepOriginId localorigin;
TransactionId localxmin;
TimestampTz localts;
+ LogicalRepRelMapEntry *part_entry_new = NULL;
+ ConflictResolver resolver;
+ bool apply_remote = true;
/* Get the matching local tuple from the partition. */
found = FindReplTupleInLocalRel(edata, partrel,
@@ -3032,38 +3129,81 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
if (!found)
{
/*
- * The tuple to be updated could not be found. Do nothing
- * except for emitting a log message.
+ * The tuple to be updated could not be found. Report the
+ * conflict and resolver. And take action based on the
+ * configured resolver.
*/
if (MySubscription->detectconflict)
- ReportApplyConflict(LOG, CT_UPDATE_MISSING,
- partrel, InvalidOid,
- InvalidTransactionId,
- InvalidRepOriginId, 0, NULL);
-
- return;
+ {
+ resolver = GetConflictResolver(partrel, CT_UPDATE_MISSING,
+ &apply_remote, newtup,
+ MyLogicalRepWorker->subid);
+
+ ReportApplyConflict(CT_UPDATE_MISSING, resolver, partrel,
+ InvalidOid, InvalidTransactionId,
+ InvalidRepOriginId, 0, NULL, apply_remote);
+
+ if (apply_remote)
+ {
+ /*
+ * Resolver is in favour of applying the remote
+ * changes. Prepare the slot for the INSERT.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_store_data(remoteslot_part, part_entry, newtup);
+ slot_fill_defaults(part_entry, estate, remoteslot_part);
+ MemoryContextSwitchTo(oldctx);
+ }
+ }
}
+ else
+ {
+ /*
+ * The tuple to be updated is found. If conflict detection
+ * is enabled, check whether the local tuple was modified
+ * by a different origin. If detected, report and resolve
+ * the conflict.
+ */
+ if (MySubscription->detectconflict &&
+ GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
+ localorigin != replorigin_session_origin)
+ {
+ resolver = GetConflictResolver(partrel, CT_UPDATE_DIFFER,
+ &apply_remote, NULL,
+ MyLogicalRepWorker->subid);
- /*
- * If conflict detection is enabled, check whether the local
- * tuple was modified by a different origin. If detected,
- * report the conflict.
- */
- if (MySubscription->detectconflict &&
- GetTupleCommitTs(localslot, &localxmin, &localorigin, &localts) &&
- localorigin != replorigin_session_origin)
- ReportApplyConflict(LOG, CT_UPDATE_DIFFER, partrel,
- InvalidOid, localxmin, localorigin,
- localts, NULL);
+ ReportApplyConflict(CT_UPDATE_DIFFER, resolver, partrel,
+ InvalidOid, localxmin, localorigin, localts,
+ NULL, apply_remote);
+ }
- /*
- * Apply the update to the local tuple, putting the result in
- * remoteslot_part.
- */
- oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
- slot_modify_data(remoteslot_part, localslot, part_entry,
- newtup);
- MemoryContextSwitchTo(oldctx);
+ if (apply_remote)
+ {
+ /*
+ * We can reach here in two cases:
+ *
+ * 1. If we found a tuple but conflict detection is
+ * OFF
+ *
+ * 2. If we found a tuple and resolver is in favor of
+ * applying the change when conflict detection is ON
+ *
+ * Putting the result in remoteslot_part.
+ */
+ oldctx = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+ slot_modify_data(remoteslot_part, localslot, part_entry,
+ newtup);
+ MemoryContextSwitchTo(oldctx);
+ }
+ else
+
+ /*
+ * apply_remote can be toggled if resolver for
+ * update_differ is set to skip. Ignore remote update.
+ */
+ return;
+
+ }
/*
* Does the updated tuple still satisfy the current
@@ -3073,27 +3213,59 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
ExecPartitionCheck(partrelinfo, remoteslot_part, estate,
false))
{
- /*
- * Yes, so simply UPDATE the partition. We don't call
- * apply_handle_update_internal() here, which would
- * normally do the following work, to avoid repeating some
- * work already done above to find the local tuple in the
- * partition.
- */
- EPQState epqstate;
-
- EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
- ExecOpenIndices(partrelinfo, false);
-
- EvalPlanQualSetSlot(&epqstate, remoteslot_part);
- TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
- ACL_UPDATE);
- ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
- localslot, remoteslot_part);
- ExecCloseIndices(partrelinfo);
- EvalPlanQualEnd(&epqstate);
+ if (found && apply_remote)
+ {
+ /*
+ * Yes, so simply UPDATE the partition. We don't call
+ * apply_handle_update_internal() here, which would
+ * normally do the following work, to avoid repeating
+ * some work already done above to find the local
+ * tuple in the partition.
+ *
+ * Do the update in cases - 1. conflict detection is
+ * OFF and found a tuple 2. conflict detection is ON,
+ * update_differ conflict is detected for the found
+ * tuple and the resolver is in favour of applying the
+ * update.
+ */
+
+ EPQState epqstate;
+
+ EvalPlanQualInit(&epqstate, estate, NULL, NIL, -1, NIL);
+ ExecOpenIndices(partrelinfo, false);
+
+ EvalPlanQualSetSlot(&epqstate, remoteslot_part);
+ TargetPrivilegesCheck(partrelinfo->ri_RelationDesc,
+ ACL_UPDATE);
+ ExecSimpleRelationUpdate(partrelinfo, estate, &epqstate,
+ localslot, remoteslot_part);
+ ExecCloseIndices(partrelinfo);
+ EvalPlanQualEnd(&epqstate);
+
+ }
+ else if (apply_remote)
+ {
+ /*
+ * Tuple is not found but update_missing resolver is
+ * in favour of applying the change as INSERT.
+ */
+ apply_handle_insert_internal(edata, partrelinfo,
+ remoteslot_part, newtup,
+ part_entry);
+ }
}
- else
+
+ /*
+ * Updated tuple doesn't satisfy the current partition's
+ * constraint.
+ *
+ * If conflict detection is OFF, proceed by always applying
+ * the update (as 'apply_remote' is by default true). If
+ * conflict detection is ON, 'apply_remote' can be OFF as well
+ * if the resolver for update_missing conflict conveys to skip
+ * the update.
+ */
+ else if (apply_remote)
{
/* Move the tuple into the new partition. */
@@ -3134,10 +3306,16 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
get_namespace_name(RelationGetNamespace(partrel_new)),
RelationGetRelationName(partrel_new));
- /* DELETE old tuple found in the old partition. */
- apply_handle_delete_internal(edata, partrelinfo,
- localslot,
- part_entry->localindexoid);
+ /*
+ * If tuple is found, delete it from old partition. We can
+ * reach this flow even for the case when the 'found' flag
+ * is false for 'update_missing' conflict and resolver is
+ * in favor of inserting the tuple.
+ */
+ if (found)
+ apply_handle_delete_internal(edata, partrelinfo,
+ localslot,
+ part_entry->localindexoid);
/* INSERT new tuple into the new partition. */
@@ -3153,19 +3331,25 @@ apply_handle_tuple_routing(ApplyExecutionData *edata,
map = ExecGetRootToChildMap(partrelinfo_new, estate);
if (map != NULL)
{
+ attrmap = map->attrMap;
remoteslot_part = execute_attr_map_slot(map->attrMap,
remoteslot,
remoteslot_part);
}
else
{
+ attrmap = NULL;
remoteslot_part = ExecCopySlot(remoteslot_part,
remoteslot);
slot_getallattrs(remoteslot);
}
MemoryContextSwitchTo(oldctx);
+ part_entry_new = logicalrep_partition_open(part_entry,
+ partrel_new,
+ attrmap);
apply_handle_insert_internal(edata, partrelinfo_new,
- remoteslot_part);
+ remoteslot_part, newtup,
+ part_entry_new);
}
}
break;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 9770752..7d4a698 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,8 @@
#include "fmgr.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/memutils.h"
@@ -655,7 +657,8 @@ extern bool RelationFindReplTupleSeq(Relation rel, LockTupleMode lockmode,
TupleTableSlot *searchslot, TupleTableSlot *outslot);
extern void ExecSimpleRelationInsert(ResultRelInfo *resultRelInfo,
- EState *estate, TupleTableSlot *slot);
+ EState *estate, TupleTableSlot *slot,
+ TupleTableSlot **conflictsloty, Oid subid);
extern void ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
EState *estate, EPQState *epqstate,
TupleTableSlot *searchslot, TupleTableSlot *slot);
diff --git a/src/include/replication/conflict.h b/src/include/replication/conflict.h
index c102f74..2793ad6 100644
--- a/src/include/replication/conflict.h
+++ b/src/include/replication/conflict.h
@@ -12,6 +12,8 @@
#include "access/xlogdefs.h"
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
+#include "replication/logicalproto.h"
+#include "replication/logicalrelation.h"
#include "utils/relcache.h"
#include "utils/timestamp.h"
@@ -82,10 +84,11 @@ typedef struct ConflictTypeResolver
extern bool GetTupleCommitTs(TupleTableSlot *localslot, TransactionId *xmin,
RepOriginId *localorigin, TimestampTz *localts);
-extern void ReportApplyConflict(int elevel, ConflictType type,
+extern void ReportApplyConflict(ConflictType type, ConflictResolver resolver,
Relation localrel, Oid conflictidx,
TransactionId localxmin, RepOriginId localorigin,
- TimestampTz localts, TupleTableSlot *conflictslot);
+ TimestampTz localts, TupleTableSlot *conflictslot,
+ bool apply_remote);
extern void InitConflictIndexes(ResultRelInfo *relInfo);
extern void SetSubConflictResolver(Oid subId, ConflictTypeResolver * resolvers, int max_types);
extern void RemoveSubscriptionConflictById(Oid confid);
@@ -95,5 +98,11 @@ extern void UpdateSubConflictResolvers(List *conflict_resolvers, Oid subid);
extern ConflictType validate_conflict_type_and_resolver(const char *conflict_type,
const char *conflict_resolver);
extern void SetDefaultResolvers(ConflictTypeResolver * conflictResolvers);
+extern ConflictResolver GetConflictResolver(Relation localrel,
+ ConflictType type,
+ bool *apply_remote,
+ LogicalRepTupleData *newtup,
+ Oid subid);
+extern bool CanCreateFullTuple(Relation localrel, LogicalRepTupleData *newtup);
#endif
diff --git a/src/test/subscription/meson.build b/src/test/subscription/meson.build
index c591cd7..00ade29 100644
--- a/src/test/subscription/meson.build
+++ b/src/test/subscription/meson.build
@@ -40,6 +40,7 @@ tests += {
't/031_column_list.pl',
't/032_subscribe_use_index.pl',
't/033_run_as_table_owner.pl',
+ 't/034_conflict_resolver.pl',
't/100_bugs.pl',
],
},
diff --git a/src/test/subscription/t/029_on_error.pl b/src/test/subscription/t/029_on_error.pl
index 496a3c6..e6f07fa 100644
--- a/src/test/subscription/t/029_on_error.pl
+++ b/src/test/subscription/t/029_on_error.pl
@@ -113,6 +113,11 @@ $node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION sub CONNECTION '$publisher_connstr' PUBLICATION pub WITH (disable_on_error = true, streaming = on, two_phase = on, detect_conflict = on)"
);
+# Set 'ERROR' conflict resolver for 'insert_exist' conflict type
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub CONFLICT RESOLVER (insert_exists=error)");
+
+
# Initial synchronization failure causes the subscription to be disabled.
$node_subscriber->poll_query_until('postgres',
"SELECT subenabled = false FROM pg_catalog.pg_subscription WHERE subname = 'sub'"
@@ -177,6 +182,10 @@ $result = $node_subscriber->safe_psql('postgres',
is($result, "0",
"check all prepared transactions are resolved on the subscriber");
+# Reset conflict resolver for 'insert_exist' conflict type to default.
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION sub CONFLICT RESOLVER (insert_exists=remote_apply)");
+
$node_subscriber->stop;
$node_publisher->stop;
diff --git a/src/test/subscription/t/034_conflict_resolver.pl b/src/test/subscription/t/034_conflict_resolver.pl
new file mode 100755
index 0000000..58751c5
--- /dev/null
+++ b/src/test/subscription/t/034_conflict_resolver.pl
@@ -0,0 +1,576 @@
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+###############################
+# Setup
+###############################
+
+# Initialize publisher node
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf',
+ qq(max_prepared_transactions = 10));
+$node_publisher->start;
+
+# Create subscriber node with track_commit_timestamp_set
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ qq(track_commit_timestamp = on));
+$node_subscriber->start;
+
+# Create table on publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Create similar table on subscriber
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE TABLE conf_tab(a int PRIMARY key, data text, comments text);
+ ALTER TABLE conf_tab ALTER COLUMN comments SET STORAGE EXTERNAL;");
+
+# Setup logical replication
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE conf_tab");
+
+# Create the subscription
+my $appname = 'tap_sub';
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on);");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT confrtype, confrres FROM pg_subscription_conflict ORDER BY confrtype"
+);
+is( $result, qq(delete_differ|remote_apply
+delete_missing|skip
+insert_exists|remote_apply
+update_differ|remote_apply
+update_exists|remote_apply
+update_missing|apply_or_skip),
+ "confirm that the default conflict resolvers are in place");
+
+############################################
+# Test 'remote_apply' for 'insert_exists'
+############################################
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'fromsub')");
+
+# Create conflicting data on the publisher
+my $log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that remote insert is converted to an update and the remote data is updated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=1);");
+
+is($result, 'frompub', "remote data is kept");
+
+
+########################################
+# Test 'keep_local' for 'insert_exists'
+########################################
+
+# Change CONFLICT RESOLVER of insert_exists to keep_local
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'keep_local');"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'fromsub')");
+
+# Confirm that row is updated
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "data 2 from local is inserted");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (2,'frompub')");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that remote insert is ignored and the local row is kept
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data FROM conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "data from local is kept");
+
+###################################
+# Test 'error' for 'insert_exists'
+###################################
+
+# Change CONFLICT RESOLVER of insert_exists to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (insert_exists = 'error');"
+);
+
+# Create local data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'fromsub')");
+
+# Create conflicting data on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (3,'frompub')");
+
+$log_offset = -s $node_subscriber->logfile;
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict insert_exists detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Truncate table on subscriber to get rid of the error
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+###################################
+# Test 'skip' for 'delete_missing'
+###################################
+
+# Delete row on publisher that is not present on the subscriber and confirm that it is skipped
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=2);");
+
+# Confirm that the missing row is skipped because 'delete_missing' is set to 'skip'
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+####################################
+# Test 'error' for 'delete_missing'
+####################################
+
+# Change CONFLICT RESOLVER of delete_missing to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_missing = 'error');"
+);
+
+# Capture the log offset before performing the delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+
+# Perform the delete on the publisher
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=1);");
+
+# Confirm that this causes an error on the subscriber
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate table on subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+
+#########################################
+# Test 'remote_apply' for 'delete_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting delete on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote delete the local updated row
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);");
+
+is($result, '', "delete from remote is applied");
+
+#########################################
+# Test 'keep_local' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'keep_local');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'delete_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of delete_differ to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (delete_differ = 'error');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict delete_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+#########################################
+# Test 'remote_apply' for 'update_differ'
+#########################################
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=1);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=1);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update overrides the local update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=1);");
+
+is($result, 'frompubnew', "update from remote is kept");
+
+#########################################
+# Test 'keep_local' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to keep_local
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_differ = 'keep_local');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the local data is untouched by the remote update
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);");
+
+is($result, 'fromsub', "update from local is kept");
+
+#########################################
+# Test 'error' for 'update_differ'
+#########################################
+
+# Change CONFLICT RESOLVER of update_differ to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_differ = 'error');"
+);
+
+# Modify data on the subscriber
+$node_subscriber->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'fromsub' WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_differ detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###########################################
+# Test 'apply_or_skip' for 'update_missing'
+###########################################
+
+#test the apply part
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=3);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=3);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=3);");
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the skip part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(4,'frompub',repeat('abcdefghij', 200));"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=4);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=4);");
+
+$node_subscriber->wait_for_log(
+ qr/DETAIL: Did not find the row to be updated. UPDATE can not be converted to INSERT, hence SKIP the update./,
+ $log_offset);
+
+###########################################
+# Test 'apply_or_error' for 'update_missing'
+###########################################
+
+# test the apply part
+
+#Change CONFLICT RESOLVER of update_missing to apply_or_error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_missing = 'apply_or_error');"
+);
+
+# Create new row on the publisher
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (5,'frompub');");
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=5);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=5);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the remote update is converted to an insert and new row applied
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=5);");
+
+is($result, 'frompubnew', "update from remote is converted to insert");
+
+#test the error part
+
+# Create new row on publisher with toast data
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO conf_tab (a, data, comments) VALUES(6,'frompub',repeat('abcdefghij', 200));"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=6);");
+
+# Update the row on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=6);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Drop the subscriber to remove error
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub;");
+
+# Truncate the table on the publisher
+$node_publisher->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Truncate the table on the subscriber
+$node_subscriber->safe_psql('postgres', "TRUNCATE conf_tab;");
+
+# Create the subscription
+$node_subscriber->safe_psql(
+ 'postgres',
+ "CREATE SUBSCRIPTION tap_sub
+ CONNECTION '$publisher_connstr application_name=$appname'
+ PUBLICATION tap_pub
+ WITH (detect_conflict = on)");
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher, $appname);
+
+# Insert data in the publisher
+$node_publisher->safe_psql(
+ 'postgres',
+ "INSERT INTO conf_tab(a, data) VALUES (1,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (2,'frompub');
+ INSERT INTO conf_tab(a, data) VALUES (3,'frompub');");
+
+###################################
+# Test 'skip' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to skip
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_missing = 'skip');"
+);
+
+# Delete the row on the subscriber
+$node_subscriber->safe_psql('postgres', "DELETE FROM conf_tab WHERE (a=2);");
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnew' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/LOG: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+# Confirm that the update does not change anything on the subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT data from conf_tab WHERE (a=2);");
+
+is($result, '', "update from remote is skipped on the subscriber");
+
+###################################
+# Test 'error' for 'update_missing'
+###################################
+
+#Change CONFLICT RESOLVER of update_missing to error
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub CONFLICT RESOLVER (update_missing = 'error');"
+);
+
+# Create a conflicting update on the publisher
+$log_offset = -s $node_subscriber->logfile;
+$node_publisher->safe_psql('postgres',
+ "UPDATE conf_tab SET data = 'frompubnewer' WHERE (a=2);");
+
+$node_subscriber->wait_for_log(
+ qr/ERROR: conflict update_missing detected on relation \"public.conf_tab\"/,
+ $log_offset);
+
+done_testing();
--
1.8.3.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
@ 2024-07-30 10:33 ` Dilip Kumar <[email protected]>
2024-07-30 11:26 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
1 sibling, 1 reply; 27+ messages in thread
From: Dilip Kumar @ 2024-07-30 10:33 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: shveta malik <[email protected]>; Nisha Moond <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>
On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian <[email protected]> wrote:
Comment in 0002,
1) I do not see any test case that set a proper conflict type and
conflict resolver, all tests either give incorrect conflict
type/conflict resolver or the conflict resolver is ignored
0003
2) I was trying to think about this patch, so suppose we consider this
case conflict_type-> update_differ resolver->remote_apply, my
question is to confirm whether my understanding is correct. So if
this is set and we have 2 nodes and set up a 2-way logical
replication, and if a conflict occurs node-1 will take the changes of
node-2 and node-2 will take the changes of node-1? Maybe so I think
to avoid such cases user needs to set the resolver more thoughtfully,
on node-1 it may be set as "skip" and on node-1 as "remote-apply" so
in such cases if conflict happens both nodes will have the value from
node-1. But maybe it would be more difficult to get a consistent
value if we are setting up a mess replication topology right? Maybe
there I think a more advanced timestamp-based option would work better
IMHO.
I am doing code level review as well and will share my comments soon
on 0003 and 0004
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
2024-07-30 10:33 ` Re: Conflict Detection and Resolution Dilip Kumar <[email protected]>
@ 2024-07-30 11:26 ` shveta malik <[email protected]>
2024-07-30 12:54 ` Re: Conflict Detection and Resolution Dilip Kumar <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: shveta malik @ 2024-07-30 11:26 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Nisha Moond <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>; shveta malik <[email protected]>
On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar <[email protected]> wrote:
>
> On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian <[email protected]> wrote:
>
> Comment in 0002,
>
> 1) I do not see any test case that set a proper conflict type and
> conflict resolver, all tests either give incorrect conflict
> type/conflict resolver or the conflict resolver is ignored
>
> 0003
> 2) I was trying to think about this patch, so suppose we consider this
> case conflict_type-> update_differ resolver->remote_apply, my
> question is to confirm whether my understanding is correct. So if
> this is set and we have 2 nodes and set up a 2-way logical
> replication, and if a conflict occurs node-1 will take the changes of
> node-2 and node-2 will take the changes of node-1?
Yes, that's right.
> Maybe so I think
> to avoid such cases user needs to set the resolver more thoughtfully,
> on node-1 it may be set as "skip" and on node-1 as "remote-apply" so
> in such cases if conflict happens both nodes will have the value from
> node-1. But maybe it would be more difficult to get a consistent
> value if we are setting up a mess replication topology right? Maybe
> there I think a more advanced timestamp-based option would work better
> IMHO.
Yes, that's correct. We can get data divergence with resolvers like
'remote_apply', 'keep_local' etc. If you meant 'mesh' replication
topology, then yes, it is difficult to get consistent value there with
resolvers other than timestamp based. And thus timestamp based
resolvers are needed and should be the default when implemented.
thanks
Shveta
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: Conflict Detection and Resolution
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
2024-07-30 10:33 ` Re: Conflict Detection and Resolution Dilip Kumar <[email protected]>
2024-07-30 11:26 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
@ 2024-07-30 12:54 ` Dilip Kumar <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Dilip Kumar @ 2024-07-30 12:54 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Nisha Moond <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Bapat <[email protected]>; Jan Wieck <[email protected]>; Tomas Vondra <[email protected]>; pgsql-hackers; Zhijie Hou (Fujitsu) <[email protected]>
On Tue, Jul 30, 2024 at 4:56 PM shveta malik <[email protected]> wrote:
>
> On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar <[email protected]> wrote:
> >
> > On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian <[email protected]> wrote:
> >
> > Comment in 0002,
> >
> > 1) I do not see any test case that set a proper conflict type and
> > conflict resolver, all tests either give incorrect conflict
> > type/conflict resolver or the conflict resolver is ignored
> >
> > 0003
> > 2) I was trying to think about this patch, so suppose we consider this
> > case conflict_type-> update_differ resolver->remote_apply, my
> > question is to confirm whether my understanding is correct. So if
> > this is set and we have 2 nodes and set up a 2-way logical
> > replication, and if a conflict occurs node-1 will take the changes of
> > node-2 and node-2 will take the changes of node-1?
>
> Yes, that's right.
>
> > Maybe so I think
> > to avoid such cases user needs to set the resolver more thoughtfully,
> > on node-1 it may be set as "skip" and on node-1 as "remote-apply" so
> > in such cases if conflict happens both nodes will have the value from
> > node-1. But maybe it would be more difficult to get a consistent
> > value if we are setting up a mess replication topology right? Maybe
> > there I think a more advanced timestamp-based option would work better
> > IMHO.
>
> Yes, that's correct. We can get data divergence with resolvers like
> 'remote_apply', 'keep_local' etc. If you meant 'mesh' replication
> topology, then yes, it is difficult to get consistent value there with
> resolvers other than timestamp based. And thus timestamp based
> resolvers are needed and should be the default when implemented.
>
Thanks for the clarification.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* [PATCH v21 8/8] Allow to print raw parse tree.
@ 2024-08-26 04:32 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: Tatsuo Ishii @ 2024-08-26 04:32 UTC (permalink / raw)
---
src/backend/tcop/postgres.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8bc6bea113..f15659bf2b 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -659,6 +659,10 @@ pg_parse_query(const char *query_string)
#endif /* DEBUG_NODE_TESTS_ENABLED */
+ if (Debug_print_parse)
+ elog_node_display(LOG, "raw parse tree", raw_parsetree_list,
+ Debug_pretty_print);
+
TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
return raw_parsetree_list;
--
2.25.1
----Next_Part(Mon_Aug_26_13_39_47_2024_878)----
^ permalink raw reply [nested|flat] 27+ messages in thread
end of thread, other threads:[~2024-08-26 04:32 UTC | newest]
Thread overview: 27+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-07-26 10:49 [PATCH v3 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-08-09 07:56 [PATCH v4 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-09-02 06:32 [PATCH v5 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-09-12 05:22 [PATCH v6 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-09-22 04:53 [PATCH v7 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-09-25 05:01 [PATCH v8 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-10-04 05:51 [PATCH v9 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-10-22 02:22 [PATCH v10 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-11-08 06:57 [PATCH v11 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2023-12-04 11:23 [PATCH v12 7/7] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-01-22 09:45 [PATCH v13 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-02-28 13:59 [PATCH v14 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-03-28 10:30 [PATCH v15 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-04-12 06:49 [PATCH v16 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-04-28 11:00 [PATCH v17 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-05-11 07:11 [PATCH v18 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-05-14 23:26 [PATCH v19 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-05-24 02:26 [PATCH v20 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
2024-07-09 09:39 Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-17 06:01 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-26 04:20 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
2024-07-30 04:19 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-31 11:24 ` Re: Conflict Detection and Resolution Ajin Cherian <[email protected]>
2024-07-30 10:33 ` Re: Conflict Detection and Resolution Dilip Kumar <[email protected]>
2024-07-30 11:26 ` Re: Conflict Detection and Resolution shveta malik <[email protected]>
2024-07-30 12:54 ` Re: Conflict Detection and Resolution Dilip Kumar <[email protected]>
2024-08-26 04:32 [PATCH v21 8/8] Allow to print raw parse tree. Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox