public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 08/21] *an exclusive 29+ messages / 4 participants [nested] [flat]
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/21] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --jI8keyz6grp/JLjh Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH 08/18] *an exclusive @ 2021-02-06 21:13 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: Justin Pryzby @ 2021-02-06 21:13 UTC (permalink / raw) 3c84046490bed3c22e0873dc6ba492e02b8b9051 --- doc/src/sgml/ref/drop_index.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/sgml/ref/drop_index.sgml b/doc/src/sgml/ref/drop_index.sgml index 85cf23bca2..b6d2c2014f 100644 --- a/doc/src/sgml/ref/drop_index.sgml +++ b/doc/src/sgml/ref/drop_index.sgml @@ -45,7 +45,7 @@ DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ] <replaceable class="parameter">name</r <para> Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal <command>DROP INDEX</command> - acquires exclusive lock on the table, blocking other accesses until the + acquires an exclusive lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed. </para> -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-Doc-review-for-psql-dX.patch" ^ permalink raw reply [nested|flat] 29+ messages in thread
* Re: CDC/ETL system on top of logical replication with pgoutput, custom client @ 2023-07-31 13:31 Amit Kapila <[email protected]> 2023-07-31 14:16 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 0 siblings, 1 reply; 29+ messages in thread From: Amit Kapila @ 2023-07-31 13:31 UTC (permalink / raw) To: José Neves <[email protected]>; +Cc: pgsql-hackers On Mon, Jul 31, 2023 at 3:06 PM José Neves <[email protected]> wrote: > > Hi there, hope to find you well. > > I'm attempting to develop a CDC on top of Postgres, currently using 12, the last minor, with a custom client, and I'm running into issues with data loss caused by out-of-order logical replication messages. > > The problem is as follows: postgres streams A, B, D, G, K, I, P logical replication events, upon exit signal we stop consuming new events at LSN K, and we wait 30s for out-of-order events. Let's say that we only got A, (and K ofc) so in the following 30s, we get B, D, however, for whatever reason, G never arrived. As with pgoutput-based logical replication we have no way to calculate the next LSN, we have no idea that G was missing, so we assumed that it all arrived, committing K to postgres slot and shutdown. In the next run, our worker will start receiving data from K forward, and G is lost forever... > Meanwhile postgres moves forward with archiving and we can't go back to check if we lost anything. And even if we could, would be extremely inefficient. > > In sum, the issue comes from the fact that postgres will stream events with unordered LSNs on high transactional systems, and that pgoutput doesn't have access to enough information to calculate the next or last LSN, so we have no way to check if we receive all the data that we are supposed to receive, risking committing an offset that we shouldn't as we didn't receive yet preceding data. > As per my understanding, we stream the data in the commit LSN order and for a particular transaction, all the changes are per their LSN order. Now, it is possible that for a parallel transaction, we send some changes from a prior LSN after sending the commit of another transaction. Say we have changes as follows: T-1 change1 LSN1-1000 change2 LSN2- 2000 commit LSN3- 3000 T-2 change1 LSN1-500 change2 LSN2-1500 commit LSN3-4000 In such a case, all the changes including the commit of T-1 are sent and then all the changes including the commit of T-2 are sent. So, one can say that some of the changes from T-2 from prior LSN arrived after T-1's commit but that shouldn't be a problem because if restart happens after we received partial T-2, we should receive the entire T-2. It is possible that you are seeing something else but if so then please try to share a more concrete example. -- With Regards, Amit Kapila. ^ permalink raw reply [nested|flat] 29+ messages in thread
* RE: CDC/ETL system on top of logical replication with pgoutput, custom client 2023-07-31 13:31 Re: CDC/ETL system on top of logical replication with pgoutput, custom client Amit Kapila <[email protected]> @ 2023-07-31 14:16 ` José Neves <[email protected]> 2023-07-31 20:39 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> 0 siblings, 1 reply; 29+ messages in thread From: José Neves @ 2023-07-31 14:16 UTC (permalink / raw) To: Amit Kapila <[email protected]>; +Cc: pgsql-hackers Hi Amit, thanks for the reply. In our worker (custom pg replication client), we care only about INSERT, UPDATE, and DELETE operations, which - sure - may be part of the issue. I can only replicate this with production-level load, not easy to get a real example, but as I'm understanding the issue (and building upon your exposition), we are seeing the following: T-1 INSERT LSN1-1000 UPDATE LSN2-2000 UPDATE LSN3-3000 COMMIT LSN4-4000 T-2 INSERT LSN1-500 UPDATE LSN2-1500 UPDATE LSN3-2500 COMMIT LSN4-5500 If we miss LSN3-3000, let's say, a bad network, and we already received all other LSNs, we will commit to Postgres LSN4-5500 before restarting. LSN3 3000 will never be reattempted. And there are a couple of issues with this scenery: 1. We have no way to match LSN operations with the respective commit, as they have unordered offsets. Assuming that all of them were received in order, we would commit all data with the commit message LSN4-4000 as other events would match the transaction start and end LSN interval of it. 2. Still we have no way to verify that we got all data for a given transaction, we will never miss LSN3-3000 of the first transaction till we look at and analyze the resulting data. So the question: how can we prevent our worker from committing LSN4-5500 without receiving LSN3-3000? Do we even have enough information out of pgoutput to do that? PS.: when I say bad network, my suspicion is that this situation may be caused by network saturation on high QPS periods. Data will still arrive eventually but by that time our worker is no longer listening. Thanks again. Regards, José Neves ________________________________ De: Amit Kapila <[email protected]> Enviado: 31 de julho de 2023 14:31 Para: José Neves <[email protected]> Cc: [email protected] <[email protected]> Assunto: Re: CDC/ETL system on top of logical replication with pgoutput, custom client On Mon, Jul 31, 2023 at 3:06 PM José Neves <[email protected]> wrote: > > Hi there, hope to find you well. > > I'm attempting to develop a CDC on top of Postgres, currently using 12, the last minor, with a custom client, and I'm running into issues with data loss caused by out-of-order logical replication messages. > > The problem is as follows: postgres streams A, B, D, G, K, I, P logical replication events, upon exit signal we stop consuming new events at LSN K, and we wait 30s for out-of-order events. Let's say that we only got A, (and K ofc) so in the following 30s, we get B, D, however, for whatever reason, G never arrived. As with pgoutput-based logical replication we have no way to calculate the next LSN, we have no idea that G was missing, so we assumed that it all arrived, committing K to postgres slot and shutdown. In the next run, our worker will start receiving data from K forward, and G is lost forever... > Meanwhile postgres moves forward with archiving and we can't go back to check if we lost anything. And even if we could, would be extremely inefficient. > > In sum, the issue comes from the fact that postgres will stream events with unordered LSNs on high transactional systems, and that pgoutput doesn't have access to enough information to calculate the next or last LSN, so we have no way to check if we receive all the data that we are supposed to receive, risking committing an offset that we shouldn't as we didn't receive yet preceding data. > As per my understanding, we stream the data in the commit LSN order and for a particular transaction, all the changes are per their LSN order. Now, it is possible that for a parallel transaction, we send some changes from a prior LSN after sending the commit of another transaction. Say we have changes as follows: T-1 change1 LSN1-1000 change2 LSN2- 2000 commit LSN3- 3000 T-2 change1 LSN1-500 change2 LSN2-1500 commit LSN3-4000 In such a case, all the changes including the commit of T-1 are sent and then all the changes including the commit of T-2 are sent. So, one can say that some of the changes from T-2 from prior LSN arrived after T-1's commit but that shouldn't be a problem because if restart happens after we received partial T-2, we should receive the entire T-2. It is possible that you are seeing something else but if so then please try to share a more concrete example. -- With Regards, Amit Kapila. ^ permalink raw reply [nested|flat] 29+ messages in thread
* Re: CDC/ETL system on top of logical replication with pgoutput, custom client 2023-07-31 13:31 Re: CDC/ETL system on top of logical replication with pgoutput, custom client Amit Kapila <[email protected]> 2023-07-31 14:16 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> @ 2023-07-31 20:39 ` Andres Freund <[email protected]> 2023-07-31 21:25 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 0 siblings, 1 reply; 29+ messages in thread From: Andres Freund @ 2023-07-31 20:39 UTC (permalink / raw) To: José Neves <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers Hi, On 2023-07-31 14:16:22 +0000, José Neves wrote: > Hi Amit, thanks for the reply. > > In our worker (custom pg replication client), we care only about INSERT, > UPDATE, and DELETE operations, which - sure - may be part of the issue. That seems likely. Postgres streams out changes in commit order, not in order of the changes having been made (that'd not work due to rollbacks etc). If you just disregard transactions entirely, you'll get something bogus after retries. You don't need to store the details for each commit in the target system, just up to which LSN you have processed *commit records*. E.g. if you have received and safely stored up to commit 0/1000, you need to remember that. Are you using the 'streaming' mode / option to pgoutput? > 1. We have no way to match LSN operations with the respective commit, as > they have unordered offsets. Not sure what you mean with "unordered offsets"? > Assuming that all of them were received in order, we would commit all data with the commit message LSN4-4000 as other events would match the transaction start and end LSN interval of it. Logical decoding sends out changes in a deterministic order and you won't see out of order data when using TCP (the entire connection can obviously fail though). Andres ^ permalink raw reply [nested|flat] 29+ messages in thread
* RE: CDC/ETL system on top of logical replication with pgoutput, custom client 2023-07-31 13:31 Re: CDC/ETL system on top of logical replication with pgoutput, custom client Amit Kapila <[email protected]> 2023-07-31 14:16 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 2023-07-31 20:39 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> @ 2023-07-31 21:25 ` José Neves <[email protected]> 2023-07-31 23:21 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> 0 siblings, 1 reply; 29+ messages in thread From: José Neves @ 2023-07-31 21:25 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers Hi Andres, thanks for your reply. Ok, if I understood you correctly, I start to see where my logic is faulty. Just to make sure that I got it right, taking the following example again: T-1 INSERT LSN1-1000 UPDATE LSN2-2000 UPDATE LSN3-3000 COMMIT LSN4-4000 T-2 INSERT LSN1-500 UPDATE LSN2-1500 UPDATE LSN3-2500 COMMIT LSN4-5500 Where data will arrive in this order: INSERT LSN1-500 INSERT LSN1-1000 UPDATE LSN2-1500 UPDATE LSN2-2000 UPDATE LSN3-2500 UPDATE LSN3-3000 COMMIT LSN4-4000 COMMIT LSN4-5500 You are saying that the LSN3-3000 will never be missing, either the entire connection will fail at that point, or all should be received in the expected order (which is different from the "numeric order" of LSNs). If the connection is down, upon restart, I will receive the entire T-1 transaction again (well, all example data again). In addition to that, if I commit LSN4-4000, even tho that LSN has a "bigger numeric value" than the ones representing INSERT and UPDATE events on T-2, I will be receiving the entire T-2 transaction again, as the LSN4-5500 is still uncommitted. This makes sense to me, but just to be extra clear, I will never receive a transaction commit before receiving all other events for that transaction. Are these statements correct? >Are you using the 'streaming' mode / option to pgoutput? No. >Not sure what you mean with "unordered offsets"? Ordered: EB53/E0D88188, EB53/E0D88189, EB53/E0D88190 Unordered: EB53/E0D88190, EB53/E0D88188, EB53/E0D88189 Extra question: When I get a begin message, I get a transaction starting at LSN-1000, and a transaction ending at LSN-2000. But as the example above shows, I can have data points from other transactions with LSNs in that interval. I have no way to identify to which transaction they belong, correct? Thanks again. Regards, José Neves ________________________________ De: Andres Freund <[email protected]> Enviado: 31 de julho de 2023 21:39 Para: José Neves <[email protected]> Cc: Amit Kapila <[email protected]>; [email protected] <[email protected]> Assunto: Re: CDC/ETL system on top of logical replication with pgoutput, custom client Hi, On 2023-07-31 14:16:22 +0000, José Neves wrote: > Hi Amit, thanks for the reply. > > In our worker (custom pg replication client), we care only about INSERT, > UPDATE, and DELETE operations, which - sure - may be part of the issue. That seems likely. Postgres streams out changes in commit order, not in order of the changes having been made (that'd not work due to rollbacks etc). If you just disregard transactions entirely, you'll get something bogus after retries. You don't need to store the details for each commit in the target system, just up to which LSN you have processed *commit records*. E.g. if you have received and safely stored up to commit 0/1000, you need to remember that. Are you using the 'streaming' mode / option to pgoutput? > 1. We have no way to match LSN operations with the respective commit, as > they have unordered offsets. Not sure what you mean with "unordered offsets"? > Assuming that all of them were received in order, we would commit all data with the commit message LSN4-4000 as other events would match the transaction start and end LSN interval of it. Logical decoding sends out changes in a deterministic order and you won't see out of order data when using TCP (the entire connection can obviously fail though). Andres ^ permalink raw reply [nested|flat] 29+ messages in thread
* Re: CDC/ETL system on top of logical replication with pgoutput, custom client 2023-07-31 13:31 Re: CDC/ETL system on top of logical replication with pgoutput, custom client Amit Kapila <[email protected]> 2023-07-31 14:16 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 2023-07-31 20:39 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> 2023-07-31 21:25 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> @ 2023-07-31 23:21 ` Andres Freund <[email protected]> 2023-08-01 09:13 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 0 siblings, 1 reply; 29+ messages in thread From: Andres Freund @ 2023-07-31 23:21 UTC (permalink / raw) To: José Neves <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers Hi, On 2023-07-31 21:25:06 +0000, José Neves wrote: > Ok, if I understood you correctly, I start to see where my logic is faulty. Just to make sure that I got it right, taking the following example again: > > T-1 > INSERT LSN1-1000 > UPDATE LSN2-2000 > UPDATE LSN3-3000 > COMMIT LSN4-4000 > > T-2 > INSERT LSN1-500 > UPDATE LSN2-1500 > UPDATE LSN3-2500 > COMMIT LSN4-5500 > > Where data will arrive in this order: > > INSERT LSN1-500 > INSERT LSN1-1000 > UPDATE LSN2-1500 > UPDATE LSN2-2000 > UPDATE LSN3-2500 > UPDATE LSN3-3000 > COMMIT LSN4-4000 > COMMIT LSN4-5500 No, they won't arrive in that order. They will arive as BEGIN INSERT LSN1-1000 UPDATE LSN2-2000 UPDATE LSN3-3000 COMMIT LSN4-4000 BEGIN INSERT LSN1-500 UPDATE LSN2-1500 UPDATE LSN3-2500 COMMIT LSN4-5500 Because T1 committed before T2. Changes are only streamed out at commit / prepare transaction (*). Within a transaction, they however *will* be ordered by LSN. (*) Unless you use streaming mode, in which case it'll all be more complicated, as you'll also receive changes for transactions that might still abort. > You are saying that the LSN3-3000 will never be missing, either the entire > connection will fail at that point, or all should be received in the > expected order (which is different from the "numeric order" of LSNs). I'm not quite sure what you mean with the "different from the numeric order" bit... > If the connection is down, upon restart, I will receive the entire T-1 > transaction again (well, all example data again). Yes, unless you already acknowledged receipt up to LSN4-4000 and/or are only asking for newer transactions when reconnecting. > In addition to that, if I commit LSN4-4000, even tho that LSN has a "bigger > numeric value" than the ones representing INSERT and UPDATE events on T-2, I > will be receiving the entire T-2 transaction again, as the LSN4-5500 is > still uncommitted. I don't quite know what you mean with "commit LSN4-4000" here. > This makes sense to me, but just to be extra clear, I will never receive a > transaction commit before receiving all other events for that transaction. Correct. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 29+ messages in thread
* RE: CDC/ETL system on top of logical replication with pgoutput, custom client 2023-07-31 13:31 Re: CDC/ETL system on top of logical replication with pgoutput, custom client Amit Kapila <[email protected]> 2023-07-31 14:16 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 2023-07-31 20:39 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> 2023-07-31 21:25 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 2023-07-31 23:21 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> @ 2023-08-01 09:13 ` José Neves <[email protected]> 0 siblings, 0 replies; 29+ messages in thread From: José Neves @ 2023-08-01 09:13 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers Hi Andres. Owh, I see the error of my way... :( By ignoring commits, and committing individual operation LSNs, I was effectively rolling back the subscription. In the previous example, if I committed the LSN of the first insert of the second transaction (LSN1-500), I was basically telling Postgres to send everything again, including the already processed T1. > what you mean with the "different from the numeric order" I'm probably lacking terminology. I mean that LSN4-5500 > LSN4-4000 > LSN3-3000 > LSN3-2500... But, if I'm understanding correctly, I can only rely on the incremental sequence to be true for the commit events. Which explains my pain. The world makes sense again. Thank you very much. Will try to implement this new logic, and hopefully not bug again with this issue. Regards, José Neves ________________________________ De: Andres Freund <[email protected]> Enviado: 1 de agosto de 2023 00:21 Para: José Neves <[email protected]> Cc: Amit Kapila <[email protected]>; [email protected] <[email protected]> Assunto: Re: CDC/ETL system on top of logical replication with pgoutput, custom client Hi, On 2023-07-31 21:25:06 +0000, José Neves wrote: > Ok, if I understood you correctly, I start to see where my logic is faulty. Just to make sure that I got it right, taking the following example again: > > T-1 > INSERT LSN1-1000 > UPDATE LSN2-2000 > UPDATE LSN3-3000 > COMMIT LSN4-4000 > > T-2 > INSERT LSN1-500 > UPDATE LSN2-1500 > UPDATE LSN3-2500 > COMMIT LSN4-5500 > > Where data will arrive in this order: > > INSERT LSN1-500 > INSERT LSN1-1000 > UPDATE LSN2-1500 > UPDATE LSN2-2000 > UPDATE LSN3-2500 > UPDATE LSN3-3000 > COMMIT LSN4-4000 > COMMIT LSN4-5500 No, they won't arrive in that order. They will arive as BEGIN INSERT LSN1-1000 UPDATE LSN2-2000 UPDATE LSN3-3000 COMMIT LSN4-4000 BEGIN INSERT LSN1-500 UPDATE LSN2-1500 UPDATE LSN3-2500 COMMIT LSN4-5500 Because T1 committed before T2. Changes are only streamed out at commit / prepare transaction (*). Within a transaction, they however *will* be ordered by LSN. (*) Unless you use streaming mode, in which case it'll all be more complicated, as you'll also receive changes for transactions that might still abort. > You are saying that the LSN3-3000 will never be missing, either the entire > connection will fail at that point, or all should be received in the > expected order (which is different from the "numeric order" of LSNs). I'm not quite sure what you mean with the "different from the numeric order" bit... > If the connection is down, upon restart, I will receive the entire T-1 > transaction again (well, all example data again). Yes, unless you already acknowledged receipt up to LSN4-4000 and/or are only asking for newer transactions when reconnecting. > In addition to that, if I commit LSN4-4000, even tho that LSN has a "bigger > numeric value" than the ones representing INSERT and UPDATE events on T-2, I > will be receiving the entire T-2 transaction again, as the LSN4-5500 is > still uncommitted. I don't quite know what you mean with "commit LSN4-4000" here. > This makes sense to me, but just to be extra clear, I will never receive a > transaction commit before receiving all other events for that transaction. Correct. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 29+ messages in thread
end of thread, other threads:[~2023-08-01 09:13 UTC | newest] Thread overview: 29+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/21] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2021-02-06 21:13 [PATCH 08/18] *an exclusive Justin Pryzby <[email protected]> 2023-07-31 13:31 Re: CDC/ETL system on top of logical replication with pgoutput, custom client Amit Kapila <[email protected]> 2023-07-31 14:16 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 2023-07-31 20:39 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> 2023-07-31 21:25 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]> 2023-07-31 23:21 ` Re: CDC/ETL system on top of logical replication with pgoutput, custom client Andres Freund <[email protected]> 2023-08-01 09:13 ` RE: CDC/ETL system on top of logical replication with pgoutput, custom client José Neves <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox