pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
14+ messages / 1 participants
[nested] [flat]

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/8a54e12a38d1545d249f1402f66c8cde2837d97c

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 17 +++++----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  6 +--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 98 insertions(+), 37 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/d1ab4bf6ed2d5d5026e13af510d0d3b025fa6ac9

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 17 +++++----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  6 +--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 98 insertions(+), 37 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
REL9_5_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/77a0e14f0b21d34c31c3eabaf249844607680fbe

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 18 ++++-----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  8 ++--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 100 insertions(+), 38 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
REL9_6_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/d683d6528dba56669f24e5fac85c7692fc9ffd43

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 18 ++++-----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  6 +--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 98 insertions(+), 38 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/be843ce29737ee4752fedabe07a75cf9db1e969f

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 17 +++++----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  6 +--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 98 insertions(+), 37 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/179775135b41a0640ec8b378b2699dca00d3a49a

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 17 +++++----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  6 +--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 98 insertions(+), 37 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact
@ 2021-01-30 08:36  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-01-30 08:36 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions.  This expands the VirtualTransactionId structure domain to
admit prepared transactions.  It may be necessary to reindex to recover
from past occurrences.  Back-patch to 9.5 (all supported versions).

Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.

Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/86a5b309c9330661fd2c4c46e4dc7f07cca139e1

Modified Files
--------------
src/backend/storage/lmgr/lmgr.c                    |  3 +-
src/backend/storage/lmgr/lock.c                    | 43 +++++++++++++---------
src/include/storage/lock.h                         | 17 +++++----
src/test/isolation/Makefile                        | 11 +++---
src/test/isolation/README                          |  6 +--
.../expected/prepared-transactions-cic.out         | 18 +++++++++
.../isolation/specs/prepared-transactions-cic.spec | 37 +++++++++++++++++++
7 files changed, 98 insertions(+), 37 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/3cd9c3b921977272e6650a5efbeade4203c4bca2

Modified Files
--------------
contrib/amcheck/t/003_cic_2pc.pl      | 188 ++++++++++++++++++++++++++++++++++
src/backend/access/transam/twophase.c |  63 +++++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 ++++++++++++++-----
src/backend/utils/cache/inval.c       |  14 +++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
8 files changed, 362 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/fe5d44a1d38fc6c2a1adddf2eaf28726f93d2399

Modified Files
--------------
contrib/amcheck/t/003_cic_2pc.pl      | 188 ++++++++++++++++++++++++++++++++++
src/backend/access/transam/twophase.c |  63 +++++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 ++++++++++++++-----
src/backend/utils/cache/inval.c       |  14 +++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
src/test/perl/PostgresNode.pm         |  90 ++++++++++++++++
9 files changed, 452 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a9d0a54094153d8714a66444e89e565bb3eb21e4

Modified Files
--------------
contrib/amcheck/t/003_cic_2pc.pl      | 188 ++++++++++++++++++++++++++++++++++
src/backend/access/transam/twophase.c |  63 +++++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 ++++++++++++++-----
src/backend/utils/cache/inval.c       |  14 +++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
src/test/perl/PostgresNode.pm         |  90 ++++++++++++++++
9 files changed, 452 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
REL9_6_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/5184932432b414478edebd74d13d4d16c7ca070c

Modified Files
--------------
src/backend/access/transam/twophase.c |  60 +++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 +++++++++++++----
src/backend/utils/cache/inval.c       |  14 +++
src/bin/pgbench/t/023_cic_2pc.pl      | 205 ++++++++++++++++++++++++++++++++++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
src/test/perl/PostgresNode.pm         |  90 +++++++++++++++
9 files changed, 466 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/560124a37c2f6ef8db606f74eda442b48e3f9601

Modified Files
--------------
src/backend/access/transam/twophase.c |  63 ++++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 +++++++++++++----
src/backend/utils/cache/inval.c       |  14 +++
src/bin/pgbench/t/023_cic_2pc.pl      | 205 ++++++++++++++++++++++++++++++++++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
src/test/perl/PostgresNode.pm         |  90 +++++++++++++++
9 files changed, 469 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/df6158139f646f71ca5159d5ce173c11b06bb29d

Modified Files
--------------
src/backend/access/transam/twophase.c |  63 ++++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 +++++++++++++----
src/backend/utils/cache/inval.c       |  14 +++
src/bin/pgbench/t/023_cic_2pc.pl      | 205 ++++++++++++++++++++++++++++++++++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
src/test/perl/PostgresNode.pm         |  90 +++++++++++++++
9 files changed, 469 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread

* pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio
@ 2021-10-24 01:40  Noah Misch <noah@leadboat.com>
  0 siblings, 0 replies; 14+ messages in thread

From: Noah Misch @ 2021-10-24 01:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts.  Otherwise, things still broke.  As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows.  It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all
supported versions).

Andrey Borodin, reviewed (in earlier versions) by Andres Freund.

Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a5b9a0000e14663fd814dd51648a544fa40140a4

Modified Files
--------------
contrib/amcheck/t/003_cic_2pc.pl      | 188 ++++++++++++++++++++++++++++++++++
src/backend/access/transam/twophase.c |  63 +++++++++++-
src/backend/access/transam/xact.c     |   8 +-
src/backend/storage/lmgr/lmgr.c       |   7 +-
src/backend/storage/lmgr/lock.c       | 105 ++++++++++++++-----
src/backend/utils/cache/inval.c       |  14 +++
src/include/access/twophase.h         |   2 +
src/include/storage/lock.h            |  11 +-
8 files changed, 362 insertions(+), 36 deletions(-)



^ permalink  raw  reply  [nested|flat] 14+ messages in thread


end of thread, other threads:[~2021-10-24 01:40 UTC | newest]

Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-01-30 08:36 pgsql: Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transact Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>
2021-10-24 01:40 pgsql: Fix CREATE INDEX CONCURRENTLY for the newest prepared transactio Noah Misch <noah@leadboat.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox