pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
9+ messages / 2 participants
[nested] [flat]
Commits from this thread:
2020-08-15 06e50d8f7e 6af0b12a58 e8e44e8909 85aa22cb12 d8a22a387b Move new LOCKTAG_DATABASE_FROZEN_IDS to end of enum LockTagType.
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/d4031d78460cbbb4ed2fb7be635f84bea0e9a0c1
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
doc/src/sgml/monitoring.sgml | 20 +++++++++++++++--
src/backend/access/transam/slru.c | 8 +++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++---------
src/backend/commands/vacuum.c | 13 +++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++
src/backend/storage/lmgr/lwlocknames.txt | 2 ++
src/backend/utils/adt/lockfuncs.c | 12 +++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 +++++++++
11 files changed, 118 insertions(+), 15 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
REL_13_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/592a589a04bd456410b853d86bd05faa9432cbbb
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
doc/src/sgml/monitoring.sgml | 16 ++++++++++++++
src/backend/access/transam/slru.c | 8 +++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++---------
src/backend/commands/vacuum.c | 13 +++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++
src/backend/storage/lmgr/lwlocknames.txt | 3 +++
src/backend/utils/adt/lockfuncs.c | 12 +++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 +++++++++
11 files changed, 117 insertions(+), 13 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
REL_12_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/30e68a2abb3890c3292ff0b2422a7ea04d62acdd
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
doc/src/sgml/monitoring.sgml | 20 +++++++++++++++--
src/backend/access/transam/slru.c | 8 +++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++---------
src/backend/commands/vacuum.c | 13 +++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++
src/backend/storage/lmgr/lwlocknames.txt | 3 +++
src/backend/utils/adt/lockfuncs.c | 12 +++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 +++++++++
11 files changed, 119 insertions(+), 15 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/566372b3d6435639e4cc4476d79b8505a0297c87
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
doc/src/sgml/monitoring.sgml | 16 ++++++++++++++
src/backend/access/transam/slru.c | 8 +++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++---------
src/backend/commands/vacuum.c | 13 +++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++
src/backend/storage/lmgr/lwlocknames.txt | 3 +++
src/backend/utils/adt/lockfuncs.c | 12 +++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 +++++++++
11 files changed, 117 insertions(+), 13 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
REL_10_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/e525770dd504e5f182229b9bee64430ebe71a2a8
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
doc/src/sgml/monitoring.sgml | 20 +++++++++++++++--
src/backend/access/transam/slru.c | 8 +++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++---------
src/backend/commands/vacuum.c | 13 +++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++
src/backend/storage/lmgr/lwlock.c | 2 +-
src/backend/storage/lmgr/lwlocknames.txt | 2 ++
src/backend/utils/adt/lockfuncs.c | 12 +++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 +++++++++
12 files changed, 119 insertions(+), 16 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
REL9_5_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/3fbbf64ceecbd0fe675850070ea1e5d77c401c4f
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
src/backend/access/transam/slru.c | 8 ++++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++++----------
src/backend/commands/vacuum.c | 13 ++++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++++
src/backend/utils/adt/lockfuncs.c | 12 ++++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 ++++++++++
src/include/storage/lwlock.h | 4 +++-
10 files changed, 101 insertions(+), 14 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 17:20 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Noah Misch @ 2020-08-15 17:20 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent concurrent SimpleLruTruncate() for any given SLRU.
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
Branch
------
REL9_6_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/5ae01df6f6ef590fd73299050c7112f69c03f3de
Modified Files
--------------
doc/src/sgml/catalogs.sgml | 4 +++-
doc/src/sgml/monitoring.sgml | 20 +++++++++++++++--
src/backend/access/transam/slru.c | 8 +++++++
src/backend/access/transam/subtrans.c | 4 ++--
src/backend/commands/async.c | 37 +++++++++++++++++++++++---------
src/backend/commands/vacuum.c | 13 +++++++++++
src/backend/storage/lmgr/lmgr.c | 20 +++++++++++++++++
src/backend/storage/lmgr/lwlocknames.txt | 2 ++
src/backend/utils/adt/lockfuncs.c | 12 +++++++++++
src/include/storage/lmgr.h | 3 +++
src/include/storage/lock.h | 10 +++++++++
11 files changed, 118 insertions(+), 15 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU.
@ 2020-08-15 20:39 Tom Lane <tgl@sss.pgh.pa.us>
parent: Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2020-08-15 20:39 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: pgsql-committers@lists.postgresql.org
Noah Misch <noah@leadboat.com> writes:
> Prevent concurrent SimpleLruTruncate() for any given SLRU.
I find it fairly scary that you've changed enum LockTagType in the
back branches. Are we quite certain that no extensions have compiled-in
values of that enum?
Safer from an ABI standpoint would be to add the new value at the end,
at least in the back branches.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2020-08-15 20:39 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
2020-08-15 20:39 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
2020-08-15 17:20 pgsql: Prevent concurrent SimpleLruTruncate() for any given SLRU. Noah Misch <noah@leadboat.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox