pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fail hard on out-of-memory failures in xlogreader.c
6+ messages / 1 participants
[nested] [flat]

* pgsql: Fail hard on out-of-memory failures in xlogreader.c
@ 2023-10-03 01:26  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-10-03 01:26 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fail hard on out-of-memory failures in xlogreader.c

This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/6b18b3fe2c2f375d4a5d8a69b67bc59d13d4c844

Modified Files
--------------
src/backend/access/transam/xlogreader.c | 47 ++++++---------------------------
1 file changed, 8 insertions(+), 39 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fail hard on out-of-memory failures in xlogreader.c
@ 2023-10-03 01:26  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-10-03 01:26 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fail hard on out-of-memory failures in xlogreader.c

This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12

Branch
------
REL_16_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a06efbc3ac72a455a63635c9cbc56ad3338988cc

Modified Files
--------------
src/backend/access/transam/xlogreader.c | 47 ++++++---------------------------
1 file changed, 8 insertions(+), 39 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fail hard on out-of-memory failures in xlogreader.c
@ 2023-10-03 01:26  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-10-03 01:26 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fail hard on out-of-memory failures in xlogreader.c

This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12

Branch
------
REL_15_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/afc79591de8e2facc3e22f8746ee2b7e59a2a741

Modified Files
--------------
src/backend/access/transam/xlogreader.c | 47 ++++++---------------------------
1 file changed, 8 insertions(+), 39 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fail hard on out-of-memory failures in xlogreader.c
@ 2023-10-03 01:26  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-10-03 01:26 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fail hard on out-of-memory failures in xlogreader.c

This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/baeb8542c1cc849e58c016f401687b9c4d3e7f20

Modified Files
--------------
src/backend/access/transam/xlogreader.c | 31 +++++--------------------------
1 file changed, 5 insertions(+), 26 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fail hard on out-of-memory failures in xlogreader.c
@ 2023-10-03 01:26  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-10-03 01:26 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fail hard on out-of-memory failures in xlogreader.c

This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/6103d2ccaed48f8e9599d67f96302d9bae208a8e

Modified Files
--------------
src/backend/access/transam/xlogreader.c | 32 +++++---------------------------
1 file changed, 5 insertions(+), 27 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Fail hard on out-of-memory failures in xlogreader.c
@ 2023-10-03 01:26  Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 6+ messages in thread

From: Michael Paquier @ 2023-10-03 01:26 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fail hard on out-of-memory failures in xlogreader.c

This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/22b2e6e9df54f4d38e2fd20a882505d87c1f169f

Modified Files
--------------
src/backend/access/transam/xlogreader.c | 32 +++++---------------------------
1 file changed, 5 insertions(+), 27 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2023-10-03 01:26 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-10-03 01:26 pgsql: Fail hard on out-of-memory failures in xlogreader.c Michael Paquier <michael@paquier.xyz>
2023-10-03 01:26 pgsql: Fail hard on out-of-memory failures in xlogreader.c Michael Paquier <michael@paquier.xyz>
2023-10-03 01:26 pgsql: Fail hard on out-of-memory failures in xlogreader.c Michael Paquier <michael@paquier.xyz>
2023-10-03 01:26 pgsql: Fail hard on out-of-memory failures in xlogreader.c Michael Paquier <michael@paquier.xyz>
2023-10-03 01:26 pgsql: Fail hard on out-of-memory failures in xlogreader.c Michael Paquier <michael@paquier.xyz>
2023-10-03 01:26 pgsql: Fail hard on out-of-memory failures in xlogreader.c Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox