public inbox for [email protected]
help / color / mirror / Atom feedFrom: Thomas Munro <[email protected]>
To: pgsql-hackers <[email protected]>
Subject: UBSan pointer overflow in xlogreader.c
Date: Wed, 6 Dec 2023 00:03:53 +1300
Message-ID: <CA+hUKGKH0oRPOX7DhiQ_b51sM8HqcPp2J3WA-Oen=dXog+AGGQ@mail.gmail.com> (raw)
Hi,
xlogreader.c has a pointer overflow bug, as revealed by the
combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl
test and Robert's incremental backup patch[1]. The bad code tests
whether an object could fit using something like base + size <= end,
which can be converted to something like size <= end - base to avoid
the overflow. See experimental fix patch, attached.
I took a while to follow up because I wanted to understand exactly why
it doesn't break in practice despite being that way since v15. I
think I have it now:
1. In the case of a bad/garbage size at end-of-WAL, the
following-page checks will fail first before anything bad happens as a
result of the overflow.
2. In the case of a real oversized record on current 64 bit
architectures including amd64, aarch64, power and riscv64, the pointer
can't really overflow anyway because the virtual address space is < 64
bit, typically around 48, and record lengths are 32 bit.
3. In the case of the 32 bit kernels I looked at including Linux,
FreeBSD and cousins, Solaris and Windows the top 1GB plus a bit more
of virtual address space is reserved for system use*, so I think a
real oversized record shouldn't be able to overflow the pointer there
either.
A 64 bit kernel running a 32 bit process could run into trouble,
though :-(. Those don't need to block out that high memory segment.
You'd need to have the WAL buffer in that address range and decode
large enough real WAL records and then things could break badly. I
guess 32/64 configurations must be rare these days outside developers
testing 32 bit code, and that is what happened here (ie CI); and with
some minor tweaks to the test it can be reached without Robert's patch
too. There may of course be other more exotic systems that could
break, but I don't know specifically what.
TLDR; this is a horrible bug, but all damage seems to be averted on
"normal" systems. The best thing I can say about all this is that the
new test found a bug, and the fix seems straightforward. I will study
and test this some more, but wanted to share what I have so far.
(*I think the old 32 bit macOS kernels might have been an exception to
this pattern but 32 bit kernels and even processes are history there.)
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGLCuW_CE9nDAbUNV40G2FkpY_kcPZkaORyVBVive8FQHQ%40m...
Attachments:
[application/octet-stream] 0001-Fix-pointer-overflow-in-xlogreader.c.patch (3.3K, ../CA+hUKGKH0oRPOX7DhiQ_b51sM8HqcPp2J3WA-Oen=dXog+AGGQ@mail.gmail.com/2-0001-Fix-pointer-overflow-in-xlogreader.c.patch)
download | inline diff:
From d9ef9a66d8cec5ace83af8f55014759249b9ef77 Mon Sep 17 00:00:00 2001
From: Thomas Munro <[email protected]>
Date: Wed, 22 Nov 2023 11:22:44 +1300
Subject: [PATCH] Fix pointer overflow in xlogreader.c.
While checking if a record could fit in the circular buffer, the coding
from commit 3f1ce973 used a formulation with pointer arithmetic that
could overflow. This couldn't break on known 64 bit systems for various
technical reasons, which probably explains the lack of problem reports.
Likewise for known 32 bit kernels. The systems at risk of problems
appear to be 32 bit processes running on 64 bit kernels.
Per complaint from ubsan.
---
src/backend/access/transam/xlogreader.c | 46 ++++++++++++++++++++-----
1 file changed, 37 insertions(+), 9 deletions(-)
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index e0baa86bd3..4dc6f06dd0 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -457,18 +457,37 @@ XLogReadRecordAlloc(XLogReaderState *state, size_t xl_tot_len, bool allow_oversi
if (state->decode_buffer_tail >= state->decode_buffer_head)
{
/* Empty, or tail is to the right of head. */
- if (state->decode_buffer_tail + required_space <=
- state->decode_buffer + state->decode_buffer_size)
+ if (required_space <=
+ state->decode_buffer_size -
+ (state->decode_buffer_tail - state->decode_buffer))
{
- /* There is space between tail and end. */
+ /*-
+ * There is space between tail and end.
+ *
+ * +-----+--------------------+-----+
+ * | |////////////////////|here!|
+ * +-----+--------------------+-----+
+ * ^ ^
+ * | |
+ * h t
+ */
decoded = (DecodedXLogRecord *) state->decode_buffer_tail;
decoded->oversized = false;
return decoded;
}
- else if (state->decode_buffer + required_space <
- state->decode_buffer_head)
+ else if (required_space <
+ state->decode_buffer_head - state->decode_buffer)
{
- /* There is space between start and head. */
+ /*-
+ * There is space between start and head.
+ *
+ * +-----+--------------------+-----+
+ * |here!|////////////////////| |
+ * +-----+--------------------+-----+
+ * ^ ^
+ * | |
+ * h t
+ */
decoded = (DecodedXLogRecord *) state->decode_buffer;
decoded->oversized = false;
return decoded;
@@ -477,10 +496,19 @@ XLogReadRecordAlloc(XLogReaderState *state, size_t xl_tot_len, bool allow_oversi
else
{
/* Tail is to the left of head. */
- if (state->decode_buffer_tail + required_space <
- state->decode_buffer_head)
+ if (required_space <
+ state->decode_buffer_head - state->decode_buffer_tail)
{
- /* There is space between tail and head. */
+ /*-
+ * There is space between tail and head.
+ *
+ * +-----+--------------------+-----+
+ * |/////|here! |/////|
+ * +-----+--------------------+-----+
+ * ^ ^
+ * | |
+ * t h
+ */
decoded = (DecodedXLogRecord *) state->decode_buffer_tail;
decoded->oversized = false;
return decoded;
--
2.39.3 (Apple Git-145)
view thread (2+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: UBSan pointer overflow in xlogreader.c
In-Reply-To: <CA+hUKGKH0oRPOX7DhiQ_b51sM8HqcPp2J3WA-Oen=dXog+AGGQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox