agora inbox for [email protected]
help / color / mirror / Atom feedRe: Update minimum SSL version
278+ messages / 4 participants
[nested] [flat]
* Re: Update minimum SSL version
@ 2019-12-02 13:09 Daniel Gustafsson <[email protected]>
2019-12-02 14:59 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
0 siblings, 2 replies; 278+ messages in thread
From: Daniel Gustafsson @ 2019-12-02 13:09 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Tom Lane <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> On 30 Nov 2019, at 03:43, Michael Paquier <[email protected]> wrote:
>
> On Fri, Nov 29, 2019 at 10:30:47AM -0500, Tom Lane wrote:
>> What's the impact going to be on buildfarm members with older openssl
>> installations? Perhaps "none", if they aren't running the ssl test
>> suite, but we should be clear about it.
>
> The buildfarm logs don't directly report the version of OpenSSL used
> as far as I recalled, and a quick lookup shows that..
Not explicitly, but it would be a nice if it did. Since the version depends on
the optional FIPS module, running "openssl version" is really the safe option,
which in itself is hard since the libraries pointed to with --with-libs aren't
guaranteed to have an openssl command installed etc. OpenSSL might also these
days be LibreSSL (or potentially even BoringSSL perhaps if someone twists the
arm of their installation enough).
However, looking at the signatures detected by autoconf we can however get an
idea of which version is used. SSL_clear_options and X509_get_signature_nid()
first shipped in 1.0.2, while SSL_get_current_compression first shipped in
0.9.8. There are also a set of functions which are new in 1.1.0 (BIO_get_data
et.al).
This tells us that for example alewife is likely running 1.0.2:
checking for SSL_new in -lssl... (cached) yes
checking for SSL_clear_options... (cached) no
checking for SSL_get_current_compression... (cached) yes
checking for X509_get_signature_nid... (cached) yes
checking for OPENSSL_init_ssl... (cached) no
checking for BIO_get_data... (cached) no
checking for BIO_meth_new... (cached) no
checking for ASN1_STRING_get0_data... (cached) no
(the careful observer notes that the SSL_clear_options() check fails even
though it should be in 1.0.2, and thats probably because SSL_clear_options is a
macro until 1.1.0 where it becomes a function).
gaur however looks like it is running 0.9.8:
checking for SSL_new in -lssl... yes
checking for SSL_clear_options... no
checking for SSL_get_current_compression... yes
checking for X509_get_signature_nid... no
checking for OPENSSL_init_ssl... no
checking for BIO_get_data... no
checking for BIO_meth_new... no
checking for ASN1_STRING_get0_data... no
checking for CRYPTO_lock... yes
scorpionfly running OpenBSD 6.6 configures as a LibreSSL on par with what we
expect for 1.1.0 (SSL_clear_options again fail here since it's still a macro in
LibreSSL):
checking for SSL_new in -lssl... (cached) yes
checking for SSL_clear_options... (cached) no
checking for SSL_get_current_compression... (cached) yes
checking for X509_get_signature_nid... (cached) yes
checking for OPENSSL_init_ssl... (cached) yes
checking for BIO_get_data... (cached) yes
checking for BIO_meth_new... (cached) yes
checking for ASN1_STRING_get0_data... (cached) yes
checking for CRYPTO_lock... (cached) yes
Randomly picking animals, and trying to target platforms where older versions
could be expected, I didn't see any <= 0.9.7; a small number 0.9.8 and most at
1.0.2 or higher (with the 0.9.8 animals being: gaur, sungazer and prairiedog).
This is not an exhaustive list of course, maybe someone with better access to
the buildfarm data can do some more clever analysis.
cheers ./daniel
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
@ 2019-12-02 14:59 ` Tom Lane <[email protected]>
2019-12-02 15:28 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-03 03:47 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
1 sibling, 2 replies; 278+ messages in thread
From: Tom Lane @ 2019-12-02 14:59 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Michael Paquier <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Daniel Gustafsson <[email protected]> writes:
> On 30 Nov 2019, at 03:43, Michael Paquier <[email protected]> wrote:
>> The buildfarm logs don't directly report the version of OpenSSL used
>> as far as I recalled, and a quick lookup shows that..
> Not explicitly, but it would be a nice if it did. Since the version depends on
> the optional FIPS module, running "openssl version" is really the safe option,
> which in itself is hard since the libraries pointed to with --with-libs aren't
> guaranteed to have an openssl command installed etc. OpenSSL might also these
> days be LibreSSL (or potentially even BoringSSL perhaps if someone twists the
> arm of their installation enough).
Yeah, I do not think that would be a good solution --- it would give wrong
answers on three of my four buildfarm animals :-(, for precisely the
reason that they're using --with-libs to point to a non-system openssl
installation.
Is there a simple way to ask the library itself for version info?
It might be worth the cycles to have configure run a small test
program to extract and print that data (not on cross-compile
builds, of course).
> (the careful observer notes that the SSL_clear_options() check fails even
> though it should be in 1.0.2, and thats probably because SSL_clear_options is a
> macro until 1.1.0 where it becomes a function).
Hmm, is it worth the trouble to fix that?
> gaur however looks like it is running 0.9.8:
gaur and prairiedog are both building with 0.9.8x, as you can tell
from their --with-libs options.
> Randomly picking animals, and trying to target platforms where older versions
> could be expected, I didn't see any <= 0.9.7; a small number 0.9.8 and most at
> 1.0.2 or higher (with the 0.9.8 animals being: gaur, sungazer and prairiedog).
According to the commit log (see 593d4e47d), we require 0.9.8 or later
in v10 and up, so any older animals got upgraded or retired some time
ago.
regards, tom lane
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-02 14:59 ` Re: Update minimum SSL version Tom Lane <[email protected]>
@ 2019-12-02 15:28 ` Daniel Gustafsson <[email protected]>
1 sibling, 0 replies; 278+ messages in thread
From: Daniel Gustafsson @ 2019-12-02 15:28 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Michael Paquier <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> On 2 Dec 2019, at 15:59, Tom Lane <[email protected]> wrote:
> Is there a simple way to ask the library itself for version info?
> It might be worth the cycles to have configure run a small test
> program to extract and print that data (not on cross-compile
> builds, of course).
Asking the lib is easy, making that fit cleanly into how autoconf does things might be trickier. I’ll take a look and will report back (on the SSL_clear_options thing as well).
cheers ./daniel
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-02 14:59 ` Re: Update minimum SSL version Tom Lane <[email protected]>
@ 2019-12-03 03:47 ` Michael Paquier <[email protected]>
1 sibling, 0 replies; 278+ messages in thread
From: Michael Paquier @ 2019-12-03 03:47 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Mon, Dec 02, 2019 at 09:59:44AM -0500, Tom Lane wrote:
> Is there a simple way to ask the library itself for version info?
> It might be worth the cycles to have configure run a small test
> program to extract and print that data (not on cross-compile
> builds, of course).
SSLeay_version():
https://www.openssl.org/docs/man1.0.2/man3/SSLeay_version.html
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
@ 2019-12-05 01:48 ` Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
1 sibling, 1 reply; 278+ messages in thread
From: Michael Paquier @ 2019-12-05 01:48 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Tom Lane <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Mon, Dec 02, 2019 at 02:09:51PM +0100, Daniel Gustafsson wrote:
> However, looking at the signatures detected by autoconf we can however get an
> idea of which version is used. SSL_clear_options and X509_get_signature_nid()
> first shipped in 1.0.2, while SSL_get_current_compression first shipped in
> 0.9.8. There are also a set of functions which are new in 1.1.0 (BIO_get_data
> et.al).
I was just looking at this problem, and something does not match with
what you wrote here. SSL_clear_options() is defined in OpenSSL from
0.9.8 to 1.0.2 as a macro (see ssl/ssl.h), and is defined as a
function since 1.1.0. So it seems to me that we are able to correctly
detect the presence of this function in the configure checks if
building with 1.1.0~, but not other versions.
In LibreSSL, the code has visibly always used a macro, even on their
latest HEAD since the code has been forked from OpenSSL 1.0.1g:
https://github.com/libressl-portable/openbsd. So we should be able
to compile our code, still we fail to detect that we can use the
macro.
It seems to me that we have quite a couple of arguments in favor of
dropping this configure check all together. (I saw the business
around a364dfa as well regarding NetBSD 5.1).
We can do more cleanup, and the discussion is quite different than the
original intent of this thread, so I am going to create a new one on
the matter.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
@ 2019-12-05 09:03 ` Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Re: Update minimum SSL version Tom Lane <[email protected]>
0 siblings, 1 reply; 278+ messages in thread
From: Daniel Gustafsson @ 2019-12-05 09:03 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Tom Lane <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> On 5 Dec 2019, at 02:48, Michael Paquier <[email protected]> wrote:
>
> On Mon, Dec 02, 2019 at 02:09:51PM +0100, Daniel Gustafsson wrote:
>> However, looking at the signatures detected by autoconf we can however get an
>> idea of which version is used. SSL_clear_options and X509_get_signature_nid()
>> first shipped in 1.0.2, while SSL_get_current_compression first shipped in
>> 0.9.8. There are also a set of functions which are new in 1.1.0 (BIO_get_data
>> et.al).
>
> I was just looking at this problem, and something does not match with
> what you wrote here. SSL_clear_options() is defined in OpenSSL from
> 0.9.8 to 1.0.2 as a macro (see ssl/ssl.h), and is defined as a
> function since 1.1.0.
Yes, I confused myself regarding the version for SSL_clear_options, except for
when it turned into a function.
> So it seems to me that we are able to correctly
> detect the presence of this function in the configure checks if
> building with 1.1.0~, but not other versions.
>
> In LibreSSL, the code has visibly always used a macro, even on their
> latest HEAD since the code has been forked from OpenSSL 1.0.1g:
> https://github.com/libressl-portable/openbsd. So we should be able
> to compile our code, still we fail to detect that we can use the
> macro.
Yes, we can't use AC_CHECK_FUNCS but would need to use AC_COMPILE_IFELSE (or a
similar check) in order to detect the macro.
> It seems to me that we have quite a couple of arguments in favor of
> dropping this configure check all together. (I saw the business
> around a364dfa as well regarding NetBSD 5.1).
>
> We can do more cleanup, and the discussion is quite different than the
> original intent of this thread, so I am going to create a new one on
> the matter.
Yes, if we're dropping older versions such that all supported versions have the
function, then keeping the autoconf check would be quite pointless.
cheers ./daniel
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
@ 2019-12-05 14:50 ` Tom Lane <[email protected]>
2019-12-05 22:29 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 278+ messages in thread
From: Tom Lane @ 2019-12-05 14:50 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Michael Paquier <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Daniel Gustafsson <[email protected]> writes:
>> On 5 Dec 2019, at 02:48, Michael Paquier <[email protected]> wrote:
>> So it seems to me that we are able to correctly
>> detect the presence of this function in the configure checks if
>> building with 1.1.0~, but not other versions.
> Yes, we can't use AC_CHECK_FUNCS but would need to use AC_COMPILE_IFELSE (or a
> similar check) in order to detect the macro.
configure already has a similar issue for isinf(). (I thought there
were more cases, actually, but I don't see another right now.)
We could just duplicate that logic, or maybe it's time to wrap it
up in an autoconf macro?
> Yes, if we're dropping older versions such that all supported versions have the
> function, then keeping the autoconf check would be quite pointless.
True as far as HEAD goes. What I'd like to know is whether not
realizing that SSL_clear_options is present causes any functional
issues that would justify back-patching a fix.
regards, tom lane
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Re: Update minimum SSL version Tom Lane <[email protected]>
@ 2019-12-05 22:29 ` Daniel Gustafsson <[email protected]>
2019-12-06 00:41 ` Re: Update minimum SSL version Tom Lane <[email protected]>
0 siblings, 1 reply; 278+ messages in thread
From: Daniel Gustafsson @ 2019-12-05 22:29 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Michael Paquier <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> On 5 Dec 2019, at 15:50, Tom Lane <[email protected]> wrote:
>
> Daniel Gustafsson <[email protected]> writes:
>>> On 5 Dec 2019, at 02:48, Michael Paquier <[email protected]> wrote:
>>> So it seems to me that we are able to correctly
>>> detect the presence of this function in the configure checks if
>>> building with 1.1.0~, but not other versions.
>
>> Yes, we can't use AC_CHECK_FUNCS but would need to use AC_COMPILE_IFELSE (or a
>> similar check) in order to detect the macro.
>
> configure already has a similar issue for isinf(). (I thought there
> were more cases, actually, but I don't see another right now.)
> We could just duplicate that logic, or maybe it's time to wrap it
> up in an autoconf macro?
>
>> Yes, if we're dropping older versions such that all supported versions have the
>> function, then keeping the autoconf check would be quite pointless.
>
> True as far as HEAD goes.
Good point.
> What I'd like to know is whether not
> realizing that SSL_clear_options is present causes any functional
> issues that would justify back-patching a fix.
ISTM that SSL_clear_options is required for turning on compression. Since
compression was introduced in 1.0.0 and SSL_clear_options was turned into a
function in 1.1.0, it affects 1.0.0, 1.0.1 and 1.0.2 with the latter two being
quite heavily used. I'm not sure how common it is to enable compression, and
especially how common it is post-CRIME, but since the option is there it seems
silly for it not to work with highly common library versions. Removing the
check only affects NetBSD 5, but breaking compilation in a stable release, even
for a rare OS, is I assume/hope a no-no. So thats a +1 from me for back-
patching a fix, while removing the check altogether in master.
The attached converts the check to use AC_LINK_IFELSE, in order to detect the
macro as well as the function (the compiled code is omitted for readability).
The patch is against master, but the check applies against backbranches except
for the AC_CHECK_FUNCS hunk which need tailoring per backbranch. I didn't
convert it to an autoconf macro, as there are only two callers in the
backbranches and it won't go into HEAD.
cheers ./daniel
Attachments:
[application/octet-stream] ssl_clear_options_check.patch (2.2K, ../../[email protected]/2-ssl_clear_options_check.patch)
download | inline diff:
From b6a6d29dbff2538c2faddbfb6cb176548a02e6cc Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <[email protected]>
Date: Thu, 5 Dec 2019 23:25:22 +0100
Subject: [PATCH] Fix autoconf check for SSL_clear_options
SSL_clear_options was implemented as a macro up until OpenSSL 1.1.0,
which means it cannot be reliably tested for with AC_CHECK_FUNCS as
that will only work for 1.1.0 and later. Reimplement the check with
a test that compiles to ensure the signature is present.
---
configure.in | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/configure.in b/configure.in
index a2cb20b5e3..37b59a2de4 100644
--- a/configure.in
+++ b/configure.in
@@ -1186,7 +1186,7 @@ if test "$with_openssl" = yes ; then
AC_SEARCH_LIBS(CRYPTO_new_ex_data, [eay32 crypto], [], [AC_MSG_ERROR([library 'eay32' or 'crypto' is required for OpenSSL])])
AC_SEARCH_LIBS(SSL_new, [ssleay32 ssl], [], [AC_MSG_ERROR([library 'ssleay32' or 'ssl' is required for OpenSSL])])
fi
- AC_CHECK_FUNCS([SSL_clear_options SSL_get_current_compression X509_get_signature_nid])
+ AC_CHECK_FUNCS([SSL_get_current_compression X509_get_signature_nid])
# Functions introduced in OpenSSL 1.1.0. We used to check for
# OPENSSL_VERSION_NUMBER, but that didn't work with 1.1.0, because LibreSSL
# defines OPENSSL_VERSION_NUMBER to claim version 2.0.0, even though it
@@ -1197,6 +1197,20 @@ if test "$with_openssl" = yes ; then
# thread-safety. In 1.1.0, it's no longer required, and CRYPTO_lock()
# function was removed.
AC_CHECK_FUNCS([CRYPTO_lock])
+ # SSL_clear_options is a macro in OpenSSL from 0.9.8 up until 1.0.2, so we
+ # can't test for it with AC_CHECK_FUNCS
+ AC_CACHE_CHECK([for SSL_clear_options], ac_cv_func_ssl_clear_options,
+ [AC_LINK_IFELSE([AC_LANG_PROGRAM([
+ #include <openssl/ssl.h>
+ #include <openssl/bio.h>
+ SSL *ssl;
+ ],
+ [return SSL_clear_options(ssl, 0);])],
+ [ac_cv_func_ssl_clear_options=yes],
+ [ac_cv_func_ssl_clear_options=no])])
+ if test $ac_cv_func_ssl_clear_options = yes ; then
+ AC_DEFINE(HAVE_SSL_CLEAR_OPTIONS, 1, [Define to 1 if you have SSL_clear_options()])
+ fi
fi
if test "$with_pam" = yes ; then
--
2.21.0 (Apple Git-122.2)
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-05 22:29 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
@ 2019-12-06 00:41 ` Tom Lane <[email protected]>
2019-12-06 00:59 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
0 siblings, 1 reply; 278+ messages in thread
From: Tom Lane @ 2019-12-06 00:41 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Michael Paquier <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Daniel Gustafsson <[email protected]> writes:
> On 5 Dec 2019, at 15:50, Tom Lane <[email protected]> wrote:
>> What I'd like to know is whether not
>> realizing that SSL_clear_options is present causes any functional
>> issues that would justify back-patching a fix.
> ISTM that SSL_clear_options is required for turning on compression. Since
> compression was introduced in 1.0.0 and SSL_clear_options was turned into a
> function in 1.1.0, it affects 1.0.0, 1.0.1 and 1.0.2 with the latter two being
> quite heavily used. I'm not sure how common it is to enable compression, and
> especially how common it is post-CRIME, but since the option is there it seems
> silly for it not to work with highly common library versions. Removing the
> check only affects NetBSD 5, but breaking compilation in a stable release, even
> for a rare OS, is I assume/hope a no-no. So thats a +1 from me for back-
> patching a fix, while removing the check altogether in master.
Agreed that we should do something about this. However, our requirement
for 0.9.8 or newer has been there since v10 (cf. 593d4e47d). So I think
what we should do is
(1) Back-patch Michael's
0002-Remove-configure-checks-for-SSL_clear_options-in-Ope.patch
from the other thread [1] as far as v10.
(2) Use this patch in 9.4-9.6.
It'd be possible to also backpatch the other thread's
0001-Remove-configure-checks-for-SSL_get_current_compress.patch
as far as v10, but I'm less excited about that -- it'd just save
a few configure cycles, no?
regards, tom lane
[1] https://www.postgresql.org/message-id/20191205083252.GE5064%40paquier.xyz
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-05 22:29 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-06 00:41 ` Re: Update minimum SSL version Tom Lane <[email protected]>
@ 2019-12-06 00:59 ` Michael Paquier <[email protected]>
2019-12-06 04:40 ` Re: Update minimum SSL version Tom Lane <[email protected]>
0 siblings, 1 reply; 278+ messages in thread
From: Michael Paquier @ 2019-12-06 00:59 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Thu, Dec 05, 2019 at 07:41:14PM -0500, Tom Lane wrote:
> It'd be possible to also backpatch the other thread's
> 0001-Remove-configure-checks-for-SSL_get_current_compress.patch
> as far as v10, but I'm less excited about that -- it'd just save
> a few configure cycles, no?
Yeah. I'd try not to meddle with stable branches more than necessary,
and the removal of the part for get_current_compression is just a
cleanup so I would just do that on HEAD and be done with it.
About clear_options, my take is to remove the check on HEAD, and to
apply Daniel's patch on *all* stable branches because I think that we
should not break the business that happened with NetBSD 5 on already
released branches. Does that sound good?
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-05 22:29 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-06 00:41 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-06 00:59 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
@ 2019-12-06 04:40 ` Tom Lane <[email protected]>
2019-12-06 06:20 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
0 siblings, 1 reply; 278+ messages in thread
From: Tom Lane @ 2019-12-06 04:40 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Michael Paquier <[email protected]> writes:
> About clear_options, my take is to remove the check on HEAD, and to
> apply Daniel's patch on *all* stable branches because I think that we
> should not break the business that happened with NetBSD 5 on already
> released branches. Does that sound good?
OK, re-reading the thread, I see the point --- old NetBSD has a weird
OpenSSL version that this would break. OK, let's stay compatible
with that on the back branches. So, your patch on HEAD and Daniel's
in the back branches is the right thing. Please push.
(Note: I didn't actually test Daniel's patch or read it closely ---
it looks like about the right thing, but please double check.)
regards, tom lane
^ permalink raw reply [nested|flat] 278+ messages in thread
* Re: Update minimum SSL version
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 01:48 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-05 09:03 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-05 22:29 ` Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-06 00:41 ` Re: Update minimum SSL version Tom Lane <[email protected]>
2019-12-06 00:59 ` Re: Update minimum SSL version Michael Paquier <[email protected]>
2019-12-06 04:40 ` Re: Update minimum SSL version Tom Lane <[email protected]>
@ 2019-12-06 06:20 ` Michael Paquier <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Michael Paquier @ 2019-12-06 06:20 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Magnus Hagander <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Thu, Dec 05, 2019 at 11:40:37PM -0500, Tom Lane wrote:
> OK, re-reading the thread, I see the point --- old NetBSD has a weird
> OpenSSL version that this would break. OK, let's stay compatible
> with that on the back branches. So, your patch on HEAD and Daniel's
> in the back branches is the right thing. Please push.
Thanks, applied. I have tested Daniel's version with OpenSSL 0.9.8,
1.0.2 and 1.1.0 and the test was able to detect correctly the
macro/function in all cases.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
* [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup.
@ 2024-05-18 01:41 Thomas Munro <[email protected]>
0 siblings, 0 replies; 278+ messages in thread
From: Thomas Munro @ 2024-05-18 01:41 UTC (permalink / raw)
When backend processes were launched in EXEC_BACKEND builds, they would
run LocalProcessControlFile() to read in pg_control and extract several
important settings.
This happens too early to acquire ControlFileLock, and the postmaster is
also not allowed to acquire ControlFileLock, so it can't safely take a
copy to give to the child.
Instead, pass down the "proto-controlfile" that was read by the
postmaster in LocalProcessControlFile(). Introduce functions
ExportProtoControlFile() and ImportProtoControlFile() to allow that.
Subprocesses will extract information from that, and then later attach
to the current control file in shared memory.
Reported-by: Melanie Plageman <[email protected]> per Windows CI failure
Discussion: https://postgr.es/m/CAAKRu_YNGwEYrorQYza_W8tU%2B%3DtoXRHG8HpyHC-KDbZqA_ZVSA%40mail.gmail.com
---
src/backend/access/transam/xlog.c | 46 +++++++++++++++++++++++--
src/backend/postmaster/launch_backend.c | 21 +++++++----
src/include/access/xlog.h | 5 +++
3 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13ec6225b85..e52517eb9c1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -575,6 +575,10 @@ static WALInsertLockPadded *WALInsertLocks = NULL;
*/
static ControlFileData *ControlFile = NULL;
+#ifdef EXEC_BACKEND
+static ControlFileData *ProtoControlFile = NULL;
+#endif
+
/*
* Calculate the amount of space left on the page after 'endptr'. Beware
* multiple evaluation!
@@ -692,6 +696,7 @@ static bool PerformRecoveryXLogAction(void);
static void InitControlFile(uint64 sysidentifier, uint32 data_checksum_version);
static void WriteControlFile(void);
static void ReadControlFile(void);
+static void ScanControlFile(void);
static void UpdateControlFile(void);
static char *str_time(pg_time_t tnow, char *buf, size_t bufsize);
@@ -4385,9 +4390,7 @@ WriteControlFile(void)
static void
ReadControlFile(void)
{
- pg_crc32c crc;
int fd;
- char wal_segsz_str[20];
int r;
/*
@@ -4420,6 +4423,15 @@ ReadControlFile(void)
close(fd);
+ ScanControlFile();
+}
+
+static void
+ScanControlFile(void)
+{
+ static char wal_segsz_str[20];
+ pg_crc32c crc;
+
/*
* Check for expected pg_control format version. If this is wrong, the
* CRC check will likely fail because we'll be checking the wrong number
@@ -4941,8 +4953,33 @@ LocalProcessControlFile(bool reset)
Assert(reset || ControlFile == NULL);
ControlFile = palloc_object(ControlFileData);
ReadControlFile();
+
+#ifdef EXEC_BACKEND
+ /* We need to be able to give this to subprocesses. */
+ ProtoControlFile = ControlFile;
+#endif
}
+#ifdef EXEC_BACKEND
+void
+ExportProtoControlFile(ControlFileData *copy)
+{
+ *copy = *ProtoControlFile;
+}
+
+/*
+ * Like LocalProcessControlFile(), but used early in EXEC_BACKEND children's
+ * startup. This receives the same file that the postmaster first read.
+ */
+void
+ImportProtoControlFile(const ControlFileData *copy)
+{
+ ControlFile = palloc(sizeof(ControlFileData));
+ *ControlFile = *copy;
+ ScanControlFile();
+}
+#endif
+
/*
* Get the wal_level from the control file. For a standby, this value should be
* considered as its active wal_level, because it may be different from what
@@ -5061,7 +5098,12 @@ XLOGShmemInit(void)
if (localControlFile)
{
memcpy(ControlFile, localControlFile, sizeof(ControlFileData));
+#ifdef EXEC_BACKEND
+ /* We still hold a reference to give to subprocesses. */
+ Assert(ProtoControlFile == localControlFile);
+#else
pfree(localControlFile);
+#endif
}
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 45690b11c99..e08a405f949 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,6 +33,9 @@
#include <unistd.h>
+#include "access/xlog.h"
+#include "catalog/pg_control.h"
+#include "common/file_utils.h"
#include "libpq/libpq-be.h"
#include "miscadmin.h"
#include "postmaster/autovacuum.h"
@@ -133,6 +136,14 @@ typedef struct
int MyPMChildSlot;
+ /*
+ * A copy of the ControlFileData from early in Postmaster startup. We
+ * need to access its contents it at a phase of initialization before we
+ * are allowed to acquire LWLocks, so we can't just use shared memory or
+ * read the file from disk.
+ */
+ ControlFileData proto_controlfile;
+
/*
* These are only used by backend processes, but are here because passing
* a socket needs some special handling on Windows. 'client_sock' is an
@@ -659,12 +670,6 @@ SubPostmasterMain(int argc, char *argv[])
*/
checkDataDir();
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
/*
* Reload any libraries that were preloaded by the postmaster. Since we
* exec'd this process, those libraries didn't come along with us; but we
@@ -752,6 +757,8 @@ save_backend_variables(BackendParameters *param,
param->MaxBackends = MaxBackends;
param->num_pmchild_slots = num_pmchild_slots;
+ ExportProtoControlFile(¶m->proto_controlfile);
+
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
if (!write_duplicated_handle(¶m->initial_signal_pipe,
@@ -1026,6 +1033,8 @@ restore_backend_variables(BackendParameters *param)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ ImportProtoControlFile(¶m->proto_controlfile);
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index fdfb572467b..f1f3ad4e96e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -208,6 +208,7 @@ typedef enum WALAvailability
struct XLogRecData;
struct XLogReaderState;
+struct ControlFileData;
extern XLogRecPtr XLogInsertRecord(struct XLogRecData *rdata,
XLogRecPtr fpw_lsn,
@@ -250,6 +251,10 @@ extern void XLOGShmemInit(void);
extern void BootStrapXLOG(uint32 data_checksum_version);
extern void InitializeWalConsistencyChecking(void);
extern void LocalProcessControlFile(bool reset);
+#ifdef EXEC_BACKEND
+extern void ExportProtoControlFile(struct ControlFileData *copy);
+extern void ImportProtoControlFile(const struct ControlFileData *copy);
+#endif
extern WalLevel GetActiveWalLevelOnStandby(void);
extern void StartupXLOG(void);
extern void ShutdownXLOG(int code, Datum arg);
--
2.47.3
--dhbc6bswyy6qufwn--
^ permalink raw reply [nested|flat] 278+ messages in thread
end of thread, other threads:[~2024-05-18 01:41 UTC | newest]
Thread overview: 278+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-12-02 13:09 Re: Update minimum SSL version Daniel Gustafsson <[email protected]>
2019-12-02 14:59 ` Tom Lane <[email protected]>
2019-12-02 15:28 ` Daniel Gustafsson <[email protected]>
2019-12-03 03:47 ` Michael Paquier <[email protected]>
2019-12-05 01:48 ` Michael Paquier <[email protected]>
2019-12-05 09:03 ` Daniel Gustafsson <[email protected]>
2019-12-05 14:50 ` Tom Lane <[email protected]>
2019-12-05 22:29 ` Daniel Gustafsson <[email protected]>
2019-12-06 00:41 ` Tom Lane <[email protected]>
2019-12-06 00:59 ` Michael Paquier <[email protected]>
2019-12-06 04:40 ` Tom Lane <[email protected]>
2019-12-06 06:20 ` Michael Paquier <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
2024-05-18 01:41 [PATCH v2] Fix pg_control corruption in EXEC_BACKEND startup. Thomas Munro <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox