pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Consistently test for in-use shared memory.
19+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2019-04-16 e12a472612 76097b42ae 4dc903947e 1254dbbd5b Don't write to stdin of a test process that could have already exited.
2019-04-04 6f0e190056 7dbc0759e0 e9d4cbeacb e9559e7add 0aa0ccfdd6 41547838b9 Silence -Wimplicit-fallthrough in sysv_shmem.c.
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-04 00:16 Noah Misch <noah@leadboat.com>
2019-04-04 02:00 ` Re: pgsql: Consistently test for in-use shared memory. Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 19+ messages in thread
From: Noah Misch @ 2019-04-04 00:16 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer recycle segments pertaining to other data directories.
That's good for production, but it's bad for integration tests that
crash a postmaster and immediately delete its data directory. Such a
test now leaks a segment indefinitely. No "make check-world" test does
that. win32_shmem.c already avoided all these problems. In 9.6 and
later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
(all supported versions).
Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/2f932f71d9f2963bbd201129d7b971c8f5f077fd
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 255 ++++++++++++++++++++----------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 184 ++++++++++++++++++++------
src/test/recovery/t/017_shm.pl | 154 ++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 468 insertions(+), 177 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
@ 2019-04-04 02:00 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 19+ messages in thread
From: Andres Freund @ 2019-04-04 02:00 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: pgsql-committers@lists.postgresql.org
Hi,
On 2019-04-04 00:16:55 +0000, Noah Misch wrote:
> Consistently test for in-use shared memory.
>
> postmaster startup scrutinizes any shared memory segment recorded in
> postmaster.pid, exiting if that segment matches the current data
> directory and has an attached process. When the postmaster.pid file was
> missing, a starting postmaster used weaker checks. Change to use the
> same checks in both scenarios. This increases the chance of a startup
> failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
> postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
> will no longer recycle segments pertaining to other data directories.
> That's good for production, but it's bad for integration tests that
> crash a postmaster and immediately delete its data directory. Such a
> test now leaks a segment indefinitely. No "make check-world" test does
> that. win32_shmem.c already avoided all these problems. In 9.6 and
> later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
> (all supported versions).
>
> Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
>
> Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
My compiler now nitpicks:
In file included from /home/andres/src/postgresql/src/include/postgres.h:47,
from pg_shmem.c:20:
pg_shmem.c: In function ‘PGSharedMemoryCreate’:
/home/andres/src/postgresql/src/include/utils/elog.h:122:5: warning: this statement may fall through [-Wimplicit-fallthrough=]
do { \
^
/home/andres/src/postgresql/src/include/utils/elog.h:142:2: note: in expansion of macro ‘ereport_domain’
ereport_domain(elevel, TEXTDOMAIN, rest)
^~~~~~~~~~~~~~
pg_shmem.c:668:5: note: in expansion of macro ‘ereport’
ereport(FATAL,
^~~~~~~
pg_shmem.c:675:4: note: here
case SHMSTATE_ENOENT:
^~~~
All of PostgreSQL successfully made. Ready to install.
I think the SHMSTATE_ATTACHED case simply should grow a (unreachable)
break. But I'd also strongly suggest adding one to SHMSTATE_UNATTACHED -
because that's actually reachable, and just seems like a trap for person
adding another case:.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-04 00:16 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-04 00:16 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer recycle segments pertaining to other data directories.
That's good for production, but it's bad for integration tests that
crash a postmaster and immediately delete its data directory. Such a
test now leaks a segment indefinitely. No "make check-world" test does
that. win32_shmem.c already avoided all these problems. In 9.6 and
later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
(all supported versions).
Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/b2307f8e3184fcfa7a1a789918c455c4e4e5bc06
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 255 ++++++++++++++++++++----------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 184 ++++++++++++++++++++------
src/test/recovery/t/017_shm.pl | 154 ++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 468 insertions(+), 177 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-04 00:16 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-04 00:16 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer recycle segments pertaining to other data directories.
That's good for production, but it's bad for integration tests that
crash a postmaster and immediately delete its data directory. Such a
test now leaks a segment indefinitely. No "make check-world" test does
that. win32_shmem.c already avoided all these problems. In 9.6 and
later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
(all supported versions).
Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
Branch
------
REL_10_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/7c414cdc39a3f183ff398be4cfcef0646f419281
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 255 ++++++++++++++++++++----------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 182 ++++++++++++++++++++-----
src/test/recovery/t/017_shm.pl | 154 ++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 467 insertions(+), 176 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-04 00:16 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-04 00:16 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer recycle segments pertaining to other data directories.
That's good for production, but it's bad for integration tests that
crash a postmaster and immediately delete its data directory. Such a
test now leaks a segment indefinitely. No "make check-world" test does
that. win32_shmem.c already avoided all these problems. In 9.6 and
later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
(all supported versions).
Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
Branch
------
REL9_6_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/891e9e0aee53977844946ef99936502e5d2942c9
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 255 ++++++++++++++++++++----------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 177 ++++++++++++++++++++-----
src/test/recovery/t/017_shm.pl | 154 ++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 464 insertions(+), 174 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-04 00:16 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-04 00:16 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer recycle segments pertaining to other data directories.
That's good for production, but it's bad for integration tests that
crash a postmaster and immediately delete its data directory. Such a
test now leaks a segment indefinitely. No "make check-world" test does
that. win32_shmem.c already avoided all these problems. In 9.6 and
later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
(all supported versions).
Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
Branch
------
REL9_5_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/e7f89a5dec0f95862d039d2ca9f0dcc3a1e20eb2
Modified Files
--------------
src/backend/port/sysv_shmem.c | 255 ++++++++++++++++++++----------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
7 files changed, 164 insertions(+), 138 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-04 00:16 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-04 00:16 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer recycle segments pertaining to other data directories.
That's good for production, but it's bad for integration tests that
crash a postmaster and immediately delete its data directory. Such a
test now leaks a segment indefinitely. No "make check-world" test does
that. win32_shmem.c already avoided all these problems. In 9.6 and
later, enhance PostgresNode to facilitate testing. Back-patch to 9.4
(all supported versions).
Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
Branch
------
REL9_4_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/3a70b66e625707bf3f8856be2a3d0ea74d6170d7
Modified Files
--------------
src/backend/port/sysv_shmem.c | 255 ++++++++++++++++++++----------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
7 files changed, 164 insertions(+), 138 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-13 05:53 Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 19+ messages in thread
From: Noah Misch @ 2019-04-13 05:53 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration
tests that crash a postmaster and immediately delete its data directory.
Such a test now leaks a segment indefinitely. No "make check-world"
test does that. win32_shmem.c already avoided all these problems. In
9.6 and later, enhance PostgresNode to facilitate testing. Back-patch
to 9.4 (all supported versions).
Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/c098509927f9a49ebceb301a2cb6a477ecd4ac3c
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 269 +++++++++++++++++++++---------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 184 +++++++++++++++++++-----
src/test/recovery/t/017_shm.pl | 200 +++++++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 525 insertions(+), 180 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
@ 2019-04-16 00:08 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 00:50 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 19+ messages in thread
From: Tom Lane @ 2019-04-16 00:08 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: pgsql-committers@lists.postgresql.org
Noah Misch <noah@leadboat.com> writes:
> Consistently test for in-use shared memory.
Hmm ... conchuela has just found another problem with this test case:
ok 2 - detected live backend via shared memory
# Running: postgres --single -D /home/pgbf/buildroot/HEAD/pgsql.build/src/test/recovery/tmp_check/t_017_shm_gnat_data/pgdata template1
ack Broken pipe: write( 16, 'SELECT 1 + 1' ) at /usr/local/lib/perl5/site_perl/IPC/Run/IO.pm line 549.
I interpret this as "the single-user backend exited before we could stuff
'SELECT 1 + 1' down the pipe to it, and the Perl script is not expecting
to get a write failure there".
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-04-16 00:50 ` Noah Misch <noah@leadboat.com>
2019-04-16 00:52 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 19+ messages in thread
From: Noah Misch @ 2019-04-16 00:50 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
On Mon, Apr 15, 2019 at 08:08:48PM -0400, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > Consistently test for in-use shared memory.
>
> Hmm ... conchuela has just found another problem with this test case:
>
> ok 2 - detected live backend via shared memory
> # Running: postgres --single -D /home/pgbf/buildroot/HEAD/pgsql.build/src/test/recovery/tmp_check/t_017_shm_gnat_data/pgdata template1
> ack Broken pipe: write( 16, 'SELECT 1 + 1' ) at /usr/local/lib/perl5/site_perl/IPC/Run/IO.pm line 549.
>
> I interpret this as "the single-user backend exited before we could stuff
> 'SELECT 1 + 1' down the pipe to it, and the Perl script is not expecting
> to get a write failure there".
Perhaps I should write the 'SELECT 1 + 1' to a regular file and redirect input
from that file.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 00:50 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
@ 2019-04-16 00:52 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:01 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 19+ messages in thread
From: Tom Lane @ 2019-04-16 00:52 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: pgsql-committers@lists.postgresql.org
Noah Misch <noah@leadboat.com> writes:
> On Mon, Apr 15, 2019 at 08:08:48PM -0400, Tom Lane wrote:
>> I interpret this as "the single-user backend exited before we could stuff
>> 'SELECT 1 + 1' down the pipe to it, and the Perl script is not expecting
>> to get a write failure there".
> Perhaps I should write the 'SELECT 1 + 1' to a regular file and redirect input
> from that file.
Yeah, that was the first idea that occurred to me as well.
Kinda grotty, but ...
Another idea is to shove the problem off on a sub-shell, that is run
echo SELECT 1 + 1 | postgres --single ...
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 00:50 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:52 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-04-16 01:01 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:18 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 19+ messages in thread
From: Tom Lane @ 2019-04-16 01:01 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: pgsql-committers@lists.postgresql.org
Noah Misch <noah@leadboat.com> writes:
>> Perhaps I should write the 'SELECT 1 + 1' to a regular file and redirect input
>> from that file.
Actually ... we don't need to run a query at all do we?
Although this would only help if Perl will optimize away an attempt
to write zero bytes to the pipe, which seems likely but not certain.
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 00:50 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:52 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:01 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-04-16 01:18 ` Noah Misch <noah@leadboat.com>
2019-04-16 01:34 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 19+ messages in thread
From: Noah Misch @ 2019-04-16 01:18 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
On Mon, Apr 15, 2019 at 09:01:45PM -0400, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> >> Perhaps I should write the 'SELECT 1 + 1' to a regular file and redirect input
> >> from that file.
>
> Actually ... we don't need to run a query at all do we?
No. We expect never to run one; it was there for the unexpected case of
"postgres --single" startup succeeding. I pushed a change to close the stdin
of "postgres --single" instead of writing to it. I probably worried that
"postgres --single" would not tolerate that, but it seems to.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: pgsql: Consistently test for in-use shared memory.
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 00:50 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:52 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:01 ` Re: pgsql: Consistently test for in-use shared memory. Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:18 ` Re: pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
@ 2019-04-16 01:34 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 19+ messages in thread
From: Tom Lane @ 2019-04-16 01:34 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: pgsql-committers@lists.postgresql.org
Noah Misch <noah@leadboat.com> writes:
> No. We expect never to run one; it was there for the unexpected case of
> "postgres --single" startup succeeding. I pushed a change to close the stdin
> of "postgres --single" instead of writing to it.
Ah, good, that's safer than what I was imagining.
> I probably worried that
> "postgres --single" would not tolerate that, but it seems to.
Sure. EOF-on-stdin is the standard way to end a standalone-backend
session. This fix means it'd have zero queries to execute rather
than one, but I'd be entirely willing to call it a backend bug if that
didn't work.
The more important point of course is that we must correctly detect
test failure if the backend doesn't complain. But since the test
explicitly looks for a particular error message text, that seems
fine.
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-13 05:53 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-13 05:53 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration
tests that crash a postmaster and immediately delete its data directory.
Such a test now leaks a segment indefinitely. No "make check-world"
test does that. win32_shmem.c already avoided all these problems. In
9.6 and later, enhance PostgresNode to facilitate testing. Back-patch
to 9.4 (all supported versions).
Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/7ef2b313e6494c2bfece61a2601eea31ebcb3430
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 269 +++++++++++++++++++++---------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 184 +++++++++++++++++++-----
src/test/recovery/t/017_shm.pl | 200 +++++++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 525 insertions(+), 180 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-13 05:53 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-13 05:53 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration
tests that crash a postmaster and immediately delete its data directory.
Such a test now leaks a segment indefinitely. No "make check-world"
test does that. win32_shmem.c already avoided all these problems. In
9.6 and later, enhance PostgresNode to facilitate testing. Back-patch
to 9.4 (all supported versions).
Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
Branch
------
REL_10_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/6d81e3c6526b6db229a68dde35593dda230e449f
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 269 +++++++++++++++++++++---------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 182 +++++++++++++++++++-----
src/test/recovery/t/017_shm.pl | 200 +++++++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 524 insertions(+), 179 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-13 05:53 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-13 05:53 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration
tests that crash a postmaster and immediately delete its data directory.
Such a test now leaks a segment indefinitely. No "make check-world"
test does that. win32_shmem.c already avoided all these problems. In
9.6 and later, enhance PostgresNode to facilitate testing. Back-patch
to 9.4 (all supported versions).
Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
Branch
------
REL9_6_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/808e1e75fb376ff7b39d0c3712fa85ff6e0f0720
Modified Files
--------------
src/Makefile.global.in | 4 +-
src/backend/port/sysv_shmem.c | 269 +++++++++++++++++++++---------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
src/test/perl/PostgresNode.pm | 177 +++++++++++++++++++-----
src/test/recovery/t/017_shm.pl | 200 +++++++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 1 +
11 files changed, 521 insertions(+), 177 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-13 05:53 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-13 05:53 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration
tests that crash a postmaster and immediately delete its data directory.
Such a test now leaks a segment indefinitely. No "make check-world"
test does that. win32_shmem.c already avoided all these problems. In
9.6 and later, enhance PostgresNode to facilitate testing. Back-patch
to 9.4 (all supported versions).
Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
Branch
------
REL9_5_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/c565de643167745c94ddeb0231c6de59ecf9da94
Modified Files
--------------
src/backend/port/sysv_shmem.c | 269 +++++++++++++++++++++---------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
7 files changed, 175 insertions(+), 141 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
* pgsql: Consistently test for in-use shared memory.
@ 2019-04-13 05:53 Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 19+ messages in thread
From: Noah Misch @ 2019-04-13 05:53 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Consistently test for in-use shared memory.
postmaster startup scrutinizes any shared memory segment recorded in
postmaster.pid, exiting if that segment matches the current data
directory and has an attached process. When the postmaster.pid file was
missing, a starting postmaster used weaker checks. Change to use the
same checks in both scenarios. This increases the chance of a startup
failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1
postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration
tests that crash a postmaster and immediately delete its data directory.
Such a test now leaks a segment indefinitely. No "make check-world"
test does that. win32_shmem.c already avoided all these problems. In
9.6 and later, enhance PostgresNode to facilitate testing. Back-patch
to 9.4 (all supported versions).
Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
Branch
------
REL9_4_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/3ef5e16c618177160bc5f95216dccab3588defdc
Modified Files
--------------
src/backend/port/sysv_shmem.c | 269 +++++++++++++++++++++---------------
src/backend/port/win32_shmem.c | 7 +-
src/backend/postmaster/postmaster.c | 12 +-
src/backend/storage/ipc/ipci.c | 14 +-
src/backend/utils/init/postinit.c | 6 +-
src/include/storage/ipc.h | 2 +-
src/include/storage/pg_shmem.h | 6 +-
7 files changed, 175 insertions(+), 141 deletions(-)
^ permalink raw reply [nested|flat] 19+ messages in thread
end of thread, other threads:[~2019-04-16 01:34 UTC | newest]
Thread overview: 19+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-04 02:00 ` Andres Freund <andres@anarazel.de>
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-04 00:16 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-16 00:08 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 00:50 ` Noah Misch <noah@leadboat.com>
2019-04-16 00:52 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:01 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-16 01:18 ` Noah Misch <noah@leadboat.com>
2019-04-16 01:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
2019-04-13 05:53 pgsql: Consistently test for in-use shared memory. Noah Misch <noah@leadboat.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox