pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Prevent port collisions between concurrent TAP tests
9+ messages / 2 participants
[nested] [flat]
* pgsql: Prevent port collisions between concurrent TAP tests
@ 2022-11-20 15:07 Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-20 15:07 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent port collisions between concurrent TAP tests
Currently there is a race condition where if concurrent TAP tests both
test that they can open a port they will assume that it is free and use
it, causing one of them to fail. To prevent this we record a reservation
using an exclusive lock, and any TAP test that discovers a reservation
checks to see if the reserving process is still alive, and looks for
another free port if it is.
Ports are reserved in a directory set by the environment setting
PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
build directory as set by meson or Makefile.global, or its own
tmp_check directory.
The prove_check recipe in Makefile.global.in is extended to export
top_builddir to the TAP tests. This was already exported by the
prove_installcheck recipes.
Per complaint from Andres Freund
This will be backpatched in due course after some testing.
Discussion: https://postgr.es/m/20221002164931.d57hlutrcz4d2zi7@awork3.anarazel.de
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/9b4eafcaf41d1192a34b574c21262b755aa455ee
Modified Files
--------------
src/Makefile.global.in | 2 +-
src/test/perl/PostgreSQL/Test/Cluster.pm | 65 +++++++++++++++++++++++++++++---
2 files changed, 61 insertions(+), 6 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent port collisions between concurrent TAP tests
@ 2022-11-22 15:56 Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-22 15:56 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent port collisions between concurrent TAP tests
Currently there is a race condition where if concurrent TAP tests both
test that they can open a port they will assume that it is free and use
it, causing one of them to fail. To prevent this we record a reservation
using an exclusive lock, and any TAP test that discovers a reservation
checks to see if the reserving process is still alive, and looks for
another free port if it is.
Ports are reserved in a directory set by the environment setting
PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
build directory as set by Makefile.global, or its own
tmp_check directory.
The prove_check recipe in Makefile.global.in is extended to export
top_builddir to the TAP tests. This was already exported by the
prove_installcheck recipes.
Per complaint from Andres Freund
Backpatched from 9b4eafcaf4 to all live branches
Discussion: https://postgr.es/m/20221002164931.d57hlutrcz4d2zi7@awork3.anarazel.de
Branch
------
REL_15_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/153e21567750d37b031123d3bbf14f9e2434f4b2
Modified Files
--------------
src/Makefile.global.in | 1 +
src/test/perl/PostgreSQL/Test/Cluster.pm | 64 +++++++++++++++++++++++++++++---
2 files changed, 60 insertions(+), 5 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent port collisions between concurrent TAP tests
@ 2022-11-22 15:56 Andrew Dunstan <andrew@dunslane.net>
2022-11-26 11:15 ` Re: pgsql: Prevent port collisions between concurrent TAP tests Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-22 15:56 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent port collisions between concurrent TAP tests
Currently there is a race condition where if concurrent TAP tests both
test that they can open a port they will assume that it is free and use
it, causing one of them to fail. To prevent this we record a reservation
using an exclusive lock, and any TAP test that discovers a reservation
checks to see if the reserving process is still alive, and looks for
another free port if it is.
Ports are reserved in a directory set by the environment setting
PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
build directory as set by Makefile.global, or its own
tmp_check directory.
The prove_check recipe in Makefile.global.in is extended to export
top_builddir to the TAP tests. This was already exported by the
prove_installcheck recipes.
Per complaint from Andres Freund
Backpatched from 9b4eafcaf4 to all live branches
Discussion: https://postgr.es/m/20221002164931.d57hlutrcz4d2zi7@awork3.anarazel.de
Branch
------
REL_14_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/870d6218e6d73dca8ec37b4d7f5076b74eeaeffe
Modified Files
--------------
src/Makefile.global.in | 1 +
src/test/perl/PostgresNode.pm | 64 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 60 insertions(+), 5 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Prevent port collisions between concurrent TAP tests
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
@ 2022-11-26 11:15 ` Michael Paquier <michael@paquier.xyz>
2022-11-26 12:40 ` Re: pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 9+ messages in thread
From: Michael Paquier @ 2022-11-26 11:15 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: pgsql-committers@lists.postgresql.org
Hi Andrew,
On Tue, Nov 22, 2022 at 03:56:43PM +0000, Andrew Dunstan wrote:
> Prevent port collisions between concurrent TAP tests
>
> Currently there is a race condition where if concurrent TAP tests both
> test that they can open a port they will assume that it is free and use
> it, causing one of them to fail. To prevent this we record a reservation
> using an exclusive lock, and any TAP test that discovers a reservation
> checks to see if the reserving process is still alive, and looks for
> another free port if it is.
>
> Ports are reserved in a directory set by the environment setting
> PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
> build directory as set by Makefile.global, or its own
> tmp_check directory.
Wouldn't it be better to update .gitgnore at the root of the source
folder to discard portlock/ so as we will never include by accident in
a commit?
Thanks,
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Prevent port collisions between concurrent TAP tests
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-26 11:15 ` Re: pgsql: Prevent port collisions between concurrent TAP tests Michael Paquier <michael@paquier.xyz>
@ 2022-11-26 12:40 ` Andrew Dunstan <andrew@dunslane.net>
2022-11-27 05:11 ` Re: pgsql: Prevent port collisions between concurrent TAP tests Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-26 12:40 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers@lists.postgresql.org
On 2022-11-26 Sa 06:15, Michael Paquier wrote:
> Hi Andrew,
>
> On Tue, Nov 22, 2022 at 03:56:43PM +0000, Andrew Dunstan wrote:
>> Prevent port collisions between concurrent TAP tests
>>
>> Currently there is a race condition where if concurrent TAP tests both
>> test that they can open a port they will assume that it is free and use
>> it, causing one of them to fail. To prevent this we record a reservation
>> using an exclusive lock, and any TAP test that discovers a reservation
>> checks to see if the reserving process is still alive, and looks for
>> another free port if it is.
>>
>> Ports are reserved in a directory set by the environment setting
>> PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
>> build directory as set by Makefile.global, or its own
>> tmp_check directory.
> Wouldn't it be better to update .gitgnore at the root of the source
> folder to discard portlock/ so as we will never include by accident in
> a commit?
>
Yes, good point. Will do.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Prevent port collisions between concurrent TAP tests
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-26 11:15 ` Re: pgsql: Prevent port collisions between concurrent TAP tests Michael Paquier <michael@paquier.xyz>
2022-11-26 12:40 ` Re: pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
@ 2022-11-27 05:11 ` Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 9+ messages in thread
From: Michael Paquier @ 2022-11-27 05:11 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: pgsql-committers@lists.postgresql.org
On Sat, Nov 26, 2022 at 07:40:52AM -0500, Andrew Dunstan wrote:
> Yes, good point. Will do.
Thanks.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent port collisions between concurrent TAP tests
@ 2022-11-22 15:56 Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-22 15:56 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent port collisions between concurrent TAP tests
Currently there is a race condition where if concurrent TAP tests both
test that they can open a port they will assume that it is free and use
it, causing one of them to fail. To prevent this we record a reservation
using an exclusive lock, and any TAP test that discovers a reservation
checks to see if the reserving process is still alive, and looks for
another free port if it is.
Ports are reserved in a directory set by the environment setting
PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
build directory as set by Makefile.global, or its own
tmp_check directory.
The prove_check recipe in Makefile.global.in is extended to export
top_builddir to the TAP tests. This was already exported by the
prove_installcheck recipes.
Per complaint from Andres Freund
Backpatched from 9b4eafcaf4 to all live branches
Discussion: https://postgr.es/m/20221002164931.d57hlutrcz4d2zi7@awork3.anarazel.de
Branch
------
REL_13_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/4b9013d64f231ddbadd2b06cf75499c8e8cd4cc1
Modified Files
--------------
src/Makefile.global.in | 1 +
src/test/perl/PostgresNode.pm | 64 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 60 insertions(+), 5 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent port collisions between concurrent TAP tests
@ 2022-11-22 15:56 Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-22 15:56 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent port collisions between concurrent TAP tests
Currently there is a race condition where if concurrent TAP tests both
test that they can open a port they will assume that it is free and use
it, causing one of them to fail. To prevent this we record a reservation
using an exclusive lock, and any TAP test that discovers a reservation
checks to see if the reserving process is still alive, and looks for
another free port if it is.
Ports are reserved in a directory set by the environment setting
PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
build directory as set by Makefile.global, or its own
tmp_check directory.
The prove_check recipe in Makefile.global.in is extended to export
top_builddir to the TAP tests. This was already exported by the
prove_installcheck recipes.
Per complaint from Andres Freund
Backpatched from 9b4eafcaf4 to all live branches
Discussion: https://postgr.es/m/20221002164931.d57hlutrcz4d2zi7@awork3.anarazel.de
Branch
------
REL_12_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/baa78ff5404c9d870309207edbb168e1ec7c5af7
Modified Files
--------------
src/Makefile.global.in | 1 +
src/test/perl/PostgresNode.pm | 64 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 60 insertions(+), 5 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Prevent port collisions between concurrent TAP tests
@ 2022-11-22 15:56 Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Dunstan @ 2022-11-22 15:56 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Prevent port collisions between concurrent TAP tests
Currently there is a race condition where if concurrent TAP tests both
test that they can open a port they will assume that it is free and use
it, causing one of them to fail. To prevent this we record a reservation
using an exclusive lock, and any TAP test that discovers a reservation
checks to see if the reserving process is still alive, and looks for
another free port if it is.
Ports are reserved in a directory set by the environment setting
PG_TEST_PORT_DIR, or if that doesn't exist a subdirectory of the top
build directory as set by Makefile.global, or its own
tmp_check directory.
The prove_check recipe in Makefile.global.in is extended to export
top_builddir to the TAP tests. This was already exported by the
prove_installcheck recipes.
Per complaint from Andres Freund
Backpatched from 9b4eafcaf4 to all live branches
Discussion: https://postgr.es/m/20221002164931.d57hlutrcz4d2zi7@awork3.anarazel.de
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/46def5267cf938e6031d82aece2d50b3f6a17de4
Modified Files
--------------
src/Makefile.global.in | 1 +
src/test/perl/PostgresNode.pm | 64 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 60 insertions(+), 5 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2022-11-27 05:11 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-11-20 15:07 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-26 11:15 ` Michael Paquier <michael@paquier.xyz>
2022-11-26 12:40 ` Andrew Dunstan <andrew@dunslane.net>
2022-11-27 05:11 ` Michael Paquier <michael@paquier.xyz>
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
2022-11-22 15:56 pgsql: Prevent port collisions between concurrent TAP tests Andrew Dunstan <andrew@dunslane.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox