pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
8+ messages / 4 participants
[nested] [flat]
* BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
@ 2021-03-04 07:43 PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: PG Bug reporting form @ 2021-03-04 07:43 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: r.zharkov@postgrespro.ru
The following bug has been logged on the website:
Bug reference: 16914
Logged by: Roman Zharkov
Email address: r.zharkov@postgrespro.ru
PostgreSQL version: Unsupported/Unknown
Operating system: Centos 7
Description:
Hello!
I found a little problem with the test on the master branch:
$ make -s -C src/test/modules/worker_spi check USE_MODULE_DB=TRUE
============== creating temporary instance ==============
============== initializing database system ==============
============== starting postmaster ==============
running on port 58080 with PID 2863
============== creating database "contrib_regression_worker_spi"
==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test worker_spi ... FAILED 131652 ms
============== shutting down postmaster
$make -s -C src/test/modules/worker_spi check USE_MODULE_DB=
============== removing existing temp instance ==============
============== creating temporary instance ==============
============== initializing database system ==============
============== starting postmaster ==============
running on port 58080 with PID 1903
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test worker_spi ... ok 246 ms
============== shutting down postmaster ==============
============== removing temporary instance ==============
=====================
All 1 tests passed.
=====================
Something like this fixes the problem:
diff --git a/src/test/modules/worker_spi/.gitignore
b/src/test/modules/worker_spi/.gitignore
index
5dcb3ff9723501c3fe639bee1c1435e47a580a6f..8050cb6716c52fbade5ac5e80eabf3dd134737b8
100644
inline patch 1: src/test/modules/worker_spi/.gitignore, src/test/modules/worker_spi/Makefile (+1/-0 lines) download inline-1.patch
--- a/src/test/modules/worker_spi/.gitignore
+++ b/src/test/modules/worker_spi/.gitignore
@@ -2,3 +2,4 @@
/log/
/results/
/tmp_check/
+dynamic_temp.conf
diff --git a/src/test/modules/worker_spi/Makefile
b/src/test/modules/worker_spi/Makefile
index
cbf9b2e37fd6da99b9250481f903eec1940e54ae..1865e578533eaaea8698f2396db01b4edb8a91ba
100644
inline patch 2: src/test/modules/worker_spi/Makefile (+0/-1 lines) download inline-2.patch
--- a/src/test/modules/worker_spi/Makefile
+++ b/src/test/modules/worker_spi/Makefile
@@ -9,7 +9,15 @@ PGFILEDESC = "worker_spi - background worker example"
REGRESS = worker_spi
# enable our module in shared_preload_libraries for dynamic bgworkers
-REGRESS_OPTS = --temp-config
$(top_srcdir)/src/test/modules/worker_spi/dynamic.conf
+REGRESS_OPTS = --temp-config
$(top_srcdir)/src/test/modules/worker_spi/dynamic_temp.conf
+
+# Set the actual database name in the temp config.
+set-db-name:
+ cp $(top_srcdir)/src/test/modules/worker_spi/dynamic.conf
$(top_srcdir)/src/test/modules/worker_spi/dynamic_temp.conf
+ echo "worker_spi.database = $(CONTRIB_TESTDB)" >>
$(top_srcdir)/src/test/modules/worker_spi/dynamic_temp.conf
+.PHONY: set-db-name
+
+check: set-db-name
# Disable installcheck to ensure we cover dynamic bgworkers.
NO_INSTALLCHECK = 1
regards,
Roman Zharkov
Patches: download all (2 files, .tar.gz)
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
@ 2021-03-04 12:42 ` Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2021-03-04 12:42 UTC (permalink / raw)
To: r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On Thu, Mar 04, 2021 at 07:43:11AM +0000, PG Bug reporting form wrote:
> I found a little problem with the test on the master branch:
>
> $ make -s -C src/test/modules/worker_spi check USE_MODULE_DB=TRUE
> ============== running regression test queries ==============
> test worker_spi ... FAILED 131652 ms
> ============== shutting down postmaster
Fun.
> Something like this fixes the problem:
One problem with this solution is that this is not portable. Instead
of generating dynamically the configuration file, here is funkier
solution as we know that worker_spi would restart if the database it
should connect to does not exist: let's use a completely different
database name, not expected by any other modules. Then, this database
gets created in worker_spi.sql, with a reconnection to it.
--
Michael
Attachments:
worker-spi-test.patch (text/x-diff, 1.5K) download
diff --git a/src/test/modules/worker_spi/dynamic.conf b/src/test/modules/worker_spi/dynamic.conf
index bfe015f664..2f480be1da 100644
--- a/src/test/modules/worker_spi/dynamic.conf
+++ b/src/test/modules/worker_spi/dynamic.conf
@@ -1,2 +1,2 @@
shared_preload_libraries = worker_spi
-worker_spi.database = contrib_regression
+worker_spi.database = regression_worker_spi
diff --git a/src/test/modules/worker_spi/expected/worker_spi.out b/src/test/modules/worker_spi/expected/worker_spi.out
index dc0a79bf75..c54d5e6cf8 100644
--- a/src/test/modules/worker_spi/expected/worker_spi.out
+++ b/src/test/modules/worker_spi/expected/worker_spi.out
@@ -1,3 +1,7 @@
+-- Create the database specific to this test, and reconnect to it.
+-- This matches with the configuration of dynamic.conf.
+CREATE DATABASE regression_worker_spi;
+\c regression_worker_spi
CREATE EXTENSION worker_spi;
SELECT worker_spi_launch(4) IS NOT NULL;
?column?
diff --git a/src/test/modules/worker_spi/sql/worker_spi.sql b/src/test/modules/worker_spi/sql/worker_spi.sql
index 4683523b29..61a9781bc1 100644
--- a/src/test/modules/worker_spi/sql/worker_spi.sql
+++ b/src/test/modules/worker_spi/sql/worker_spi.sql
@@ -1,3 +1,8 @@
+-- Create the database specific to this test, and reconnect to it.
+-- This matches with the configuration of dynamic.conf.
+CREATE DATABASE regression_worker_spi;
+\c regression_worker_spi
+
CREATE EXTENSION worker_spi;
SELECT worker_spi_launch(4) IS NOT NULL;
-- wait until the worker completes its initialization
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
@ 2021-03-04 21:13 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-03-04 22:14 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Alvaro Herrera @ 2021-03-04 21:13 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On 2021-Mar-04, Michael Paquier wrote:
> > test worker_spi ... FAILED 131652 ms
> > ============== shutting down postmaster
>
> Fun.
Eek.
> > Something like this fixes the problem:
>
> One problem with this solution is that this is not portable. Instead
> of generating dynamically the configuration file, here is funkier
> solution as we know that worker_spi would restart if the database it
> should connect to does not exist: let's use a completely different
> database name, not expected by any other modules. Then, this database
> gets created in worker_spi.sql, with a reconnection to it.
It's not that I *like* this solution, but let's say that I dislike it
less than the other one. If there are no other ideas, I +1 this one.
--
Álvaro Herrera 39°49'30"S 73°17'W
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2021-03-04 22:14 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 05:30 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2021-03-04 22:14 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Michael Paquier <michael@paquier.xyz>; r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2021-Mar-04, Michael Paquier wrote:
>> One problem with this solution is that this is not portable. Instead
>> of generating dynamically the configuration file, here is funkier
>> solution as we know that worker_spi would restart if the database it
>> should connect to does not exist: let's use a completely different
>> database name, not expected by any other modules. Then, this database
>> gets created in worker_spi.sql, with a reconnection to it.
> It's not that I *like* this solution, but let's say that I dislike it
> less than the other one. If there are no other ideas, I +1 this one.
The real problem here is that worker_spi illustrates a fundamentally
useless way to define which database a worker ought to connect to.
It's basically impossible for it to use anything except the hard-wired
default for the worker_spi.database GUC. You could wish that we could
fix it like
CREATE EXTENSION worker_spi;
+SELECT set_config('worker_spi.database', current_database(), false);
SELECT worker_spi_launch(4) IS NOT NULL;
-- wait until the worker completes its initialization
but that doesn't work because the session-local setting of the GUC won't
be seen over in the worker process.
Since the alleged point of this module is to be a prototype for
useful background workers, I think we should think of another way.
Maybe we need to work a bit harder on passing values from
worker_spi_launch to the worker, so that the DB name could be a
parameter to worker_spi_launch.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-03-04 22:14 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
@ 2021-03-05 05:30 ` Michael Paquier <michael@paquier.xyz>
2021-03-05 05:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2021-03-05 05:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On Thu, Mar 04, 2021 at 05:14:57PM -0500, Tom Lane wrote:
> CREATE EXTENSION worker_spi;
> +SELECT set_config('worker_spi.database', current_database(), false);
> SELECT worker_spi_launch(4) IS NOT NULL;
> -- wait until the worker completes its initialization
>
> but that doesn't work because the session-local setting of the GUC won't
> be seen over in the worker process.
>
> Since the alleged point of this module is to be a prototype for
> useful background workers, I think we should think of another way.
> Maybe we need to work a bit harder on passing values from
> worker_spi_launch to the worker, so that the DB name could be a
> parameter to worker_spi_launch.
Yeah, or this is converted to a TAP test where we don't care about
the database used, but that would consume more cycles without more
coverage done. Don't we need to worry about the memory context where
bgw_main_arg is pushed into for dynamic workers?
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-03-04 22:14 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 05:30 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
@ 2021-03-05 05:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 06:19 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2021-03-05 05:42 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
Michael Paquier <michael@paquier.xyz> writes:
> On Thu, Mar 04, 2021 at 05:14:57PM -0500, Tom Lane wrote:
>> Since the alleged point of this module is to be a prototype for
>> useful background workers, I think we should think of another way.
>> Maybe we need to work a bit harder on passing values from
>> worker_spi_launch to the worker, so that the DB name could be a
>> parameter to worker_spi_launch.
> Yeah, or this is converted to a TAP test where we don't care about
> the database used, but that would consume more cycles without more
> coverage done. Don't we need to worry about the memory context where
> bgw_main_arg is pushed into for dynamic workers?
Yeah, the fact that bgw_main_arg is declared as Datum is really kind
of a lie, because the only thing that actually works there is a pass
by value datatype (not that our docs tell you so).
We could stuff the database name into bgw_extra perhaps, but that
seems restrictive as well. I wonder if there's a reasonable way to
pass over a struct of caller-defined size.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-03-04 22:14 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 05:30 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-05 05:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
@ 2021-03-05 06:19 ` Michael Paquier <michael@paquier.xyz>
2021-03-05 13:43 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Paquier @ 2021-03-05 06:19 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On Fri, Mar 05, 2021 at 12:42:43AM -0500, Tom Lane wrote:
> Yeah, the fact that bgw_main_arg is declared as Datum is really kind
> of a lie, because the only thing that actually works there is a pass
> by value datatype (not that our docs tell you so).
Exactly.
> We could stuff the database name into bgw_extra perhaps, but that
> seems restrictive as well. I wonder if there's a reasonable way to
> pass over a struct of caller-defined size.
Or we could just rely on a custom memory context that dynamic workers
could use to store their data? It could be simpler IMO to just merge
together bgw_main_arg and bgw_extra, then pass down the whole as an
argument of the main routine of the bgworker, say with a buffer of 256
bytes for example. I am not sure that workers really need that much
for themselves at startup out of GUCs, but I may be wrong :)
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set.
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-03-04 22:14 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 05:30 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
2021-03-05 05:42 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 06:19 ` Re: BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. Michael Paquier <michael@paquier.xyz>
@ 2021-03-05 13:43 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 8+ messages in thread
From: Alvaro Herrera @ 2021-03-05 13:43 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; r.zharkov@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On 2021-Mar-05, Michael Paquier wrote:
> Or we could just rely on a custom memory context that dynamic workers
> could use to store their data? It could be simpler IMO to just merge
> together bgw_main_arg and bgw_extra, then pass down the whole as an
> argument of the main routine of the bgworker, say with a buffer of 256
> bytes for example.
I think the only way to make that work portably is by using shared memory.
--
Álvaro Herrera Valdivia, Chile
"La rebeldía es la virtud original del hombre" (Arthur Schopenhauer)
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2021-03-05 13:43 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-03-04 07:43 BUG #16914: Regression test of the worker_spi fails if USE_MODULE_DB environment variable is set. PG Bug reporting form <noreply@postgresql.org>
2021-03-04 12:42 ` Michael Paquier <michael@paquier.xyz>
2021-03-04 21:13 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-03-04 22:14 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 05:30 ` Michael Paquier <michael@paquier.xyz>
2021-03-05 05:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-03-05 06:19 ` Michael Paquier <michael@paquier.xyz>
2021-03-05 13:43 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox