pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Harden new test case against force_parallel_mode = regress.
9+ messages / 3 participants
[nested] [flat]

* pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-02 22:47  Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 9+ messages in thread

From: Tom Lane @ 2023-03-02 22:47 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Harden new test case against force_parallel_mode = regress.

Per buildfarm: worker processes can't see a role created in
the current transaction.

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/98a88bc2bcd60e41ca70e2f1e13eee827e23eefb

Modified Files
--------------
src/test/regress/expected/psql.out | 3 ++-
src/test/regress/sql/psql.sql      | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 16:16  Robert Haas <robertmhaas@gmail.com>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 2 replies; 9+ messages in thread

From: Robert Haas @ 2023-03-03 16:16 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

On Thu, Mar 2, 2023 at 5:47 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Harden new test case against force_parallel_mode = regress.
>
> Per buildfarm: worker processes can't see a role created in
> the current transaction.

Now why would that happen? Surely the snapshot for each command is
passed down from leader to worker, and the worker is not free to
invent a snapshot from nothing.

-- 
Robert Haas
EDB: http://www.enterprisedb.com





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 16:37  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Robert Haas <robertmhaas@gmail.com>
  1 sibling, 2 replies; 9+ messages in thread

From: Tom Lane @ 2023-03-03 16:37 UTC (permalink / raw)
  To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Mar 2, 2023 at 5:47 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Per buildfarm: worker processes can't see a role created in
>> the current transaction.

> Now why would that happen? Surely the snapshot for each command is
> passed down from leader to worker, and the worker is not free to
> invent a snapshot from nothing.

The workers were failing at startup, eg (from [1]):

+ERROR:  role "regress_psql_user" does not exist
+CONTEXT:  while setting parameter "session_authorization" to "regress_psql_user"

Maybe this says that worker startup needs to install the snapshot before
doing any catalog accesses?  Anyway, I'd be happy to revert this test
hack if you care to make the case work.

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 16:38  Matthias van de Meent <boekewurm+postgres@gmail.com>
  parent: Robert Haas <robertmhaas@gmail.com>
  1 sibling, 0 replies; 9+ messages in thread

From: Matthias van de Meent @ 2023-03-03 16:38 UTC (permalink / raw)
  To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

On Fri, 3 Mar 2023 at 17:16, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, Mar 2, 2023 at 5:47 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Harden new test case against force_parallel_mode = regress.
> >
> > Per buildfarm: worker processes can't see a role created in
> > the current transaction.
>
> Now why would that happen? Surely the snapshot for each command is
> passed down from leader to worker, and the worker is not free to
> invent a snapshot from nothing.

Probably because we nitialize which user and database to use in the
backend before we load the parent process' snapshot:

in ParallelWorkerMain (parallel.c, as of HEAD @ b6a0d469):

  /* Restore database connection. */
  BackgroundWorkerInitializeConnectionByOid(fps->database_id,
                                  fps->authenticated_user_id,
                                  0);
[...]

  /* Crank up a transaction state appropriate to a parallel worker. */
  tstatespace = shm_toc_lookup(toc, PARALLEL_KEY_TRANSACTION_STATE, false);
  StartParallelWorkerTransaction(tstatespace);

  /* Restore combo CID state. */
  combocidspace = shm_toc_lookup(toc, PARALLEL_KEY_COMBO_CID, false);
  RestoreComboCIDState(combocidspace);

-Matthias





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 16:47  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-03-03 16:47 UTC (permalink / raw)
  To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

I wrote:
> The workers were failing at startup, eg (from [1]):

argh, forgot to add the link:

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hippopotamus&dt=2023-03-02%2022%3A31%3A1...

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 17:40  Robert Haas <robertmhaas@gmail.com>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 1 reply; 9+ messages in thread

From: Robert Haas @ 2023-03-03 17:40 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

On Fri, Mar 3, 2023 at 11:37 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> The workers were failing at startup, eg (from [1]):
>
> +ERROR:  role "regress_psql_user" does not exist
> +CONTEXT:  while setting parameter "session_authorization" to "regress_psql_user"
>
> Maybe this says that worker startup needs to install the snapshot before
> doing any catalog accesses?  Anyway, I'd be happy to revert this test
> hack if you care to make the case work.

Oh, that's interesting (and sad). A parallel worker has a "startup
transaction" that is used to restore library and GUC state, and then
after that transaction commits, it starts up a new transaction that
uses the same snapshot and settings as the transaction in the parallel
leader. So the problem here is that the startup transaction can't see
the uncommitted work of some unrelated (as far as it knows)
transaction, and that prevents restoring the session_authorization
GUC.

That startup transaction has broken stuff before, and it would be nice
to get rid of it. Unfortunately, I don't remember right now why we
need it in the first place. I'm fairly sure that if you load the
library and GUC state without any transaction, that doesn't work,
because a bunch of important processing gets skipped. And I think if
you try to do those things in the "real" transaction that fails for
some reason too, maybe that there's no guarantee that all the relevant
GUCs can be changed at that point, but I'm fuzzy on the details at the
moment.

So I don't know how to fix this right now, but thanks for the details.

-- 
Robert Haas
EDB: http://www.enterprisedb.com





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 17:46  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Robert Haas <robertmhaas@gmail.com>
  0 siblings, 1 reply; 9+ messages in thread

From: Tom Lane @ 2023-03-03 17:46 UTC (permalink / raw)
  To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

Robert Haas <robertmhaas@gmail.com> writes:
> On Fri, Mar 3, 2023 at 11:37 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> +ERROR:  role "regress_psql_user" does not exist
>> +CONTEXT:  while setting parameter "session_authorization" to "regress_psql_user"

> Oh, that's interesting (and sad). A parallel worker has a "startup
> transaction" that is used to restore library and GUC state, and then
> after that transaction commits, it starts up a new transaction that
> uses the same snapshot and settings as the transaction in the parallel
> leader. So the problem here is that the startup transaction can't see
> the uncommitted work of some unrelated (as far as it knows)
> transaction, and that prevents restoring the session_authorization
> GUC.

Got it.

> That startup transaction has broken stuff before, and it would be nice
> to get rid of it. Unfortunately, I don't remember right now why we
> need it in the first place. I'm fairly sure that if you load the
> library and GUC state without any transaction, that doesn't work,
> because a bunch of important processing gets skipped. And I think if
> you try to do those things in the "real" transaction that fails for
> some reason too, maybe that there's no guarantee that all the relevant
> GUCs can be changed at that point, but I'm fuzzy on the details at the
> moment.

Couldn't we install the leader's snapshot into both transactions?

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 17:56  Robert Haas <robertmhaas@gmail.com>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 9+ messages in thread

From: Robert Haas @ 2023-03-03 17:56 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

On Fri, Mar 3, 2023 at 12:46 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Couldn't we install the leader's snapshot into both transactions?

Yeah, maybe that would Just Work. Not sure.

-- 
Robert Haas
EDB: http://www.enterprisedb.com





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Harden new test case against force_parallel_mode = regress.
@ 2023-03-03 18:03  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Robert Haas <robertmhaas@gmail.com>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-03-03 18:03 UTC (permalink / raw)
  To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>

Robert Haas <robertmhaas@gmail.com> writes:
> On Fri, Mar 3, 2023 at 12:46 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Couldn't we install the leader's snapshot into both transactions?

> Yeah, maybe that would Just Work. Not sure.

Well, IIUC the worker is currently getting a brand new snapshot
for its startup transaction, which is exactly what you said upthread
it should never do.  Seems like that could have more failure modes
than just this one.

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread


end of thread, other threads:[~2023-03-03 18:03 UTC | newest]

Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-03-02 22:47 pgsql: Harden new test case against force_parallel_mode = regress. Tom Lane <tgl@sss.pgh.pa.us>
2023-03-03 16:16 ` Robert Haas <robertmhaas@gmail.com>
2023-03-03 16:37   ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-03 16:47     ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-03 17:40     ` Robert Haas <robertmhaas@gmail.com>
2023-03-03 17:46       ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-03 17:56         ` Robert Haas <robertmhaas@gmail.com>
2023-03-03 18:03           ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-03 16:38   ` Matthias van de Meent <boekewurm+postgres@gmail.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox