From: Benoit Lobréau <benoit.lobreau@dalibo.com>
To: Robert Haas <robertmhaas@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Melanie Plageman <melanieplageman@gmail.com>
Subject: Re: Logging parallel worker draught
Date: Tue, 2 May 2023 12:44:58 +0200
Message-ID: <3ed1b53c-15c0-5638-a059-69bd6a2fb11d@dalibo.com> (raw)
In-Reply-To: <CA+Tgmob91fKoE1QoTqGomBTUZT9JcXwFYrPEmm3arapak-WTsA@mail.gmail.com>
References: <b4220d15-2e21-0e98-921b-b9892543cc93@dalibo.com>
<CAA4eK1+2_te1KYFkSoagdK4Okj0Ob_17KmMGf=qdZ0H4Muzjcw@mail.gmail.com>
<CA+Tgmob91fKoE1QoTqGomBTUZT9JcXwFYrPEmm3arapak-WTsA@mail.gmail.com>
On 5/1/23 18:33, Robert Haas wrote:
> Why not? It seems like something some people might want to log and
> others not. Running the whole server at DEBUG1 to get this information
> doesn't seem like a suitable answer.
Since the statement is also logged, it could spam the log with huge
queries, which might also be a reason to stop logging this kind of
problems until the issue is fixed.
I attached the patch without the guc anyway just in case.
> What I was wondering was whether we would be better off putting this
> into the statistics collector, vs. doing it via logging. Both
> approaches seem to have pros and cons.
We tried to explore different options with my collegues in another
thread [1]. I feel like both solutions are worthwhile, and would be
helpful. I plan to take a look at the pg_stat_statement patch [2] next.
Since it's my first patch, I elected to choose the easiest solution to
implement first. I also proposed this because I think logging can help
pinpoint a lot of problems at a minimal cost, since it is easy to setup
and exploit for everyone, everywhere
[1]
https://www.postgresql.org/message-id/d657df20-c4bf-63f6-e74c-cb85a81d0383@dalibo.com
[2]
https://www.postgresql.org/message-id/flat/6acbe570-068e-bd8e-95d5-00c737b865e8%40gmail.com
--
Benoit Lobréau
Consultant
http://dalibo.com
From fc71ef40f33f94b0506a092fb5b3dcde6de6d60a Mon Sep 17 00:00:00 2001
From: benoit <benoit.lobreau@dalibo.com>
Date: Tue, 2 May 2023 10:08:00 +0200
Subject: [PATCH] Add logging for exceeded parallel worker slot limits
Procude a log message when a backend attempts to spawn a parallel worker
but fails due to insufficient worker slots. The shortage can stem from
max_worker_processes, max_parallel_worker, or
max_parallel_maintenance_workers. The log message can help database
administrators and developers diagnose performance issues related to
parallelism and optimize the configuration of the system accordingly.
---
src/backend/access/transam/parallel.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.cindex b26f2a64fb..c60d1bd739 100644--- a/src/backend/access/transam/parallel.c+++ b/src/backend/access/transam/parallel.c@@ -630,6 +630,11 @@ LaunchParallelWorkers(ParallelContext *pcxt)
pcxt->nknown_attached_workers = 0;
}
+ if (pcxt->nworkers_launched < pcxt->nworkers_to_launch)+ ereport(LOG,+ (errmsg("Parallel Worker draught during statement execution: workers spawned %d, requested %d",+ pcxt->nworkers_launched, pcxt->nworkers_to_launch)));+
/* Restore previous memory context. */
MemoryContextSwitchTo(oldcontext);
}
--
2.39.2
Attachments:
[text/plain] 0001-Add-logging-for-exceeded-parallel-worker-slot-limits.patch_v2_no_guc (1.3K, ../3ed1b53c-15c0-5638-a059-69bd6a2fb11d@dalibo.com/2-0001-Add-logging-for-exceeded-parallel-worker-slot-limits.patch_v2_no_guc)
download | inline diff:
From fc71ef40f33f94b0506a092fb5b3dcde6de6d60a Mon Sep 17 00:00:00 2001
From: benoit <benoit.lobreau@dalibo.com>
Date: Tue, 2 May 2023 10:08:00 +0200
Subject: [PATCH] Add logging for exceeded parallel worker slot limits
Procude a log message when a backend attempts to spawn a parallel worker
but fails due to insufficient worker slots. The shortage can stem from
max_worker_processes, max_parallel_worker, or
max_parallel_maintenance_workers. The log message can help database
administrators and developers diagnose performance issues related to
parallelism and optimize the configuration of the system accordingly.
---
src/backend/access/transam/parallel.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.cindex b26f2a64fb..c60d1bd739 100644--- a/src/backend/access/transam/parallel.c+++ b/src/backend/access/transam/parallel.c@@ -630,6 +630,11 @@ LaunchParallelWorkers(ParallelContext *pcxt)
pcxt->nknown_attached_workers = 0;
}
+ if (pcxt->nworkers_launched < pcxt->nworkers_to_launch)+ ereport(LOG,+ (errmsg("Parallel Worker draught during statement execution: workers spawned %d, requested %d",+ pcxt->nworkers_launched, pcxt->nworkers_to_launch)));+
/* Restore previous memory context. */
MemoryContextSwitchTo(oldcontext);
}
--
2.39.2
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: benoit.lobreau@dalibo.com, robertmhaas@gmail.com, amit.kapila16@gmail.com, melanieplageman@gmail.com
Subject: Re: Logging parallel worker draught
In-Reply-To: <3ed1b53c-15c0-5638-a059-69bd6a2fb11d@dalibo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox