From: Benoit Lobréau <benoit.lobreau@dalibo.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Imseih (AWS), Sami <simseih@amazon.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Melanie Plageman <melanieplageman@gmail.com>
Subject: Re: Logging parallel worker draught
Date: Tue, 27 Feb 2024 10:55:27 +0100
Message-ID: <0b7ef57e-bf30-440b-9478-d8f587f2637c@dalibo.com> (raw)
In-Reply-To: <0d08b978-10cb-4512-bcef-ca1f192049ab@enterprisedb.com>
References: <10D5C31A-A303-4440-BCA2-921440A9E1CC@amazon.com>
<202310111011.lyo66rk7k7pu@alvherre.pgsql>
<9E9E69BD-BABB-49BB-8B69-61939179F20D@amazon.com>
<11e34b80-b0a6-e2e4-1606-1f5077379a34@dalibo.com>
<D04977E3-9F54-452C-A4C4-CDA67F392BD1@amazon.com>
<0d08b978-10cb-4512-bcef-ca1f192049ab@enterprisedb.com>
On 2/25/24 20:13, Tomas Vondra wrote:
> 1) name of the GUC
...
> 2) logging just the failures provides an incomplete view
> log_parallel_workers = {none | failures | all}>
> where "failures" only logs when at least one worker fails to start, and
> "all" logs everything.
>
> AFAIK Sami made the same observation/argument in his last message [1].
I like the name and different levels you propose. I was initially
thinking that the overall picture would be better summarized (an easier
to implement) with pg_stat_statements. But with the granularity you
propose, the choice is in the hands of the DBA, which is great and
provides more options when we don't have full control of the installation.
> 3) node-level or query-level?
...
> There's no value in having information about every individual temp file,
> because we don't even know which node produced it. It'd be perfectly
> sufficient to log some summary statistics (number of files, total space,
> ...), either for the query as a whole, or perhaps per node (because
> that's what matters for work_mem). So I don't think we should mimic this
> just because log_temp_files does this.
I must admit that, given my (poor) technical level, I went for the
easiest way.
If we go for the three levels you proposed, "node-level" makes even less
sense (and has great "potential" for spam).
I can see one downside to the "query-level" approach: it might be more
difficult to to give information in cases where the query doesn't end
normally. It's sometimes useful to have hints about what was going wrong
before a query was cancelled or crashed, which log_temp_files kinda does.
> I don't know how difficult would it be to track/collect this information
> for the whole query.
I am a worried this will be a little too much for me, given the time and
the knowledge gap I have (both in C and PostgreSQL internals). I'll try
to look anyway.
--
Benoit Lobréau
Consultant
http://dalibo.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: benoit.lobreau@dalibo.com, tomas.vondra@enterprisedb.com, simseih@amazon.com, alvherre@alvh.no-ip.org, robertmhaas@gmail.com, amit.kapila16@gmail.com, melanieplageman@gmail.com
Subject: Re: Logging parallel worker draught
In-Reply-To: <0b7ef57e-bf30-440b-9478-d8f587f2637c@dalibo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox