Message-ID: <0b7ef57e-bf30-440b-9478-d8f587f2637c@dalibo.com>
Date: Tue, 27 Feb 2024 10:55:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Logging parallel worker draught
Content-Language: fr
To: Tomas Vondra <tomas.vondra@enterprisedb.com>,
 "Imseih (AWS), Sami" <simseih@amazon.com>,
 Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Robert Haas <robertmhaas@gmail.com>, Amit Kapila
 <amit.kapila16@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>,
 Melanie Plageman <melanieplageman@gmail.com>
References: <10D5C31A-A303-4440-BCA2-921440A9E1CC@amazon.com>
 <202310111011.lyo66rk7k7pu@alvherre.pgsql>
 <9E9E69BD-BABB-49BB-8B69-61939179F20D@amazon.com>
 <11e34b80-b0a6-e2e4-1606-1f5077379a34@dalibo.com>
 <D04977E3-9F54-452C-A4C4-CDA67F392BD1@amazon.com>
 <0d08b978-10cb-4512-bcef-ca1f192049ab@enterprisedb.com>
From: =?UTF-8?Q?Benoit_Lobr=C3=A9au?= <benoit.lobreau@dalibo.com>
Autocrypt: addr=benoit.lobreau@dalibo.com; keydata=
 xsBNBF3FR7IBCADDvtR0p09SI9OvnbWw+VEvO5IXefBWFWefxrPd0H3Sf3gsBLiKbRdkSjvz
 2y+ksRgBUkhK1Kiqe/QuddgvuT9OUFBCrOtZxkBgJYRbxBNTOnIehhEoK9sdqjwvwNyxJUEX
 er1UgErP+MJTvwpW1J8Uq9oV+NbqgNQ7ca5G2D3x7DI2Tbbw0iu+lmCIdXQpTk7hacErGscC
 p/eaJQFd6EPAZPS7BQEgjuDDjGCy4RzTSiobtqn0ocaEj+w7365FVKKZD2LjEm1mfBK/4nml
 H/vlssXkKKpUGetaib6AVpZK3a0xBTMi04ft/V8iGGJuVhE45WjjKSckmeLafTZCGxN9ABEB
 AAHNK0Jlbm9pdCBMb2Jyw6lhdSA8YmVub2l0LmxvYnJlYXVAZGFsaWJvLmNvbT7CwJQEEwEI
 AD4CGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AWIQRkbHJAht3VPY+ptaDDJY+vq0O76wUC
 Za56QgUJC6uZkAAKCRDDJY+vq0O760ECCACDPj6p9VXaX4GF9p+3zhoDwxsUWh0xODwsK//h
 SBGsRmPaElApbOR5rs2WDFaPE95XG3Vdo2rno7i4mt0fASls7vq4P0BPJiPM4fKEr+kfXAxP
 2clqmkBMn2o+EEeUGDmnn3RAVzDDoyML6N/unpyOTHWfscxvvj6AKreUtxzQm0LVQRL+hIaG
 SAlcwpVWQBPhA6Roi1/7Uz9qm8nWjFOuV6vO5yVe0sfRrrO9AFkjBGFT/B06sSs33Os6Hjs5
 2ckLn5Gxlty8XgTpbaG0bJiV+GDTcXh5UV8XVKaMomOLJxWw8US6PSSqQxodWuJqsvlxXV93
 ae6cMIosHsriaUo5zsBNBF3FR7IBCADEWwsQar1qtLKdoAoqKThz+kQXu3UADQ0c8JCgsp+8
 a6hFDg+uIw0j852lkJOPHll414Qw+kLC1bUo7H6+mPHzko/aR3TFz6vvOrm0afwFP7sm9Vbe
 UbpeohnrXBBwb1PjNhDphaXeMVovJqSgRifhc/sAR6fXi+2OCrH+8CUYa0aHhwoecStgw/tm
 EDPZ8lCsg5IdyO5Er4ZMOl/EbaRuTOpl7U/GzJBQSYf7t4KHotIpEeuJVv4X/+nTW3ALG+dG
 pRA5GplNnlQNfsEymnT4W7RbfafDp7Q7nK6w7d55ZU8UyiS6N/1wh/DL1hntC65lQbHlmwNx
 RG132ecNOcqFABEBAAHCwHwEGAEIACYCGwwWIQRkbHJAht3VPY+ptaDDJY+vq0O76wUCZa56
 dQUJC6uZwwAKCRDDJY+vq0O76y9jB/0fVyz7D56D2A6MUGASNkh6IiuHCF/UgEfU0m5fC6pd
 38+fmyUMkGR3QwrtZP3SCLaBKT4LHxhVu7tEUWI+6WK7/s0Jsx5L8aIfDKY4zjqQ9InP+LkA
 azkKW4JpIb9Ny0FzRjUyf9gReU5nMy9sUNS4DhoHG6c/bQYqkj9uA5AokiDDCRg2dobfpaNP
 C6qQQIXD+wi0aZ4jR5r4rXMxmwFutandoaxDpPBj4EXlyRHhzyKfgb2c6TXdCVQ5YJUFUi9k
 gl7CXplN8vBpMuzN5y6NPd3Xekcp16D5XCQNBQ2drIsnFR4IbDPI1Mqd6OJYH0vum6g8FmoN
 s7ygc4oi12nU
In-Reply-To: <0d08b978-10cb-4512-bcef-ca1f192049ab@enterprisedb.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/0b7ef57e-bf30-440b-9478-d8f587f2637c%40dalibo.com>
Precedence: bulk

On 2/25/24 20:13, Tomas Vondra wrote:
 > 1) name of the GUC
...
 > 2) logging just the failures provides an incomplete view
 > log_parallel_workers = {none | failures | all}>
 > where "failures" only logs when at least one worker fails to start, and
 > "all" logs everything.
 >
 > AFAIK Sami made the same observation/argument in his last message [1].

I like the name and different levels you propose. I was initially 
thinking that the overall picture would be better summarized (an easier 
to implement) with pg_stat_statements. But with the granularity you 
propose, the choice is in the hands of the DBA, which is great and 
provides more options when we don't have full control of the installation.

 > 3) node-level or query-level?
...
 > There's no value in having information about every individual temp file,
 > because we don't even know which node produced it. It'd be perfectly
 > sufficient to log some summary statistics (number of files, total space,
 > ...), either for the query as a whole, or perhaps per node (because
 > that's what matters for work_mem). So I don't think we should mimic this
 > just because log_temp_files does this.

I must admit that, given my (poor) technical level, I went for the 
easiest way.

If we go for the three levels you proposed, "node-level" makes even less 
sense (and has great "potential" for spam).

I can see one downside to the "query-level" approach: it might be more 
difficult to to give information in cases where the query doesn't end 
normally. It's sometimes useful to have hints about what was going wrong 
before a query was cancelled or crashed, which log_temp_files kinda does.

 > I don't know how difficult would it be to track/collect this information
 > for the whole query.

I am a worried this will be a little too much for me, given the time and 
the knowledge gap I have (both in C and PostgreSQL internals). I'll try 
to look anyway.

-- 
Benoit Lobréau
Consultant
http://dalibo.com