public inbox for [email protected]
help / color / mirror / Atom feedPossible infinite loop on buildfarm animals
4+ messages / 3 participants
[nested] [flat]
* Possible infinite loop on buildfarm animals
@ 2024-03-15 18:42 Tom Lane <[email protected]>
2024-03-17 17:29 ` Re: Possible infinite loop on buildfarm animals Noah Misch <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Tom Lane @ 2024-03-15 18:42 UTC (permalink / raw)
To: [email protected]
Between approximately 11:05 UTC and 13:25 UTC today (15 Mar 2024),
the Postgres git repo contained a buggy test recipe that caused
an infinite loop that will eventually exhaust disk space.
If you have any animals that might have launched a test run on
HEAD in that interval, you might want to check up on them.
A manual kill of the process that's consuming 100% CPU should
be enough to get out of it.
Another idea to consider is to set the wait_timeout parameter
in your animals' configuration files, to put an upper bound
on the total elapsed time for a run. By default that's
infinite, since it's really hard to select a one-size-fits-all
value ... but it's a good backstop if you don't mind picking
machine-specific limits.
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Possible infinite loop on buildfarm animals
2024-03-15 18:42 Possible infinite loop on buildfarm animals Tom Lane <[email protected]>
@ 2024-03-17 17:29 ` Noah Misch <[email protected]>
2024-03-17 19:38 ` Re: Possible infinite loop on buildfarm animals Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Noah Misch @ 2024-03-17 17:29 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected]
On Fri, Mar 15, 2024 at 02:42:14PM -0400, Tom Lane wrote:
> Another idea to consider is to set the wait_timeout parameter
> in your animals' configuration files, to put an upper bound
> on the total elapsed time for a run. By default that's
> infinite, since it's really hard to select a one-size-fits-all
> value ... but it's a good backstop if you don't mind picking
> machine-specific limits.
Other than CLOBBER_CACHE animals, the server rejects results older than 24h:
https://github.com/PGBuildFarm/server-code/blob/8572ac7/cgi-bin/pgstatus.pl#L273
The same 24h should probably be the default wait_timeout. One might use a
longer timeout if wanting to attach a debugger to a process of a stuck run.
If one just wants an intervention-free buildfarm animal, 24h is good.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Possible infinite loop on buildfarm animals
2024-03-15 18:42 Possible infinite loop on buildfarm animals Tom Lane <[email protected]>
2024-03-17 17:29 ` Re: Possible infinite loop on buildfarm animals Noah Misch <[email protected]>
@ 2024-03-17 19:38 ` Alvaro Herrera <[email protected]>
2024-03-17 19:43 ` Re: Possible infinite loop on buildfarm animals Tom Lane <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Alvaro Herrera @ 2024-03-17 19:38 UTC (permalink / raw)
To: Noah Misch <[email protected]>; +Cc: Tom Lane <[email protected]>; [email protected]
On 2024-Mar-17, Noah Misch wrote:
> Other than CLOBBER_CACHE animals, the server rejects results older than 24h:
> https://github.com/PGBuildFarm/server-code/blob/8572ac7/cgi-bin/pgstatus.pl#L273
> The same 24h should probably be the default wait_timeout. One might use a
> longer timeout if wanting to attach a debugger to a process of a stuck run.
> If one just wants an intervention-free buildfarm animal, 24h is good.
Maybe that should be the default value embedded in the buildfarm client
script, which can be overridden for specific purposes such as
CLOBBER_CACHE animals?
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"No hay hombre que no aspire a la plenitud, es decir,
la suma de experiencias de que un hombre es capaz"
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Possible infinite loop on buildfarm animals
2024-03-15 18:42 Possible infinite loop on buildfarm animals Tom Lane <[email protected]>
2024-03-17 17:29 ` Re: Possible infinite loop on buildfarm animals Noah Misch <[email protected]>
2024-03-17 19:38 ` Re: Possible infinite loop on buildfarm animals Alvaro Herrera <[email protected]>
@ 2024-03-17 19:43 ` Tom Lane <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Tom Lane @ 2024-03-17 19:43 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Noah Misch <[email protected]>; [email protected]
Alvaro Herrera <[email protected]> writes:
> On 2024-Mar-17, Noah Misch wrote:
>> Other than CLOBBER_CACHE animals, the server rejects results older than 24h:
>> https://github.com/PGBuildFarm/server-code/blob/8572ac7/cgi-bin/pgstatus.pl#L273
>> The same 24h should probably be the default wait_timeout. One might use a
>> longer timeout if wanting to attach a debugger to a process of a stuck run.
>> If one just wants an intervention-free buildfarm animal, 24h is good.
> Maybe that should be the default value embedded in the buildfarm client
> script, which can be overridden for specific purposes such as
> CLOBBER_CACHE animals?
We don't normally hard-wire such choices in the script, but it could
be plausible to change build-farm.conf.sample, perhaps like:
# max time in seconds allowed for a single branch run
# undef/0 means unlimited
- wait_timeout => undef,
+ wait_timeout => 24 * 60 * 60,
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2024-03-17 19:43 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-03-15 18:42 Possible infinite loop on buildfarm animals Tom Lane <[email protected]>
2024-03-17 17:29 ` Noah Misch <[email protected]>
2024-03-17 19:38 ` Alvaro Herrera <[email protected]>
2024-03-17 19:43 ` Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox