Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1veKPw-00BwVx-0G for pgsql-hackers@arkaria.postgresql.org; Fri, 09 Jan 2026 21:53:13 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1veKPu-008lSB-1D for pgsql-hackers@arkaria.postgresql.org; Fri, 09 Jan 2026 21:53:11 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1veKPt-008lS2-30 for pgsql-hackers@lists.postgresql.org; Fri, 09 Jan 2026 21:53:10 +0000 Received: from mout.gmx.net ([212.227.15.15]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1veKPs-0058US-30 for pgsql-hackers@lists.postgresql.org; Fri, 09 Jan 2026 21:53:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.net; s=s31663417; t=1767995586; x=1768600386; i=mbanck@gmx.net; bh=tO8Jf1DFXnrs0ac5ez/w/F24WIQ2dp+eb0ok4pEgZe0=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:Content-Type:In-Reply-To:Content-Transfer-Encoding: cc:content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=nqRg2TX9NY8YhncpJAef23/wmJj9A/P7GDWB8fzJ9LhKD3FJqGKSbip4ZcwcS4Hc eKR27eGjX6w0sctXxugzqkK2D2MfhHP0MgTlzeHYraxU1wmddV+mtCFv4D9ZjeqrO F4C2SdfHsoQXMnVJ+DUZh31tyeHpBSFX5rEHzh4K2/Xa6I5S0H0tGIZW7xamYlgYS EEl+p4V7ufX+xEo9n5Xza2/WDg4TBHLgoK6FrGAFPcwG+5h26BJXQzzmpYAXqocKW w9p7pAHV0dRkxCB+Cjndwd0t3jset9HP8Xf3FH+ADGsnUcboWZ+Pj+3+LMEoKlFMt MRX8HM67g9Eiumz1FA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from lightning.p46.dedyn.io ([88.217.119.76]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1M3DJv-1vhLky3QVx-005jAT; Fri, 09 Jan 2026 22:53:05 +0100 Received: from mbanck by lightning.p46.dedyn.io with local (Exim 4.92) (envelope-from ) id 1veKPo-0001Vg-QL; Fri, 09 Jan 2026 22:53:04 +0100 Date: Fri, 9 Jan 2026 22:53:04 +0100 From: Michael Banck To: Tom Lane Cc: Andrew Dunstan , pgsql-hackers@lists.postgresql.org Subject: Re: Maybe BF "timedout" failures are the client script's fault? Message-ID: <20260109215304.GC21237@p46.dedyn.io;lightning.p46.dedyn.io> References: <2423164.1767991263@sss.pgh.pa.us> <20260109213255.GB21237@p46.dedyn.iolightning.p46.dedyn.io> <2430115.1767994942@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <2430115.1767994942@sss.pgh.pa.us> User-Agent: Mutt/1.10.1 (2018-07-13) X-Provags-ID: V03:K1:ePEEIHdXn5Xi19lUsn20OcaAqjDrz5byO0rV1ta2uP3Ak1V8Asa +ZVaRUn/oL9YRDUekutYwalITrXcEVmkEO/VD3A9uqRgBEmhoMTaO7aUZf7FOjTitStbl7P dldnFCri28pQ/gIxOhnmWFxxj1rwmRpYHeJ5uG5nhKjGQq6f+o+3HlZlhadGy8DnEWf79pl WjMYrPyI9al7EH8YO1C0A== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:2cAYa4iAjzg=;8eTMsN3ul9gNPabYm8CuG1UYaLF 3pKJmkcIrMDanhgNCzv+Kvwtvcq8Avw3CguQU0irwydjSou6F2TPpmzoulBi/9i5lP132QYaY 29wZNVCUn1/ttKcKUjtQGTXVdQZdf5fLn5vwItpAhpfxKEi1q6boZMrFDAcopm6Au+oMMI00u dtxai4rNR1tnwcuWCToEE7hPVsdju5PCQG30p4CVNLL6X5do6JFVhCg0YlBcfpsf/vTb3qCez /eGNyi2S+qTuu8kUDguP3qFmAniU9YVjwuDVUX30Qi3EktwXILTAQMiGkRraxLzFwUnwzZZRt XC51oQyipTV4yMIavQSaMa1DxR722putijTI/xn93XR8A9T2mSHbIPQOn22jRvjmBdPTKzxw7 sTWvESlJSU7Ir0NypCLK5shTQzabye5FjhQjk6iABkwcHQOapyPggRDK8o3jnFmm6s2ReEs5u Qo21+JIkb/u9E6i5FOFPG+BaKaw/NHXtJVKB1Lk6t5n60J7mn4VPt8rlGmeORkCUs1QKXIlwK MCnbzOeBuMe/8IBPhX0x98zVHNs9pCFantg53idl6Skxo7WMl2K9MwNixg9zM9rQa079LYiCU 0zGpp8Q0A+7rvlQ/oImTIMTo0Ool5D3RZhOwLEK6jL+emTC8WIhhtmESiGbhdrGtqlUf3RqzF gxHU+8IXDbvUaXEmdiZ0kEHPGpiiTVHKXsG3iFYCljYU2CFVLFUuOmb+EknlQYmub+7nYtASy 8XzAXCR8/RQDZSP+GTxCa6yXlg/RiyLwGR0S2uRJ7i5T9THe3sV6eDTZX2T4JtCNX0zgA8vGg p9kmMy4d2H5p1pIAZ0zcom6Gzgi8QQks9RXGyBSUNxE6oJupOgSiad+pv+7Yf3stsg6tT+9P3 l2WyK8jD2r0gdvJPfPHALN27QVMS+SyRAkbhesQj99XG3YyoSuSd2cLQKekdfrWMtmKY5igBA iVj5h32kfhdfonIEbq9rvntDGxd/nZvav4AGQoOkUeTaLBgNQORVKCJDc5nHALaoC/irHoZW4 GyCobG09v7ksxNgIhSnxguCAMD/CIc1caZmmDGLQh1HlqnCVLumlcCG/Ht08j4/kMPLrhDL8H h7EzE/XAHGteLT+NZB2c6e8JoFfX4tS+OBnNnJ9vZykM0Q/tZPfMug86lDzT7Vc/4aXUyIb44 GsDtigMZPUJY4G8ilt4spfU6TF9dfzSCDqaJ3+yhe3qfhPRaZK0WG8BtMQjF55ESbwqyoF2UV RdO6PXC71zPybaG0oMRe+/mtpcgJqyWEVRi3bc310PUKGo8ApaMLvx0POtYsJYCPAQ8nuDEd5 z72oqT+D1425J31rObWijeDSzjl1pDlP2FXTyNEplrgqvLZraQiZpcjsvN+nhp2v/e1Q8XGRU LwK60YwoJdC0W628P1qic+srJi7ufo/MqP5GvNeawmwx+cyVHU1ADwknfB4F+dLIqdDsyfFSZ bnrjI95upzJVxfdaPT3kCjVZpTrsOv8md/Ws68GgIT92INNJOLWxUAmwBTYDjkgSkryzacRQt kj6gw7bF3Rd43kkAwi+dCYexq7QmswQVTIe3o1V3SDMAuWiA2679CcdbJXOwLbfT/6YC/wJOC YjMprv/sSwd2sByPc3FFKD2XTkxsnrKiCNLAJImPp+JdohyAAPJ8uZuY+KtCtcjFDanvwT5Dn cr9ttm7AT1j9FHPRw4M/vg0iIbtMS9pZ5Br6cCCCPXf5GXYwenlDXyWVjk9BGTxq93UItYX2H YitvPutPABdhRnibLbX8Be//veOVlUVOUTPsJR8ty/WTc7m7wPf8QTGkKyBzQ4lhp4QiAbbMR E8cM7tw/mUEaAvDMheW76goIBWUinKuee0KI42zjK+U0Fc9ceNDb9t5w5wYAOI24TZiiEus1W VpNCSmy8onqiNHimHMJiCBvCiklPiZ8AL8qu91xOwUP9KaoUjVXJ7XdbbPv8xDgAfrX087rjP Z9qBhXHYsDYvHljp4+NTpNynhzA2uWRPG0i0te5us2IfMYwBwZbqfmiHAHq3qMK64KsMYKikd u78pxaAl/jMg8fcwTZtrMk6mb/yKS31Ue6SKDA30F20Wt4Z3o9jIEFvGZdJNwFRDnll4g4AIJ JxNHtn8p8hvad8XJrO2SAgj3JSoXs6Dv15SFIoaPbeNchedLJzD/jDUob2wv4Va+eB5RUdJ07 H8DkP4cFr7xhsnptv+/Sl7U7t1wczo6kwISESX+FhT3V2h5tjPEZ3RyI0Sh0hQt8NFo4w4bYX +cmyJSSBi6BNwcZYQCODETudpZj/biNd0k5A71jBGS/6zBnve5eO6UtmLtaUfIsMUxEhj4/z0 cOeSIKA4UDprcRhb7xl4LW6qjwIQ3ybTPEovB0S+enPocopEac/vdE6mI+IpBA/MW6aeCkQDd xfNCmNyuSMXfM6pHbjka6u3SdrRFstLhT4DG2WI6S7YFQe4sFMsgUZThsVtKpF4BNEZl1/JAi qhCfIUSnqpOsIDDzwl+fIcMyTRuJSfH7MyfppT8tiyhtsXWskadSpngRHJWWV1CcQJgv5ma5N LlM5LOTjuJgsDaNuil72VpjlWPFDyS7wlRtWWGq9MftEMqutmsKKNmGHv/HRKRXizSbqAyAa+ CoSlrpQl7/CgFNccS3RfYxfWN8cMbSSF4lI18fZvXBTxKAPASufQ5XFHAag+G9cD9DjSfyJKw kZVpKsMPUIea46Z/qUjo84GrkZv5J6gHj+7aZRbltLhonijEP7OVh+vsajKrygmxi+4kDAi+I LkRM8Q+ye69tIVK03k+PPifUhCFVDSrcDD9s5SK8lQbn3zHGViR73xMwbcojxl23pt1l8/OUj +SXSvLuAJqDWvB2SRnV/AtYCgb405jDbacna3UlR3r2lhFmhzbxYm6rhRMv5IKGUvE3/PZ6dP mSfIXNarwVIb4rGnF54HXdJk8hRMVCAVCc28J5LW7dJ7N3nc0ZDgqJjQHrB4WBONo9SavTboP hQbBJfp3GxP/Xa6PyivVdiTu7fjnwue9IDm4yk0QJ1Xj1zedHqXd9OmjW/b6jc27X68WUCZlt V2A5xQJl5+4OattzCRAjydIZR3ggAPoZvVpTmynwo9a+48Mew9rdF2qcRx9YiPY53InKFwiOx heOX+hi2cVoto96njn7y7l4gSdd6jSzDkk2E1mas8Yxrq6OvVjihQT0Y1L3ggVNgh9HOGQMJh h1VXBYzWB424QLhn0rscvaFXWyvPpuspXZXL8genH7p3qYBFdhiFang5w/TkNHIxdPxpplPR4 1R+gqgKTtw3jubnr7nZMIOaVFoiFXNRgWMzp7h7I9J2/jR502xhGH1TVjXHAfnN4UpAXV7y8k 1uLZVOUS5XYi+tKWr6WemeKE8r3H7ZrTX6ZZWyxDep8tMlLakcAbXvOZtyOxy4ySn4gFPtns/ PHOs5WjQopcqLF7OPJhi86+BvjsjsNXMYdhZLDKn+kE7eMJZwDURfHUxBvD+tV7jxF3LXE166 NywtTRAR6NdvRhRvHf5D4mGvAm6gfhQdYS/7E/Eoz4Yz92O+SsPN1tzHm01uR2z1LSPT/MoyK 3EjKjGzaMlMJBdX9zBymR+xqrL07ADYU3mY8GjOgUYrppAH3pWaKEyDt63f/ScbrwC5kadecL T+/8RnXpj82OPSLYq3MZ2pjBXqlDaZuFRp1OhJ6jsfaT9rAHCmNiehvYUHvhb4Oz5ggpqlDT0 4bhbt3kKl7RQ/HmE1FIIWTJ7/1RUwJrauRvHjQexpmZkceNEi3rDnb9kXF7IEC98aV9vJOpCG o4qk2VnHfi9kfekG/K3OSZClto8yhxJLoEjzz3+NWczz60MgtpOR20XmcnI8ZdA6XVzwD0NP7 qEum+86s9lOw65eFYhRy/ZS5Gyb/WsM2fWgDGbenvCGAWjFsyjFc5BeKY5Q0Qs1hJ8G2T3RcK tS7XraOoivCH0njL2EVqJS+XJnpTl++Ej3spNK3VanupcU5DZUsc5WOEPC8fLZ+PXEtV8aY9d zCQWcx5+pl3KZg8PgyNuqBo6WUKjo5JfS7WpbgoxSQ9t2cvyytrrMBYut0rYvRHwAEMT3UBY0 5JE2zjq1QiJcB1LVMmizwy4Sh7oPpZv9SzNI/sKfXobJhNLSIqah3D/QMxg5DEuvlsYIjd+Fr lM4fu0ReiSBImpD2i5nMr4wUtehh3TpZrE/evRwrBpV4nan1JLu0zoDrlcCxD9pn6bxpDiDR4 epJaME5TKec8eOJ6OLn3+uecEwKL3+/XOzrA1GVeMEHfCtqOWHU9ZhAxB73NC8cIvfvdoDvaO jB27VJaBwEn1WYHKI6kGuf7fOv+wM2JHdmztwEpEDiyOuT636dWiLw/31vpxM50dT5fhNEzSn SZSkZm4leYOWITmHBcEPSjLi6ctU7twH6Rku79/DSsINcn1Ok5GSXhPDdY62QMj2hSoCuN19d WnNXCwTnkFrH5aImQeL4aS3tugm+IK6h2JcfSnYqstzE52TY1iwXQVpZ6hbPTiZrKu5y4+dZb 9N6Mnnquapmmq3WRmtc+g1RbOAc8Ba1hcErLJ4kznELxqr6MIjy3a2aP/MGJvWoORLElOZKyB Ks9iMsxnb6q6wfh3/t8jux90fQrAI+MMZuog1AxbfC2WkcZkBG1zlQKLMORRH9k9x7mmq0D5H xb1jfnpkfsK2TaknV1EnjkdaxS06Bp+nhNRk1dAUuGbUVjd2lJWi+1tZifF7USpHGtl2APKWk ejCVV10Q0NuAPipOk9LouFFAaCuB2szNJD0EgrSBugfzDzfxJRUilVsjOcUh+Lhyih2UFON+F imesJr3lNlPfCJH/Al+o7DVy4OKqn99rbgRx7wYHaJsuOYV179Fihr7BQWVycmRxAPzTmCODf 44ns0i4PoVk+UAxZdCUw1E4PiHh9wScvZnr8r6UqToW4o5/Fqu1rLEjHVimRz2V4cQ7qMFJGK MI9nkhriDv57Wj4jnKhs9BfkK2XLKZwuBI3kUlhCTm3eb9lsbBn5RTbGlGE0c900NgidBbIbW byuv7OQGD05wj7n0ceYz4OQuv8ASldYVJ9synIIr7gfvTPFZtCAlCn1N09Ej5wjHWAS0YlHbP KsFSvpaGa1dZUygaygcS84PaK+TysAxdbWzq5ukTDeiCTGujX/VjS08Cp4xziopVjZm72F5JN qy6tIa5iAEOaxHCSfovLX/nmVqiMq2gSpFjkayTAD3fr7gqMGlEUbTLwcdzQ== Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hi, On Fri, Jan 09, 2026 at 04:42:22PM -0500, Tom Lane wrote: > Michael Banck writes: > > I've ran that test manually quite a lot and either it finishes in 10-1= 5 > > seconds, or (presumably) never. This is not really easy to see in the > > public builfarm logs (at least I can't find it on a quick glance), but > > I've routinely checked the log timestamps of the runs, and they really > > take one hour (wait_timeout) in the case of a hang. >=20 > Hmm. Then why is the BF report showing that the total runtime is > nowhere near that? I wonder how those times are gathered ... Looks like the server takes the timestamp of the logfile as an educated guess when the particular stage finished: https://github.com/PGBuildFarm/server-code/blob/main/cgi-bin/pgstatus.pl#L= 496 But this does not work when the stage hangs and gets terminated externally, it seems no output is appended to the stage log in this case by the buildfarm client and (at least for fruitcrow) neither by the stage itself, so the server thinks the stage duration was whatever time it took until the last piece of output before the hang. Michael