public inbox for [email protected]  
help / color / mirror / Atom feed
From: Stefan Kaltenbrunner <[email protected]>
To: Magnus Hagander <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: sysadmins <[email protected]>
Subject: Re: 503 Backend fetch failed errors
Date: Thu, 8 Nov 2018 09:42:22 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <CABUevExZhPyUCzeNkH6_sja6YA1PZemm=_NQvedAdYi-=oC-vg@mail.gmail.com>
References: <[email protected]>
	<[email protected]>
	<CABUevExk3ytB9JPuamcpAy-d75j24JwQ657+Y-Jgt3MYuaK-Qg@mail.gmail.com>
	<[email protected]>
	<CABUevExZhPyUCzeNkH6_sja6YA1PZemm=_NQvedAdYi-=oC-vg@mail.gmail.com>

On 11/8/18 9:35 AM, Magnus Hagander wrote:
> 
> 
> On Thu, Nov 8, 2018 at 9:31 AM Stefan Kaltenbrunner 
> <[email protected]> wrote:
> 
>     On 11/8/18 9:19 AM, Magnus Hagander wrote:
>      >
>      >
>      > On Thu, Nov 8, 2018 at 9:01 AM Stefan Kaltenbrunner
>      > <[email protected]> wrote:
>      >
>      >     On 11/7/18 6:40 PM, Rémi Zara wrote:
>      >      > Hi,
>      >
>      >     Hi Rémi!
>      >
>      >      >
>      >      > I’m getting a lot of these errors with coypu (several per
>     day),
>      >     but not systematically.
>      >      > Is this a problem on my end, or is this on the sever end ?
>      >      >
>      >      > Query for: stage=OK&animal=coypu&ts=1541573277
>      >      > Target:
>      >
>     https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl/53b137e7c765b781699bbe73e3aec7751a8c4ab7
>      >      > Status Line: 503 Backend fetch failed
>      >      > Web txn failed with status: 1
>      >      > Query for: stage=OK&animal=coypu&ts=1541575423
>      >      > Target:
>      >
>     https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl/5e8e0913dc9f9a580a4125264d74fff95f26c926
>      >      > Status Line: 503 Backend fetch failed
>      >      > Web txn failed with status: 1
>      >
>      >     given the error this is something that is created by the varnish
>      >     instance that is in front of the buildfarm. On a quick look I
>     could
>      >     immediately figure out what the problem is - but it looks
>     like you (or
>      >     somebody else) tried at least to click one of the links above
>     using
>      >     hist
>      >     desktop browser and got an error about a missing branch
>     specification ;)
>      >
>      >
>      >
>      > AFAICT:
>      >
>      > A quick look in the logs indicates that the buildfarm is responding:
>      > -   RespHeader     Status: 492 bad branch parameter
>      >
>      > However, 492 is not a valid http status code, so Varnish can't
>     handle it
>      > and thus returns 503 failure to the client.
> 
>     I think that is not the actual error that Rémi is experiencing- the 492
>     case (which is indeed an invalid http error code) only happens when one
>     actually klicks the link in the mail above(which I guess some did and
>     you found in the logs) because the actual BF client will add a
>     parameter
>     to the "Target" URL.
> 
>     The actual "errors" dont seem to show up in the lighttpd logs afaiks.
> 
> 
> Oh, sorry. I was checking the one called "target", I assumed that was 
> the URL that failed.
> 
> Assuming for the original ones the ts is part of the URL, none of that 
> is still in the logs. Or are they post parameters? Do we know exactly 
> which URL is actually failing, and when (exactly) this happened?

well - most of the parameters to each url are in the error report (f.e. 
"Query for: stage=OK&animal=coypu&ts=1541575423") I dunno whether Rémi 
knows which branch that was for? - that one also has a unix timestamp, 
though I "think" that is the timestamp from when the build started on 
the bf-client and not the ts when the request was made)

Afaiks the two requests are not at all in the lighly log so only varnish 
might have seen them (though its unclear what error it got while 
connecting to lighty)



Stefan




view thread (7+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: 503 Backend fetch failed errors
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox