Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gKfmv-0006eK-Ss for buildfarm-members@arkaria.postgresql.org; Thu, 08 Nov 2018 08:35:42 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gKfmu-0005vn-IJ for buildfarm-members@arkaria.postgresql.org; Thu, 08 Nov 2018 08:35:40 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gKfmu-0005vg-9X for buildfarm-members@lists.postgresql.org; Thu, 08 Nov 2018 08:35:40 +0000 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gKfmq-0001XA-Os for buildfarm-members@lists.postgresql.org; Thu, 08 Nov 2018 08:35:39 +0000 Received: by mail-lf1-x12e.google.com with SMTP id z13so3537217lfe.11 for ; Thu, 08 Nov 2018 00:35:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hagander-net.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7TbnW9+3Qq/5kP30qpXx7RkXsN9PbAWG5DTHIGBpNog=; b=T8J6C/O4ZqfMQ6+XEP9SsOD0JIvxvo+BSMtbB98DvhyTslnBzIj5GtNlWvlW/oo3+a CVWzk031PPfrf1X0pqAE3IIUAKYDcIyCvDaUjRSbbcl9YnR2l0sNVflsXDqzN40XUY6W tCDGF20UhDvHnaU80qageHKUsG/YYPGloM080lwvM530eMP9TDBWXpYfE7Az2ocf+jP8 CIpt+N8rAyHwBGIIJX92nYI4SknCRV03ba4SrGTmAY7LTO7dCl/6DyPTFD7xNCV3lchI vjslz4B9bfahVy0tCYOzxkshKRDdZTamkolHU3kt7c2nlvoQxhnyZ3EqdezOLLb5df7N ByWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7TbnW9+3Qq/5kP30qpXx7RkXsN9PbAWG5DTHIGBpNog=; b=ctUtI37eVky/4OYiPvTRVBBulEHkIBj3R4BHQ+36hpw4ZinQZHRPFcdQdQa1p/aD2b 1oOceRXCx8QJcVTcNHt6i0nEw5bZoegjaGAoYdkkvTUWLQe5KJInVNiVSayg94yNAZuZ 2BzjIaLrNeBnm2Vo8XJetJ1V3chlj+3t/FiCZjAki6fYC82B12QtZjGJoJ38rQlxfp76 fNehFres+lUixVheOy9BSyx16D28a43Yacg12psNrwnX0AOg3s0gK3Ocu+5+u9FECaLz oR+1cTRk/HvIm4AmA7T/+K69gcCYPqI1Cg/PcK4woonbbNu304Y5HjU9IHPeRlJ0pVtv /ffw== X-Gm-Message-State: AGRZ1gLEWGMWb1d4ef3l8ucwfK+THnja3HZtJgexmKyF84mHqKGk8pJY KLL+UMycBoFZyc0uS3Fn3HYmRCQEwyKxT2upCd7jzw== X-Google-Smtp-Source: AJdET5e/DnQG5z5DjDeqtpN5L4bV8exvZcbvDxTaRmfWqoib4V2XA+wd2jtCub6rkXdVh6zs/Pl5villkJkUKxN+91k= X-Received: by 2002:a19:1cb:: with SMTP id 194mr2140591lfb.61.1541666134109; Thu, 08 Nov 2018 00:35:34 -0800 (PST) MIME-Version: 1.0 References: <428FDD47-5325-4904-9306-F2D6F9D8232D@mac.com> <8832f403-b6ea-e581-b631-a6253731fde6@kaltenbrunner.cc> In-Reply-To: <8832f403-b6ea-e581-b631-a6253731fde6@kaltenbrunner.cc> From: Magnus Hagander Date: Thu, 8 Nov 2018 09:35:22 +0100 Message-ID: Subject: Re: 503 Backend fetch failed errors To: Stefan Kaltenbrunner Cc: remi_zara@mac.com, buildfarm-members@lists.postgresql.org, sysadmins Content-Type: multipart/alternative; boundary="00000000000049481f057a231e04" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --00000000000049481f057a231e04 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 8, 2018 at 9:31 AM Stefan Kaltenbrunner wrote: > On 11/8/18 9:19 AM, Magnus Hagander wrote: > > > > > > On Thu, Nov 8, 2018 at 9:01 AM Stefan Kaltenbrunner > > wrote: > > > > On 11/7/18 6:40 PM, R=C3=A9mi Zara wrote: > > > Hi, > > > > Hi R=C3=A9mi! > > > > > > > > I=E2=80=99m getting a lot of these errors with coypu (several pe= r day), > > but not systematically. > > > Is this a problem on my end, or is this on the sever end ? > > > > > > Query for: stage=3DOK&animal=3Dcoypu&ts=3D1541573277 > > > Target: > > > https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl/53b137e7c765b781699b= be73e3aec7751a8c4ab7 > > > Status Line: 503 Backend fetch failed > > > Web txn failed with status: 1 > > > Query for: stage=3DOK&animal=3Dcoypu&ts=3D1541575423 > > > Target: > > > https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl/5e8e0913dc9f9a580a41= 25264d74fff95f26c926 > > > Status Line: 503 Backend fetch failed > > > Web txn failed with status: 1 > > > > given the error this is something that is created by the varnish > > instance that is in front of the buildfarm. On a quick look I could > > immediately figure out what the problem is - but it looks like you > (or > > somebody else) tried at least to click one of the links above using > > hist > > desktop browser and got an error about a missing branch > specification ;) > > > > > > > > AFAICT: > > > > A quick look in the logs indicates that the buildfarm is responding: > > - RespHeader Status: 492 bad branch parameter > > > > However, 492 is not a valid http status code, so Varnish can't handle i= t > > and thus returns 503 failure to the client. > > I think that is not the actual error that R=C3=A9mi is experiencing- the = 492 > case (which is indeed an invalid http error code) only happens when one > actually klicks the link in the mail above(which I guess some did and > you found in the logs) because the actual BF client will add a parameter > to the "Target" URL. > > The actual "errors" dont seem to show up in the lighttpd logs afaiks. > Oh, sorry. I was checking the one called "target", I assumed that was the URL that failed. Assuming for the original ones the ts is part of the URL, none of that is still in the logs. Or are they post parameters? Do we know exactly which URL is actually failing, and when (exactly) this happened? --=20 Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/ --00000000000049481f057a231e04 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu= , Nov 8, 2018 at 9:31 AM Stefan Kaltenbrunner <stefan@kaltenbrunner.cc&g= t; wrote:
On 11/8/18 9:19 AM, Magnu= s Hagander wrote:
>
>
> On Thu, Nov 8, 2018 at 9:01 AM Stefan Kaltenbrunner
> <stefan@kaltenbrunner.cc> wrote:
>
>=C2=A0 =C2=A0 =C2=A0On 11/7/18 6:40 PM, R=C3=A9mi Zara wrote:
>=C2=A0 =C2=A0 =C2=A0 > Hi,
>
>=C2=A0 =C2=A0 =C2=A0Hi R=C3=A9mi!
>
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > I=E2=80=99m getting a lot of these errors wit= h coypu (several per day),
>=C2=A0 =C2=A0 =C2=A0but not systematically.
>=C2=A0 =C2=A0 =C2=A0 > Is this a problem on my end, or is this on th= e sever end ?
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > Query for: stage=3DOK&animal=3Dcoypu&= ts=3D1541573277
>=C2=A0 =C2=A0 =C2=A0 > Target:
>=C2=A0 =C2=A0 =C2=A0https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl/53b13= 7e7c765b781699bbe73e3aec7751a8c4ab7
>=C2=A0 =C2=A0 =C2=A0 > Status Line: 503 Backend fetch failed
>=C2=A0 =C2=A0 =C2=A0 > Web txn failed with status: 1
>=C2=A0 =C2=A0 =C2=A0 > Query for: stage=3DOK&animal=3Dcoypu&= ts=3D1541575423
>=C2=A0 =C2=A0 =C2=A0 > Target:
>=C2=A0 =C2=A0 =C2=A0https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl/5e8e0= 913dc9f9a580a4125264d74fff95f26c926
>=C2=A0 =C2=A0 =C2=A0 > Status Line: 503 Backend fetch failed
>=C2=A0 =C2=A0 =C2=A0 > Web txn failed with status: 1
>
>=C2=A0 =C2=A0 =C2=A0given the error this is something that is created b= y the varnish
>=C2=A0 =C2=A0 =C2=A0instance that is in front of the buildfarm. On a qu= ick look I could
>=C2=A0 =C2=A0 =C2=A0immediately figure out what the problem is - but it= looks like you (or
>=C2=A0 =C2=A0 =C2=A0somebody else) tried at least to click one of the l= inks above using
>=C2=A0 =C2=A0 =C2=A0hist
>=C2=A0 =C2=A0 =C2=A0desktop browser and got an error about a missing br= anch specification ;)
>
>
>
> AFAICT:
>
> A quick look in the logs indicates that the buildfarm is responding: > -=C2=A0 =C2=A0RespHeader=C2=A0 =C2=A0 =C2=A0Status: 492 bad branch par= ameter
>
> However, 492 is not a valid http status code, so Varnish can't han= dle it
> and thus returns 503 failure to the client.

I think that is not the actual error that R=C3=A9mi is experiencing- the 49= 2
case (which is indeed an invalid http error code) only happens when one actually klicks the link in the mail above(which I guess some did and
you found in the logs) because the actual BF client will add a parameter to the "Target" URL.

The actual "errors" dont seem to show up in the lighttpd logs afa= iks.

Oh, sorry. I was checking the one= called "target", I assumed that was the URL that failed.

Assuming for the original ones the ts is part of the URL,= none of that is still in the logs. Or are they post parameters? Do we know= exactly which URL is actually failing, and when (exactly) this happened?

--
=C2=A0Magnus Hagander=C2=A0Me: https://w= ww.hagander.net/
=C2=A0Work: https://www.redpill-linpro.com/
=
--00000000000049481f057a231e04--