Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1U5qNf-0006kw-8k for pgsql-www@arkaria.postgresql.org; Thu, 14 Feb 2013 04:25:07 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1U5qNe-0003YB-OH for pgsql-www@arkaria.postgresql.org; Thu, 14 Feb 2013 04:25:06 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1U5qNe-0003Xi-77 for pgsql-www@postgresql.org; Thu, 14 Feb 2013 04:25:06 +0000 Received: from mail-wi0-f171.google.com ([209.85.212.171]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1U5qNZ-0001Kc-Ee for pgsql-www@postgresql.org; Thu, 14 Feb 2013 04:25:05 +0000 Received: by mail-wi0-f171.google.com with SMTP id hn17so6546725wib.10 for ; Wed, 13 Feb 2013 20:24:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=JCob/oyCRqiq6YXY+OV7FvPEp+Ze85TQp88jgAAGqHk=; b=EFGsQ4mUqr7FXrfvJCqg2XfagEvzpND+jHYGwpLdZMLL68epGaTYIQAMkoYTf07tZp mbTjfZ3B0Y2yWnfE4HR84eiLXbAQZo6RnPSsPzuN7ZMdELGM+pPy4tTJb8ES/yDNMpKj xLOD8VZiRhvKGJxBbVUjB1toZCJcbpL9xxYyBYlUKiT7LM/TEI4tr61Fe63kFHkSMKO/ JxV4UqyEZM4IHapJjoE33/VSfyQFnNk6FtA/ChlNFgsBbpeZKMpw+89wk55u5qc1K/6C XWXaDcEkl/jXIZ4jCVOAJwQaVLNKjjfiG1rVjt9e02C1spVFL9oEJRaJNmSV9bVElA4O mvSA== MIME-Version: 1.0 X-Received: by 10.180.100.163 with SMTP id ez3mr14078952wib.32.1360815899910; Wed, 13 Feb 2013 20:24:59 -0800 (PST) Received: by 10.194.152.105 with HTTP; Wed, 13 Feb 2013 20:24:59 -0800 (PST) Received: by 10.194.152.105 with HTTP; Wed, 13 Feb 2013 20:24:59 -0800 (PST) In-Reply-To: References: <20314.1360801157@sss.pgh.pa.us> <20130214002454.GJ4546@alvh.no-ip.org> Date: Thu, 14 Feb 2013 05:24:59 +0100 Message-ID: Subject: Re: 403 - Forbidden on gitweb pages From: Magnus Hagander To: Dave Page Cc: PostgreSQL WWW , Alvaro Herrera , Tom Lane Content-Type: multipart/alternative; boundary=f46d0444edff51a2bf04d5a7a468 X-Gm-Message-State: ALoCoQlSde2gcBBoxJ6GMpToUyoQraNYwAVS7V3Vn/nymU/7smoDlVJ6SIKG+wurTvItrwenLN6k X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-www Precedence: bulk Sender: pgsql-www-owner@postgresql.org --f46d0444edff51a2bf04d5a7a468 Content-Type: text/plain; charset=ISO-8859-1 On Feb 14, 2013 1:35 AM, "Dave Page" wrote: > > > > On Thursday, February 14, 2013, Alvaro Herrera wrote: >> >> Tom Lane wrote: >> > Every so often (a few percent of the time, I think) I get "403 - >> > Forbidden" from following a link on http://git.postgresql.org/gitweb/. >> > So far, a reload has always worked first time, but there's definitely >> > something fishy in there. I don't recall exactly when this started >> > happening, maybe a month or so back; it hasn't been there all along. >> > >> > Anybody else see this, or have an idea what broke? >> >> Yes, I see this also. I reported it to Magnus, who muttered something >> about Varnish possibly misbehaving, but that was the end of it. >> > > He told me it was intentional rate limiting, applied because gitweb is so inefficient. Correct. And varnish is not involved at all. I think alvaro is confusing it with me saying that varnish might be a way for us to actually fix the problem. The problem is basically that gitweb is a big perl cgi (yes, cgi only) which means that for every request it had to reload all of perl and reparse the whole script. And on top of that the actual implementation of several features is fairly slow in itself. This caused easy overload - for example the whole site went down when a search engine spidered us... The bandaid solution was to rate limit access per ip. The error you hit is when you're rate limited. Now, I was supposed to have on my to do list to make a better fix. But as this one was implemented a long time ago (measured in years, really, I have to admit I forgot... I'll put bit back on the list. /Magnus --f46d0444edff51a2bf04d5a7a468 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable


On Feb 14, 2013 1:35 AM, "Dave Page" <dpage@pgadmin.org> wrote:
>
>
>
> On Thursday, February 14, 2013, Alvaro Herrera wrote:
>>
>> Tom Lane wrote:
>> > Every so often (a few percent of the time, I think) I get &qu= ot;403 -
>> > Forbidden" from following a link on http://git.postgresql.org/gitweb/.
>> > So far, a reload has always worked first time, but there'= s definitely
>> > something fishy in there. =A0I don't recall exactly when = this started
>> > happening, maybe a month or so back; it hasn't been there= all along.
>> >
>> > Anybody else see this, or have an idea what broke?
>>
>> Yes, I see this also. =A0I reported it to Magnus, who muttered som= ething
>> about Varnish possibly misbehaving, but that was the end of it. >>
>
> He told me it was intentional rate limiting, applied because gitweb is= =A0so inefficient.

Correct. And varnish is not involved at all. I think alvaro = is confusing it with me saying that varnish might be a way for us to actual= ly fix the problem.

The problem is basically that gitweb is a big perl cgi (yes,= cgi only) which means that for every request it had to reload all of perl = and reparse the whole script. And on top of that the actual implementation = of several features is fairly slow in itself.

This caused easy overload - for example the whole site went = down when a search engine spidered us...

The bandaid solution was to rate limit access per ip. The er= ror you hit is when you're rate limited.

Now, I was supposed to have on my to do list to make a bette= r fix. But as this one was implemented a long time ago (measured in years, = really, I have to admit I forgot... I'll put bit back on the list.

/Magnus

--f46d0444edff51a2bf04d5a7a468--