public inbox for [email protected]
help / color / mirror / Atom feedFrom: Magnus Hagander <[email protected]>
To: Andres Freund <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: Kevin Grittner <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: PostgreSQL WWW <[email protected]>
Cc: Stephen Frost <[email protected]>
Subject: Re: gitweb is no longer a real-time view
Date: Mon, 4 Mar 2013 14:19:27 +0100
Message-ID: <CABUevEwH3ju-kFimY6En+ujPOMMn5gXE0OcoCWC-vWCATZ8LAw@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CABUevEwfw24iF9-czueLSmdFQwck012qfVmLXcoKbG37xG56jQ@mail.gmail.com>
<CABUevEzM2F8qAxVds9Me-41L5BqeEGO4DuosCQhC0C6v05HPUw@mail.gmail.com>
<[email protected]>
List-Unsubscribe: <mailto:[email protected]?body=unsub%20pgsql-www>
On Mon, Mar 4, 2013 at 1:50 PM, Andres Freund <[email protected]> wrote:
> On 2013-03-04 08:32:35 +0100, Magnus Hagander wrote:
>> On Mon, Mar 4, 2013 at 8:16 AM, Magnus Hagander <[email protected]> wrote:
>> > On Mon, Mar 4, 2013 at 3:33 AM, Stephen Frost <[email protected]> wrote:
>> >> * Tom Lane ([email protected]) wrote:
>> >>> cron job? I was under the impression there was some sort of push
>> >>> operation driven by a commit trigger. The web site has certainly
>> >>> updated nearly immediately for as long as we've been using git.
>> >>> Until this week, that is.
>> >>
>> >> Curiously, there's two cron jobs, apparently. There's a 'push' one and
>> >> then another, independent, 'pull' one. I'll assume they're actually
>> >> doing different things, but I wonder if the pull isn't just a
>> >> hold-over.. In any case, the push-to-anon, which I hadn't seen
>> >
>> > No, the pull one doesn't do the main repository - that one pulls
>> > "third party" repositories in as mirrors, such as e.g. Bucardo. That
>> > one was, btw, broken for several months and nobody noticed - so that's
>> > clearly a less important one :)
>> >
>> >> initially (looking at the pull side instead of the push side), does run
>> >> once a minute, though it looks like there's a hook mechanism which
>> >> would allow us to trigger the webserver to do a pull when a commit
>> >> happens and would still be better than a once-a-minute cronjob.
>> >
>> > The one that deals with the main repo mirroring runs once per minute.
>> > The commit trigger only drops a file in the filesystem so the cronjob
>> > knows there is something to do. So that we don't risk holding up the
>> > actual commit in case there is a problem with the anonymous mirror
>> > server.
>> >
>> >
>> >> Even with that, however, the concern raised was that the gitweb perl
>> >> script is quite expensive to run for every request, hence the reason for
>> >> doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and
>> >> 15m grace and I'll keep an eye on it.
>> >
>> > That's fixing the wrong problem. The cache is supposed to be
>> > automatically purged whenever the push happens, to make sure that the
>> > data never gets more than maybe a second stale. So clearly something
>> > in that is not working - I didn't specifically test it with the
>> > postgres.git repository, but I tested it with a couple of other ones
>> > where it worked fine. But maybe we did something special with the main
>> > one...
>>
>> Actually, looking closer, I'm seeing a failure when it actually tries the push:
>>
>> To ssh://[email protected]/postgresql.git
>> ! [rejected] master -> master (non-fast-forward)
>>
>>
>> So the problem might have nothing at all to do with the cacheing.
>>
>> AFAICT, the three missing commits are materialized views,
>> accidentally committed .orig file and \l support.
>>
>> But. The *anonymous* repository also has:
>> bc61878682051678ade5f59da7bfd90ab72ce13b Fix
>> map_sql_value_to_xml_value() to treat domains like their base types.
>>
>> This patch is *not* in the master repository, it's only in anonymous.
>> (The object is in the repository, but it's not part of any branch)
>>
>> How the hell did *that* happen?
>>
>> The master repo has:
>> commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28
>> Author: Peter Eisentraut <[email protected]>
>>
>> psql: Let \l accept a pattern
>>
>> commit d63977eea3ab18fdec05e370b633d10b9fd20179
>> Author: Kevin Grittner <[email protected]>
>>
>> Remove accidentally-committed .orig file.
>>
>> commit 3bf3ab8c563699138be02f9dc305b7b77a724307
>> Author: Kevin Grittner <[email protected]>
>>
>> Add a materialized view relations.
>>
>> commit b15a6da29217b14f02895af1d9271e84415a91ae
>> Author: Tom Lane <[email protected]>
>>
>> Get rid of any toast table when converting a table to a view.
>>
>>
>> And the anonymous one has:
>> commit bc61878682051678ade5f59da7bfd90ab72ce13b
>> Author: Tom Lane <[email protected]>
>>
>> Fix map_sql_value_to_xml_value() to treat domains like their base types.
>>
>> commit 3bf3ab8c563699138be02f9dc305b7b77a724307
>> Author: Kevin Grittner <[email protected]>
>>
>> Add a materialized view relations.
>>
>> commit b15a6da29217b14f02895af1d9271e84415a91ae
>> Author: Tom Lane <[email protected]>
>>
>> Get rid of any toast table when converting a table to a view.
>>
>>
>>
>> Does anybody have an explanation for that? Did someone do a force-push
>> on the master repository, overwriting some old history?
>
> This really looks like Kevin forced a push.
>
> Could you show
> git show --pretty=raw d63977eea3ab18fdec05e370b633d10b9fd20179
> git show --pretty=raw bc61878682051678ade5f59da7bfd90ab72ce13b
> git show --pretty=raw 5db5974c692b0fc68e7608dd85a6b4e6173a0f28
In which of the repositories - master, anonymous, or both?
> ?
> I don't have all the commits here...
>
> The raw format will explicitly tell which parents are referenced in the
> commits...
>
>> We don't explicitly forbid this on the master repo, since we expect
>> committers to know how things work.. Maybe we need to do that, and
>> manually turn it off in case someone actually *needs* to do a non fast
>> forward push? But either way, it would be good to actually know how
>> tihs happened...
>
> If you allow it the mirroring script should always force a push,
> otherwise it will stop pushing in those cases.
Yeah. But we shouldn't do it, so disallowing it seems like a good thing.
> I think the most realistic way to resolve this is to push the current
> anongit state to master and cherry-pick
> bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only
> committers need to deal with rebasing their branches and not everyone
> else.
Yeah, I agree that's probably the best way to do it - the number of
committers is a lot more limited...
Anyone else have an opinion on that part?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-www mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-www
view thread (43+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: gitweb is no longer a real-time view
In-Reply-To: <CABUevEwH3ju-kFimY6En+ujPOMMn5gXE0OcoCWC-vWCATZ8LAw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox