public inbox for [email protected]
help / color / mirror / Atom feedFrom: Dave Page <[email protected]>
To: Devrim GUNDUZ <[email protected]>
Cc: PostgreSQL WWW Mailing List <[email protected]>
Subject: Re: Problem with mirrorring
Date: Sat, 2 Oct 2004 21:24:12 +0100
Message-ID: <E7F85A1B5FF8D44C8A1AF6885BC9A0E4306887@ratbert.vale-housing.co.uk> (raw)
> -----Original Message-----
> From: Devrim GUNDUZ [mailto:[email protected]]
> Sent: 02 October 2004 21:00
> To: Dave Page
> Cc: PostgreSQL WWW Mailing List
> Subject: Re: [pgsql-www] Problem with mirrorring
>
> >> There was not an 'error' exactly...
> >
> > Ahh, yes. Of course, that's actually rsync working correctly! No
> > changes on the server, so no transfers.
>
> Umm, every cron job outputs an email of 259K, but when there
> is a lockfile, then it decreases to 800 bytes. Rsync gets all
> the files, even if there is no change!
It shouldn't do. Andrew Tridgell (sp?) of the Samba team wrote it for
his Phd thesis, designing it specifically to minimise the data
transferred. Iirc, it only even transfers the differences between files
rather than the whole thing. It's pretty clever stuff.
Anyway (I'll stop babbling now :-) ), if it's transferring everything,
even if it hasn't even been touched at out end, then something,
somewhere is broken.
> > This does mean that our mirroring is quite innefficient. We really
> > should diff the results of each build and only update the
> html files
> > if there is a change. At the moment each mirror is probably pulling
> > the whole site each time :-(
>
> Agreed but... diffing all the files that we have might be
> quite hard for us -- But I now don't have a better idea...
>
> Umm... If we can keep a track of modified files in a db (a
> new or updated FAQ, doc, news, etc...) maybe we could publish
> only them...
It's more tricky than that, because files change even if a new user join
Gborg for example (because of the count in the left column of the
'portal' pages). Besides, thinking about it some more I don't think it
really is that much of a problem. If rsync does it's job as I remember
it should (bear in mind it's a while since I read the thesis, and it's
Saturday night!), then it should minimise the transfer even if the mod
time has been touched.
Regards, Dave.
view thread (12+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: Problem with mirrorring
In-Reply-To: <E7F85A1B5FF8D44C8A1AF6885BC9A0E4306887@ratbert.vale-housing.co.uk>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox