Subject: Re: Problem with mirrorring

From: "Dave Page" <[email-masked]>
Date: Sat, 2 Oct 2004 20:46:07 +0100

> -----Original Message-----
> From: Devrim GUNDUZ [mailto:[email-masked]] 
> Sent: 02 October 2004 20:21
> To: Dave Page
> Cc: PostgreSQL WWW Mailing List
> Subject: Re: [pgsql-www] Problem with mirrorring
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> Merhaba, (heh it means 'hello' in Turkish)
> 
> On Sat, 2 Oct 2004, Dave Page wrote:
> 
> > There was a lockfile from this morning, but it won't stop you 
> > mirroring
> > - you actually mirror from a completely different box. The 
> lockfile is 
> > only on the master site and is used to stop lot's of site builds 
> > running at once if for some reason the database server 
> slows right down.
> >
> > Do you get any errors when you rsync?
> 
> :( Sorry to bother you but what I meant was the problem that 
> has happened
> before:
> 
> http://archives.postgresql.org/pgsql-www/2004-09/msg00149.php
> and your reply:
> http://archives.postgresql.org/pgsql-www/2004-09/msg00152.php
> 
> There was not an 'error' exactly...

Ahh, yes. Of course, that's actually rsync working correctly! No changes
on the server, so no transfers.

This does mean that our mirroring is quite innefficient. We really
should diff the results of each build and only update the html files if
there is a change. At the moment each mirror is probably pulling the
whole site each time :-(

Regards, Dave.

---

From: Devrim GUNDUZ <[email-masked]>
Date: Sat, 2 Oct 2004 22:59:46 +0300 (EEST)

Hi,

> on the server, so no transfers.

Umm, every cron job outputs an email of 259K, but when there is a 
lockfile, then it decreases to 800 bytes. Rsync gets all the files, even 
if there is no change!

> there is a change. At the moment each mirror is probably pulling the

Agreed but... diffing all the files that we have might be quite hard for 
us -- But I now don't have a better idea...

Umm... If we can keep a track of modified files in a db (a new or updated 
FAQ, doc, news, etc...) maybe we could publish only them...

Regards,

- --
Devrim GUNDUZ 
devrim~gunduz.org				devrim.gunduz~linux.org.tr
 			http://www.tdmsoft.com
 			http://www.gunduz.org

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sat, 2 Oct 2004 18:39:35 -0300 (ADT)

my understanding of rsync is that even if a file changes, only the changes 
are sent across ... I believe its effectively a 'diff' of the file ... so 
even if all files timestamps do change, very little data is sent across 
...

But, aren't there args available for rsync to have it check file size vs 
timestamp?  so it would miss a reversing of characters in a word, but if a 
single character is added, then the byte size changes and, therefore, it 
gets updated ... ?

On Sat, 2 Oct 2004, Devrim GUNDUZ wrote:

> Umm, every cron job outputs an email of 259K, but when there is a lockfile, 
> then it decreases to 800 bytes. Rsync gets all the files, even if there is no 
> change!
>
> Agreed but... diffing all the files that we have might be quite hard for us 
> -- But I now don't have a better idea...
>
> Devrim GUNDUZ devrim~gunduz.org 
> devrim.gunduz~linux.org.tr
> 			http://www.tdmsoft.com
> 			http://www.gunduz.org
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.1 (GNU/Linux)
>
> iD8DBQFBXwi2tl86P3SPfQ4RAm2QAKC8muqKCQbalZp1OK+T6uEMRTYtygCfeiqe
> pTOc+oXlqx0ju1R+PpTfXB4=
> =8ZdP
> -----END PGP SIGNATURE-----
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
>              http://www.postgresql.org/docs/faqs/FAQ.html
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email-masked]           Yahoo!: yscrappy              ICQ: 7615664