Date: Sun, 31 Oct 2004 17:12:07 -0400 (AST)
From: "Marc G. Fournier" <scrappy@postgresql.org>
To: Alexey Borzov <borz_off@cs.msu.su>
Cc: pgsql-www@postgresql.org
Subject: Re: Inadequate hosting for www.postgresql.org?
In-Reply-To: <41854A72.2060608@cs.msu.su>
Message-ID: <20041031171023.I33702@ganymede.hub.org>
References: <4184C8EE.3080304@cs.msu.su>
	<20041031130007.R33702@ganymede.hub.org>
	<41854A72.2060608@cs.msu.su>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed


I just email'd Alexey to test after making some changes (moving the whole 
VM to a faster server) and reporting much better response times ...

On Sun, 31 Oct 2004, Alexey Borzov wrote:

> Hi,
>
> Marc G. Fournier wrote:
>> That's why mirrors are pulled from http://www.postgresql.org, nor are they 
>> pulled using what I'd guess you are using ( wget? ) below, but using rsync 
>> ... hasn't been for years now ...
>> 
>> In fact, over the next little while, http://www.postgresql.org will 
>> actually be pointing to an RR DNS entry, and not even the 'base server' ... 
>> I've just gotta sit down and update the DNS generation scripts for this ...
>
> Uh, seems I was unclear about what I was doing.
>
> I understand of course that the site is mirrored via rsync, but consider the 
> following: the site which resides in CVS is just a few PHP scripts, to have 
> something to rsync you first need to *generate* the static pages. And here 
> comes the problem: page generation is outrageously slow.
>
> In current version special scripts are used to generate static pages, but 
> these are terrible hacks, f.e. you have to make the same change to design 
> both in the dynamic page and the mirroring script. But they maybe run in 
> acceptable time, I don't know, ask Dave.
>
> For the next version I just wrote a generic mirroring script behaving like 
> wget but taking into account some specific needs (like i18n). The plus is 
> that you don't need to change this script *at all* if you add pages or even 
> completely redesign the site.
>
> But now it looks as if generation of full static website will take several 
> hours. And no, this has nothing to do with network latency, the mirror 
> generation script was run locally.
>
> Marc, page generation times of 3+ seconds are only acceptable if you do some 
> extremely complex database queries in them, think reports. This is definitely 
> *not* the case with the code of postgresql.org, my experience tells me that 
> modern hardware can happily churn 5-10 of such pages *per second*. Thus: 
> either hardware is completely inadequate or the servers are overloaded.
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664