Date: Mon, 1 Nov 2004 00:22:41 +0300 (MSK)
From: Oleg Bartunov <oleg@sai.msu.su>
To: "Marc G. Fournier" <scrappy@postgresql.org>
Cc: Alexey Borzov <borz_off@cs.msu.su>, pgsql-www@postgresql.org
Subject: Re: Inadequate hosting for www.postgresql.org?
In-Reply-To: <20041031171023.I33702@ganymede.hub.org>
Message-ID: <Pine.GSO.4.61.0411010014040.24492@ra.sai.msu.su>
References: <4184C8EE.3080304@cs.msu.su>
	<20041031130007.R33702@ganymede.hub.org>
	<41854A72.2060608@cs.msu.su> <20041031171023.I33702@ganymede.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

Marc,

I think main problem is available memory, not cpu.
Do you have single-server setup for www.postgresql.org ?
I see from headers 
zen:~/downloads$>curl -I http://www.postgresql.org/
HTTP/1.1 200 OK
Date: Sun, 31 Oct 2004 21:13:50 GMT
Server: Apache/1.3.31 (Unix) PHP/4.3.9
Last-Modified: Sun, 31 Oct 2004 21:00:07 GMT

Do you have persistent connection from php to database ?
What about cacheing of dynamic pages ?

Also, I recommend webperf.org  to see how fast selected page is loaded
http://www.webperf.org/breakdown.html?URL=www.postgresql.org
Connect DNS 	FirstByte End 	Size(Bytes) 	URL 			Status
0.033 	0.071 	0.383 	1.073 	21429 	http://www.postgresql.org 	200
0.412 	0.412 	1.161 	1.270 	4447 	http://www.postgresql.org/images/banner-left.jpg 	200
0.532 	0.532 	1.152 	1.254 	2929 	http://www.postgresql.org/images/banner-right.jpg 	200
0.650 	0.650 	1.143 	1.146 	929 	http://www.postgresql.org/postgresql.css 	200
0.768 	0.793 	1.207 	1.418 	8616 	http://ads.postgresql.org/display_image.php?lvl=sponsor 	200
0.905 	0.905 	1.156 	1.277 	7465 	http://ads.postgresql.org/display_image.php?lvl=free 	200


 	Oleg
On Sun, 31 Oct 2004, Marc G. Fournier wrote:

>
> I just email'd Alexey to test after making some changes (moving the whole VM 
> to a faster server) and reporting much better response times ...
>
> On Sun, 31 Oct 2004, Alexey Borzov wrote:
>
>> Hi,
>> 
>> Marc G. Fournier wrote:
>>> That's why mirrors are pulled from http://www.postgresql.org, nor are they 
>>> pulled using what I'd guess you are using ( wget? ) below, but using rsync 
>>> ... hasn't been for years now ...
>>> 
>>> In fact, over the next little while, http://www.postgresql.org will 
>>> actually be pointing to an RR DNS entry, and not even the 'base server' 
>>> ... I've just gotta sit down and update the DNS generation scripts for 
>>> this ...
>> 
>> Uh, seems I was unclear about what I was doing.
>> 
>> I understand of course that the site is mirrored via rsync, but consider 
>> the following: the site which resides in CVS is just a few PHP scripts, to 
>> have something to rsync you first need to *generate* the static pages. And 
>> here comes the problem: page generation is outrageously slow.
>> 
>> In current version special scripts are used to generate static pages, but 
>> these are terrible hacks, f.e. you have to make the same change to design 
>> both in the dynamic page and the mirroring script. But they maybe run in 
>> acceptable time, I don't know, ask Dave.
>> 
>> For the next version I just wrote a generic mirroring script behaving like 
>> wget but taking into account some specific needs (like i18n). The plus is 
>> that you don't need to change this script *at all* if you add pages or even 
>> completely redesign the site.
>> 
>> But now it looks as if generation of full static website will take several 
>> hours. And no, this has nothing to do with network latency, the mirror 
>> generation script was run locally.
>> 
>> Marc, page generation times of 3+ seconds are only acceptable if you do 
>> some extremely complex database queries in them, think reports. This is 
>> definitely *not* the case with the code of postgresql.org, my experience 
>> tells me that modern hardware can happily churn 5-10 of such pages *per 
>> second*. Thus: either hardware is completely inadequate or the servers are 
>> overloaded.
>> 
>
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83