Subject: Inadequate hosting for www.postgresql.org?

From: Alexey Borzov <[email-masked]>
Date: Sun, 31 Oct 2004 14:13:50 +0300

Hi,

I'm testing the mirroring script for new version of postgresql.org and the 
results are *interesting*:

Oct 31 10:11:32 mirror [info] Mirroring finished. 423 page(s) saved, 1226 
second(s) spent

~3 seconds per page is a bit too much, don't you think? Also please note that it 
took ~20 minutes to save the pages *not including* the documentation, which is 
*several thousand* pages in size!

Here are the results for mirroring of just 7.4 "interactive" docs:

Oct 31 11:12:46 mirror [info] Mirroring finished. 603 page(s) saved, 2716 
second(s) spent

That's >4 seconds per page...

These are the pages from wwwdevel, but as www and wwwdevel are on one box 
physically, you get the unpleasant picture. It actually worked faster on my test 
server and that is P2-233. Also please note that today is Sunday, I doubt it 
will be faster on weekday.

Dunno about you, but if that was my site I'd already be searching for another 
hosting. I'm sure that some hosters will be glad to even donate that to 
PostgreSQL project.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Sun, 31 Oct 2004 07:37:41 -0800

>
> Dunno about you, but if that was my site I'd already be searching for 
> another hosting. I'm sure that some hosters will be glad to even 
> donate that to PostgreSQL project.

This has been brought multiple times in the past. Command Prompt and 
Gavin both have offered hosting resources. Command Prompt already
hosts the archives. I know Gavin does something but I can't recall what 
(sorry Gavin ;).

The resources are available but it is up to the project to request them.

Sincerely,

Joshua D. Drake

>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
>               http://archives.postgresql.org

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - [email-masked] - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 31 Oct 2004 13:04:23 -0400 (AST)

That's why mirrors are pulled from http://www.postgresql.org, nor are they 
pulled using what I'd guess you are using ( wget? ) below, but using rsync 
... hasn't been for years now ...

In fact, over the next little while, http://www.postgresql.org will 
actually be pointing to an RR DNS entry, and not even the 'base server' 
... I've just gotta sit down and update the DNS generation scripts for 
this ...

On Sun, 31 Oct 2004, Alexey Borzov wrote:

> ~3 seconds per page is a bit too much, don't you think? Also please note that 
> it took ~20 minutes to save the pages *not including* the documentation, 
> which is *several thousand* pages in size!
>
> physically, you get the unpleasant picture. It actually worked faster on my 
> test server and that is P2-233. Also please note that today is Sunday, I 
> doubt it will be faster on weekday.
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email-masked]           Yahoo!: yscrappy              ICQ: 7615664

---

From: Alexey Borzov <[email-masked]>
Date: Sun, 31 Oct 2004 23:26:26 +0300

Hi,

Marc G. Fournier wrote:
> That's why mirrors are pulled from http://www.postgresql.org, nor are 
> they pulled using what I'd guess you are using ( wget? ) below, but 
> using rsync ... hasn't been for years now ...
> 

Uh, seems I was unclear about what I was doing.

I understand of course that the site is mirrored via rsync, but consider the 
following: the site which resides in CVS is just a few PHP scripts, to have 
something to rsync you first need to *generate* the static pages. And here comes 
the problem: page generation is outrageously slow.

In current version special scripts are used to generate static pages, but these 
are terrible hacks, f.e. you have to make the same change to design both in the 
dynamic page and the mirroring script. But they maybe run in acceptable time, I 
don't know, ask Dave.

For the next version I just wrote a generic mirroring script behaving like wget 
but taking into account some specific needs (like i18n). The plus is that you 
don't need to change this script *at all* if you add pages or even completely 
redesign the site.

But now it looks as if generation of full static website will take several 
hours. And no, this has nothing to do with network latency, the mirror 
generation script was run locally.

Marc, page generation times of 3+ seconds are only acceptable if you do some 
extremely complex database queries in them, think reports. This is definitely 
*not* the case with the code of postgresql.org, my experience tells me that 
modern hardware can happily churn 5-10 of such pages *per second*. Thus: either 
hardware is completely inadequate or the servers are overloaded.

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 31 Oct 2004 17:12:07 -0400 (AST)

I just email'd Alexey to test after making some changes (moving the whole 
VM to a faster server) and reporting much better response times ...

On Sun, 31 Oct 2004, Alexey Borzov wrote:

>> actually be pointing to an RR DNS entry, and not even the 'base server' ... 
>> I've just gotta sit down and update the DNS generation scripts for this ...
>
> something to rsync you first need to *generate* the static pages. And here 
> comes the problem: page generation is outrageously slow.
>
> In current version special scripts are used to generate static pages, but 
> these are terrible hacks, f.e. you have to make the same change to design 
> both in the dynamic page and the mirroring script. But they maybe run in 
> acceptable time, I don't know, ask Dave.
>
> For the next version I just wrote a generic mirroring script behaving like 
> wget but taking into account some specific needs (like i18n). The plus is 
> that you don't need to change this script *at all* if you add pages or even 
> completely redesign the site.
>
> modern hardware can happily churn 5-10 of such pages *per second*. Thus: 
> either hardware is completely inadequate or the servers are overloaded.
>

----

---

From: Oleg Bartunov <[email-masked]>
Date: Mon, 1 Nov 2004 00:22:41 +0300 (MSK)

Marc,

I think main problem is available memory, not cpu.
Do you have single-server setup for www.postgresql.org ?
I see from headers 
zen:~/downloads$>curl -I http://www.postgresql.org/
HTTP/1.1 200 OK
Date: Sun, 31 Oct 2004 21:13:50 GMT
Server: Apache/1.3.31 (Unix) PHP/4.3.9
Last-Modified: Sun, 31 Oct 2004 21:00:07 GMT

Do you have persistent connection from php to database ?
What about cacheing of dynamic pages ?

Also, I recommend webperf.org  to see how fast selected page is loaded
http://www.webperf.org/breakdown.html?URL=www.postgresql.org
Connect DNS 	FirstByte End 	Size(Bytes) 	URL 			Status
0.033 	0.071 	0.383 	1.073 	21429 	http://www.postgresql.org 	200
0.412 	0.412 	1.161 	1.270 	4447 	http://www.postgresql.org/images/banner-left.jpg 	200
0.532 	0.532 	1.152 	1.254 	2929 	http://www.postgresql.org/images/banner-right.jpg 	200
0.650 	0.650 	1.143 	1.146 	929 	http://www.postgresql.org/postgresql.css 	200
0.768 	0.793 	1.207 	1.418 	8616 	http://ads.postgresql.org/display_image.php?lvl=sponsor 	200
0.905 	0.905 	1.156 	1.277 	7465 	http://ads.postgresql.org/display_image.php?lvl=free 	200

 	Oleg
On Sun, 31 Oct 2004, Marc G. Fournier wrote:

>
> I just email'd Alexey to test after making some changes (moving the whole VM 
> to a faster server) and reporting much better response times ...
>
>> I understand of course that the site is mirrored via rsync, but consider 
>> the following: the site which resides in CVS is just a few PHP scripts, to 
>> have something to rsync you first need to *generate* the static pages. And 
>> here comes the problem: page generation is outrageously slow.
>> 
>> Marc, page generation times of 3+ seconds are only acceptable if you do 
>> some extremely complex database queries in them, think reports. This is 
>> definitely *not* the case with the code of postgresql.org, my experience 
>> tells me that modern hardware can happily churn 5-10 of such pages *per 
>> second*. Thus: either hardware is completely inadequate or the servers are 
>> overloaded.
>> 
>
> ----
> TIP 8: explain analyze is your friend
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: [email-masked], http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

---

From: Alexey Borzov <[email-masked]>
Date: Mon, 01 Nov 2004 00:32:08 +0300

Hi,

Marc G. Fournier wrote:
> I just email'd Alexey to test after making some changes (moving the 
> whole VM to a faster server) and reporting much better response times ...

Yes, now it does ~5 pages per second, thus the whole mirroring process will 
probably complete in under an hour.

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 31 Oct 2004 17:39:14 -0400 (AST)

> probably complete in under an hour.

Does the mirroring process mirror *all* files, or just those that have 
changed?  Most of the files on the site *should* be static, no?  What 
consititutes the bulk of the pages?  I would think the various versions 
docs, no?  Which are static ...

BTW, does this script you are working on have any access controls, or does 
it allow *everyone* to mirror the site?

----

---

From: Alexey Borzov <[email-masked]>
Date: Mon, 01 Nov 2004 01:04:29 +0300

Hi,

Marc G. Fournier wrote:
>> Yes, now it does ~5 pages per second, thus the whole mirroring process 
>> will probably complete in under an hour.
> 

All files except docs, since there are dynamic parts like survey and 
particularly gborg stats on all pages. But there are not very many pages here.

As for the docs, there is version without user comments which will not need to 
be mirrored unless the design changes or the docs for the newer version replace 
the older docs. The version with user comments will need to be mirrored every 
time, obviously. Though we may consider removing comment form from docs for the 
older versions and ease this requirement.

Of course one can add Last-Modified header and 304 response to the pages, but 
there still will be a request to make.

> BTW, does this script you are working on have any access controls, or 
> does it allow *everyone* to mirror the site?

If you mean whether anyone will be able to access the script via web then the 
answer is "no", it will be run from console. If the question is different then I 
don't get it.

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 31 Oct 2004 21:04:07 -0400 (AST)

On Mon, 1 Nov 2004, Alexey Borzov wrote:

> If you mean whether anyone will be able to access the script via web 
> then the answer is "no", it will be run from console. If the question is 
> different then I don't get it.

'k, then I'm confused ... what exactly does this script do?  or, rather, 
what is it replacing?  building the static files for mirroring purposes?

----

---

From: Alexey Borzov <[email-masked]>
Date: Mon, 01 Nov 2004 11:09:11 +0300

Hi,

Marc G. Fournier wrote:
>> then the answer is "no", it will be run from console. If the question 
>> is different then I don't get it.
> 

Exactly. Sorry for confusing you. :)

---

From: Alexey Borzov <[email-masked]>
Date: Mon, 01 Nov 2004 15:20:18 +0300

Hi,

> will probably complete in under an hour.

Hmm, looks like I was too fast to celebrate:

Nov 01 11:27:51 mirror [info] Mirroring finished. 423 page(s) saved, 
2407 second(s) spent

Something is still very wrong with wwwdevel

---

From: "Marc G. Fournier" <[email-masked]>
Date: Mon, 1 Nov 2004 11:10:29 -0400 (AST)

Am looking into this with Alexey offlist ...

On Mon, 1 Nov 2004, Alexey Borzov wrote:

> Nov 01 11:27:51 mirror [info] Mirroring finished. 423 page(s) saved, 2407 
> second(s) spent
>

----

---

From: Alexey Borzov <[email-masked]>
Date: Tue, 02 Nov 2004 13:25:22 +0300

Hi,

> Am looking into this with Alexey offlist ...

Looking into this stuff offlist highlighted the following obvious 
problem: postgresql.org (as well as wwwdevel) runs on a shared server 
that has *lots* of other tasks that can negatively affect performance. 
It is difficult to even point finger at one particular task that does this.

So the solution (as was also pointed out in a private mail to me) is to 
get a dedicated server for running the "master" website.

I am not in the position to request such a donation, but PostgreSQL core 
and/or community is.