Subject: Re: Inadequate hosting for www.postgresql.org

From: "Dave Page" <[email-masked]>
Date: Wed, 3 Nov 2004 17:17:51 -0000

> -----Original Message-----
> From: Mitch Pirtle [mailto:[email-masked]] 
> Sent: 03 November 2004 16:58
> To: Alexey Borzov
> Cc: Oleg Bartunov; Marc G. Fournier; Dave Page; 
> [email-masked]
> Subject: Re: [pgsql-www] Inadequate hosting for www.postgresql.org
> 
> And several people have offered over the last few months, 
> including myself.  I am thinking about a dual xeon with 1 GB 
> RAM that has a load average of 0.01...  Nightly backups too, 
> with disk a non-factor.  I asked about bandwidth usage 
> estimates but never got that far :-(
> 
> This I simply do not understand.  If there is a need for 
> dedicated hardware, and it is being offered, then why is it 
> not being accepted? 
> Didn't CommandPrompt also offer dedicated hardware?
> 
> Not wanting to start another heated debate, just wanting to 
> understand what the deal is - as this topic keeps getting 
> regurgitated without a conclusion.

The problem is not the hardware from what Marc & I can tell - the
current hardware (a dual PIII with 4GB) is just ticking along with
little load, and the current website performs quite nicely. What we seem
to have is a php performance issue that we haven't tracked down yet
which only noticably affects the beta site. Yes, we can throw more
hardware at it if it really comes down to it (I can also rustle
something pretty nippy up), but it makes more sense to figure out why
we're having this problem in the first place.

Your offer is noted and appreciated though :-)

Regards, Dave

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 3 Nov 2004 13:31:23 -0400 (AST)

> we're having this problem in the first place.

Just to add to this ... if you go to 
http://www.postgresql.org/index-test.php, at the bottom there are 
'timings' that I've added to the code ... my last load looked like (with 
mmcache enabled):

loading globals.php: 0.135524
loading loading news : 0.015047
loading loading events: 0.013862
total load time: 0.165179

A total of .16sec to load that page ... but, if you notice when you go 
that page, it 'hangs' on start up for a few seconds ...

turning off mmcache, the numbers get even better:
loading globals.php: 0.082831
loading loading news : 0.020089
loading loading events: 0.008389
total load time: 0.111649

a hang that isn't apparent if you go to http://www.postgresql.org (a 
static page) ...

Alexey's site uses *alot* more PHP then the current page does, so it shows 
up the issue alot more ...

Now, to any PHP gurus that might be out there ... FreeBSD ports recently 
went to an 'extensions' format for php ... so, you build apache, you add 
in mod_php4, and then for the various extensions you want, you have a 
/usr/local/etc/php/extensions.ini file that you enable/disable in ... when 
Apache forks off a new Child process, is it loading all of those up again, 
each time?  Or, even better, does it only load up the PHP stuff when a 
pages calls for it?  ie. a .html wouldn't load in the mod_php, but a .php 
would have to load it all up before it could run?  Would that load time be 
the 'lag' we're seeing?

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email-masked]           Yahoo!: yscrappy              ICQ: 7615664

---

From: "Joshua D. Drake" <[email-masked]>
Date: Wed, 03 Nov 2004 09:52:00 -0800

> which only noticably affects the beta site. 

Have you considered using a PHP optimizer such as Tucker?

Yes, we can throw more
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
> 
>                http://archives.postgresql.org

-- 
Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - [email-masked] - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 3 Nov 2004 14:01:17 -0400 (AST)

> Have you considered using a PHP optimizer such as Tucker?

I've tried using truck-mmcache, if that is what you mean ... it made no 
noticeable difference from what I could tell ... and, after starting to 
play wtih the current site, it doesn't look like a 'processing' issue ... 
I'm going to work this afternoon on building a 'static apache/php' server 
... I don't know how apache is forking its children, but the way that the 
FreeBSD ports went with PHP, I'm wondering if the problem is application 
loading, since it only seems to affect mod_php that I can tell, not static 
files :(

----

---

From: Alexey Borzov <[email-masked]>
Date: Wed, 03 Nov 2004 23:37:08 +0300

Hi,

Dave Page wrote:
> little load, and the current website performs quite nicely. 

You mean, of course, "current *static* website performs quite nicely", since PHP 
is horribly slow there as well.

I'd like to repeat, the problem is that the server is not dedicated to 
PostgreSQL project's needs and it has a lot of other stuff that can affect 
performance and is extremely hard to track:

> What we seem

---

From: Mitch Pirtle <[email-masked]>
Date: Wed, 3 Nov 2004 15:52:12 -0500

> would have to load it all up before it could run?  Would that load time be

Not with turck-mmcache going.  Turck loads every PHP script once,
parses and compiles it to opcode, and then waits for requests, so I
cannot see how PHP scripts could incur a significant overhead if they
were really being cached.  I would suggest two things:

1) looking at apache's minimum number of processes (less forking after
a restart)
2) seeing if mmcache is really configured properly - this can be done
by copying mmcache.php from the mmcache distribution files to
somewhere in your document root, and loading it in your browser.  This
will show you how many files are cached, how many hits, and how many
reloads.

I'm not a BSD guy so that is a pretty limiting factor for me helping,
but I can tell you that a dual XEON with mmcache enabled and only 1GB
of memory can serve 1.5M page views daily without strain.  I inherited
a very poorly written PHP app (more like a hodgepodge of
loosely-cobbled scripts) and even with lousy design it scales to over
a million views daily without effort.

-- Mitch

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 3 Nov 2004 17:00:35 -0400 (AST)

> 1) looking at apache's minimum number of processes (less forking after

running 15/30 for min/max right now ...

> will show you how many files are cached, how many hits, and how many

check out http://www.postgresql.org/mmc/mmcache.php ... also, 
http://www.postgresql.org/phpinfo.php ... if you can suggest any changes, 
please feel free to do so ...

----

---

From: Mitch Pirtle <[email-masked]>
Date: Wed, 3 Nov 2004 16:15:52 -0500

Hmm,

Are you certain disk is an issue?

Regarding mmcache:

1) I'm setting mmcache.content, mmcache.keys and mmcache.sessions to 'disk'
2) mmcache.compress is set to 1, 1 on my machines
3) mmcache.shm_only is 0, 0 on my machines

Try this out, and see if it is the shm that is causing the hit.  I
would be completely blown away if the filesystem access was an issue,
as even IDE systems can access disk faster than an anatomic
load/parse/compile for each script...

Try that, and we can go from there ;-)

-- Mitch

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 3 Nov 2004 17:41:58 -0400 (AST)

Changes done, server restarted ... try it and see if it helps any ...

----

---

From: Mitch Pirtle <[email-masked]>
Date: Wed, 3 Nov 2004 16:59:09 -0500

Hmm, seems to shave a second off of the results posted by Oleg.  Progress?

http://www.postgresql.org/index-test.php now reports a bunch of
pg_result errors, so I cannot comment there.

The thing that has my attention is the overall time reported by
Webperf for www.alexey.beta.postgresql.org.  I did the same tests on
www.spacemonkeylabs.com, and  receive timings 1/3rd, and this is a
stock Mambo 4/5/1a with mmcache disabled (something provided by Plesk7
conflicts so I cannot load it on this machine).  FWIW Mambo is all
OO-based PHP, with several external libraries and 5 different RSS
exports on the home page (something like 36 queries, the horror).