Message-ID: <42EA6966.8090402@aptalaska.net>
Date: Fri, 29 Jul 2005 09:37:42 -0800
From: Matthew Schumacher <matt.s@aptalaska.net>
User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050317)
MIME-Version: 1.0
To: Andrew McMillan <andrew@catalyst.net.nz>
Cc: pgsql-performance@postgresql.org
Subject: Re: Performance problems testing with Spamassassin 3.1.0
References: <42E80C28.2040608@aptalaska.net>	
	<1122532023.25574.1.camel@localhost.localdomain>	
	<42E9749F.6000709@aptalaska.net>
	<1122616211.5691.340.camel@lamb.mcmillan.net.nz>
In-Reply-To: <1122616211.5691.340.camel@lamb.mcmillan.net.nz>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Andrew McMillan wrote:
> On Thu, 2005-07-28 at 16:13 -0800, Matthew Schumacher wrote:
> 
>>Ok, I finally got some test data together so that others can test
>>without installing SA.
>>
>>The schema and test dataset is over at
>>http://www.aptalaska.net/~matt.s/bayes/bayesBenchmark.tar.gz
>>
>>I have a pretty fast machine with a tuned postgres and it takes it about
>>2 minutes 30 seconds to load the test data.  Since the test data is the
>>bayes information on 616 spam messages than comes out to be about 250ms
>>per message.  While that is doable, it does add quite a bit of overhead
>>to the email system.
> 
> 
> On my laptop this takes:
> 
> real    1m33.758s
> user    0m4.285s
> sys     0m1.181s
> 
> One interesting effect is the data in bayes_vars has a huge number of
> updates and needs vacuum _frequently_.  After the run a vacuum full
> compacts it down from 461 pages to 1 page.
> 
> Regards,
> 					Andrew.
> 

I wonder why your laptop is so much faster.  My 2 min 30 sec test was
done on a dual xeon with a LSI megaraid with 128MB cache and writeback
caching turned on.

Here are my memory settings:

shared_buffers = 16384
work_mem = 32768
maintenance_work_mem = 65536

I tried higher values before I came back to these but it didn't help my
performance any.  I should also mention that this is a production
database server that was servicing other queries when I ran this test.

How often should this table be vacuumed, every 5 minutes?

Also, this test goes a bit faster with sync turned off, if mysql isn't
using sync that would be why it's so much faster.  Anyone know what the
default for mysql is?

Thanks,
schu