pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedpgbench results on a new server
6+ messages / 4 participants
[nested] [flat]
* pgbench results on a new server
@ 2010-06-25 18:53 Craig James <craig_james@emolecules.com>
0 siblings, 2 replies; 6+ messages in thread
From: Craig James @ 2010-06-25 18:53 UTC (permalink / raw)
To: pgsql-performance
I've got a new server and want to make sure it's running well. Are these pretty decent numbers?
8 cores (2x4 Intel Nehalem 2 GHz)
12 GB memory
12 x 7200 SATA 500 GB disks
3WARE 9650SE-12ML RAID controller with BBU
WAL on ext2, 2 disks: RAID1 500GB, blocksize=4096
Database on ext4, 8 disks: RAID10 2TB, stripe size 64K, blocksize=4096
Ubuntu 10.04 LTS (Lucid)
Postgres 8.4.4
pgbench -i -s 100 -U test
pgbench -c 5 -t 20000 -U test
tps = 4903
pgbench -c 10 -t 10000 -U test
tps = 4070
pgbench -c20 -t 5000 -U test
tps = 5789
pgbench -c30 -t 3333 -U test
tps = 6961
pgbench -c40 -t 2500 -U test
tps = 2945
Thanks,
Craig
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgbench results on a new server
@ 2010-06-25 19:01 Scott Marlowe <scott.marlowe@gmail.com>
parent: Craig James <craig_james@emolecules.com>
1 sibling, 0 replies; 6+ messages in thread
From: Scott Marlowe @ 2010-06-25 19:01 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
On Fri, Jun 25, 2010 at 2:53 PM, Craig James <craig_james@emolecules.com> wrote:
> I've got a new server and want to make sure it's running well. Are these
> pretty decent numbers?
>
> 8 cores (2x4 Intel Nehalem 2 GHz)
> 12 GB memory
> 12 x 7200 SATA 500 GB disks
> 3WARE 9650SE-12ML RAID controller with BBU
> WAL on ext2, 2 disks: RAID1 500GB, blocksize=4096
> Database on ext4, 8 disks: RAID10 2TB, stripe size 64K, blocksize=4096
> Ubuntu 10.04 LTS (Lucid)
> Postgres 8.4.4
>
> pgbench -i -s 100 -U test
> pgbench -c 5 -t 20000 -U test
> tps = 4903
> pgbench -c 10 -t 10000 -U test
> tps = 4070
> pgbench -c20 -t 5000 -U test
> tps = 5789
> pgbench -c30 -t 3333 -U test
> tps = 6961
> pgbench -c40 -t 2500 -U test
> tps = 2945
Numbers are okay, but you likely need much longer tests to see how
they average out with the bgwriter / checkpoints happening, and keep
track of your IO numbers to see where your dips are. I usually run
pgbench runs, once they seem to get decent numbers, for several hours
non-stop. Sometimes days during burn in. Note that running pgbench
on a machine other than the actual db is often a good idea so you're
not measuring how fast pgbench can run in contention with your own
database.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgbench results on a new server
@ 2010-06-25 19:03 Greg Smith <greg@2ndquadrant.com>
parent: Craig James <craig_james@emolecules.com>
1 sibling, 1 reply; 6+ messages in thread
From: Greg Smith @ 2010-06-25 19:03 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
Craig James wrote:
> I've got a new server and want to make sure it's running well.
Any changes to the postgresql.conf file? Generally you need at least a
moderate shared_buffers (1GB or so at a minimum) and checkpoint_segments
(32 or higher) in order for the standard pgbench test to give good results.
> pgbench -c20 -t 5000 -U test
> tps = 5789
> pgbench -c30 -t 3333 -U test
> tps = 6961
> pgbench -c40 -t 2500 -U test
> tps = 2945
General numbers are OK, the major drop going from 30 to 40 clients is
larger than it should be. I'd suggest running the 40 client count one
again to see if that's consistent. If it is, that may just be pgbench
itself running into a problem. It doesn't handle high client counts
very well unless you use the 9.0 version that supports multiple pgbench
workers with the "-j" option.
--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com www.2ndQuadrant.us
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgbench results on a new server
@ 2010-06-28 17:12 Craig James <craig_james@emolecules.com>
parent: Greg Smith <greg@2ndquadrant.com>
0 siblings, 2 replies; 6+ messages in thread
From: Craig James @ 2010-06-28 17:12 UTC (permalink / raw)
To: Greg Smith <greg@2ndquadrant.com>; +Cc: pgsql-performance
On 6/25/10 12:03 PM, Greg Smith wrote:
> Craig James wrote:
>> I've got a new server and want to make sure it's running well.
>
> Any changes to the postgresql.conf file? Generally you need at least a
> moderate shared_buffers (1GB or so at a minimum) and checkpoint_segments
> (32 or higher) in order for the standard pgbench test to give good results.
max_connections = 500
shared_buffers = 1000MB
work_mem = 128MB
synchronous_commit = off
full_page_writes = off
wal_buffers = 256kB
checkpoint_segments = 30
effective_cache_size = 4GB
For fun I ran it with the installation defaults, and it never got above 1475 TPS.
>> pgbench -c20 -t 5000 -U test
>> tps = 5789
>> pgbench -c30 -t 3333 -U test
>> tps = 6961
>> pgbench -c40 -t 2500 -U test
>> tps = 2945
>
> General numbers are OK, the major drop going from 30 to 40 clients is
> larger than it should be. I'd suggest running the 40 client count one
> again to see if that's consistent.
It is consistent. When I run pgbench from a different server, I get this:
pgbench -c40 -t 2500 -U test
tps = 7999
pgbench -c100 -t 1000 -U test
tps = 6693
Craig
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgbench results on a new server
@ 2010-06-29 13:18 Merlin Moncure <mmoncure@gmail.com>
parent: Craig James <craig_james@emolecules.com>
1 sibling, 0 replies; 6+ messages in thread
From: Merlin Moncure @ 2010-06-29 13:18 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; +Cc: Greg Smith <greg@2ndquadrant.com>; pgsql-performance
On Mon, Jun 28, 2010 at 1:12 PM, Craig James <craig_james@emolecules.com> wrote:
> On 6/25/10 12:03 PM, Greg Smith wrote:
>>
>> Craig James wrote:
>>>
>>> I've got a new server and want to make sure it's running well.
>>
>> Any changes to the postgresql.conf file? Generally you need at least a
>> moderate shared_buffers (1GB or so at a minimum) and checkpoint_segments
>> (32 or higher) in order for the standard pgbench test to give good
>> results.
>
> max_connections = 500
> shared_buffers = 1000MB
> work_mem = 128MB
> synchronous_commit = off
> full_page_writes = off
> wal_buffers = 256kB
> checkpoint_segments = 30
> effective_cache_size = 4GB
>
> For fun I ran it with the installation defaults, and it never got above 1475
> TPS.
>
>>> pgbench -c20 -t 5000 -U test
>>> tps = 5789
>>> pgbench -c30 -t 3333 -U test
>>> tps = 6961
>>> pgbench -c40 -t 2500 -U test
>>> tps = 2945
>>
>> General numbers are OK, the major drop going from 30 to 40 clients is
>> larger than it should be. I'd suggest running the 40 client count one
>> again to see if that's consistent.
>
> It is consistent. When I run pgbench from a different server, I get this:
>
> pgbench -c40 -t 2500 -U test
> tps = 7999
>
> pgbench -c100 -t 1000 -U test
> tps = 6693
6k tps over 8 7200 rpm disks is quite good imo. synchronous_commit
setting is making that possible. building a server that could handle
that much was insanely expensive just a few years ago, on relatively
cheap sorage. that's 21m transactions an hour or ~ half a billion
transactions a day (!). running this kind of load 24x7 on postgres
7.x would have been an enormous headache. how quickly we forget! :-)
your 'real' tps write, 1475 tps, spread over 4 disks doing the actual
writing, is giving you ~ 370 tps/device. not bad at all -- the raid
controller is doing a good job (the raw drive might get 200 or so). I
bet performance will be somewhat worse with a higher scaling factor
(say, 500) because there is less locality of writes -- something to
consider if you expect your database to get really big.
merlin
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgbench results on a new server
@ 2010-06-30 01:23 Greg Smith <greg@2ndquadrant.com>
parent: Craig James <craig_james@emolecules.com>
1 sibling, 0 replies; 6+ messages in thread
From: Greg Smith @ 2010-06-30 01:23 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
Craig James wrote:
> synchronous_commit = off
> full_page_writes = off
I don't have any numbers handy on how much turning synchronous_commit
and full_page_writes off improves performance on a system with a
battery-backed write cache. Your numbers are therefore a bit inflated
against similar ones that are doing a regular sync commit. Just
something to keep in mind when comparing against other people's results.
Also, just as a general comment, increase in work_mem and
effective_cache_size don't actually do anything to the built-in pgbench
test results.
>> General numbers are OK, the major drop going from 30 to 40 clients is
>> larger than it should be. I'd suggest running the 40 client count one
>> again to see if that's consistent.
>
> It is consistent. When I run pgbench from a different server, I get
> this:
>
> pgbench -c40 -t 2500 -U test
> tps = 7999
>
> pgbench -c100 -t 1000 -U test
> tps = 6693
Looks like you're just running into the limitations of the old pgbench
code failing to keep up with high client count loads when run on the
same system as the server. Nothing to be concerned about--that the drop
is only small with the pgbench client remote says there's not actually a
server problem here.
With that sorted out, your system looks in the normal range for the sort
of hardware you're using. I'm always concerned about the potential
reliability issues that come with async commit and turning off full page
writes though, so you might want to re-test with those turned on and see
if you can live with the results.
--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com www.2ndQuadrant.us
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2010-06-30 01:23 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-06-25 18:53 pgbench results on a new server Craig James <craig_james@emolecules.com>
2010-06-25 19:01 ` Scott Marlowe <scott.marlowe@gmail.com>
2010-06-25 19:03 ` Greg Smith <greg@2ndquadrant.com>
2010-06-28 17:12 ` Craig James <craig_james@emolecules.com>
2010-06-29 13:18 ` Merlin Moncure <mmoncure@gmail.com>
2010-06-30 01:23 ` Greg Smith <greg@2ndquadrant.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox