pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedHardware spec
12+ messages / 6 participants
[nested] [flat]
* Hardware spec
@ 2007-09-06 06:54 Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
0 siblings, 1 reply; 12+ messages in thread
From: Willo van der Merwe @ 2007-09-06 06:54 UTC (permalink / raw)
To: pgsql-performance
Hi guys,
I'm have the rare opportunity to spec the hardware for a new database
server. It's going to replace an older one, driving a social networking
web application. The current server (a quad opteron with 4Gb of RAM and
80Gb fast SCSI RAID10) is coping with an average load of ranging between
1.5 and 3.5.
The new machine spec I have so far:
2 x Intel Xeon 2.33 GHz Dual Core Woodcrest Processors
4 Gb RAM
5x73 GB Ultra320 SCSI RAID 5 (288 GB storage)
I've heard that RAID 5 is not necessarily the best performer. Also, are
there any special tricks when partition the file system?
Regards,
Willo
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
@ 2007-09-06 08:17 ` Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
0 siblings, 1 reply; 12+ messages in thread
From: Richard Huxton @ 2007-09-06 08:17 UTC (permalink / raw)
To: Willo van der Merwe <willo@mirasol.co.za>; +Cc: pgsql-performance
Willo van der Merwe wrote:
> Hi guys,
>
> I'm have the rare opportunity to spec the hardware for a new database
> server. It's going to replace an older one, driving a social networking
> web application. The current server (a quad opteron with 4Gb of RAM and
> 80Gb fast SCSI RAID10) is coping with an average load of ranging between
> 1.5 and 3.5.
>
> The new machine spec I have so far:
What's the limiting factor on your current machine - disk, memory, cpup?
--
Richard Huxton
Archonet Ltd
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
@ 2007-09-06 09:26 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 11:16 ` Re: Hardware spec Jean-David Beyer <jeandavid8@verizon.net>
2007-09-11 20:35 ` Re: Hardware spec Decibel! <decibel@decibel.org>
0 siblings, 2 replies; 12+ messages in thread
From: Willo van der Merwe @ 2007-09-06 09:26 UTC (permalink / raw)
To: Richard Huxton <dev@archonet.com>; +Cc: pgsql-performance
Richard Huxton wrote:
> Willo van der Merwe wrote:
>> Hi guys,
>>
>> I'm have the rare opportunity to spec the hardware for a new database
>> server. It's going to replace an older one, driving a social networking
>> web application. The current server (a quad opteron with 4Gb of RAM and
>> 80Gb fast SCSI RAID10) is coping with an average load of ranging between
>> 1.5 and 3.5.
>>
>> The new machine spec I have so far:
> What's the limiting factor on your current machine - disk, memory, cpup?
I'm a bit embarrassed to admit that I'm not sure. The reason we're
changing machines is that we might be changing ISPs and we're renting /
leasing the machines from the ISP.
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
@ 2007-09-06 11:16 ` Jean-David Beyer <jeandavid8@verizon.net>
2007-09-06 11:37 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
1 sibling, 1 reply; 12+ messages in thread
From: Jean-David Beyer @ 2007-09-06 11:16 UTC (permalink / raw)
To: pgsql-performance
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Willo van der Merwe wrote:
> Richard Huxton wrote:
>> Willo van der Merwe wrote:
>>> Hi guys,
>>>
>>> I'm have the rare opportunity to spec the hardware for a new database
>>> server. It's going to replace an older one, driving a social
>>> networking web application. The current server (a quad opteron with
>>> 4Gb of RAM and 80Gb fast SCSI RAID10) is coping with an average load
>>> of ranging between 1.5 and 3.5.
>>>
>>> The new machine spec I have so far:
>> What's the limiting factor on your current machine - disk, memory,
>> cpup?
> I'm a bit embarrassed to admit that I'm not sure. The reason we're
> changing machines is that we might be changing ISPs and we're renting /
> leasing the machines from the ISP.
>
Before you get rid of the current ISP, better examine what is going on with
the present setup. It would be good to know if you are memory, processor, or
IO limited. That way you could increase what needs to be increased, and not
waste money where the bottleneck is not.
- --
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 07:10:01 up 28 days, 10:32, 4 users, load average: 5.48, 4.77, 4.37
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with CentOS - http://enigmail.mozdev.org
iD8DBQFG3+FwPtu2XpovyZoRAmp+AJ9R4mvznqJ24ZCPK8DcTAsz2d34+QCfQzhH
vmXnoJO0vm/A/f/Ol0TOy6o=
=9rsm
-----END PGP SIGNATURE-----
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 11:16 ` Re: Hardware spec Jean-David Beyer <jeandavid8@verizon.net>
@ 2007-09-06 11:37 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 14:19 ` Re: Hardware spec Florian Weimer <fweimer@bfk.de>
0 siblings, 1 reply; 12+ messages in thread
From: Willo van der Merwe @ 2007-09-06 11:37 UTC (permalink / raw)
To: Jean-David Beyer <jeandavid8@verizon.net>; +Cc: pgsql-performance
Jean-David Beyer wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Willo van der Merwe wrote:
>
>> Richard Huxton wrote:
>>
>>> Willo van der Merwe wrote:
>>>
>>>> Hi guys,
>>>>
>>>> I'm have the rare opportunity to spec the hardware for a new database
>>>> server. It's going to replace an older one, driving a social
>>>> networking web application. The current server (a quad opteron with
>>>> 4Gb of RAM and 80Gb fast SCSI RAID10) is coping with an average load
>>>> of ranging between 1.5 and 3.5.
>>>>
>>>> The new machine spec I have so far:
>>>>
>>> What's the limiting factor on your current machine - disk, memory,
>>> cpup?
>>>
>> I'm a bit embarrassed to admit that I'm not sure. The reason we're
>> changing machines is that we might be changing ISPs and we're renting /
>> leasing the machines from the ISP.
>>
>>
> Before you get rid of the current ISP, better examine what is going on with
> the present setup. It would be good to know if you are memory, processor, or
> IO limited. That way you could increase what needs to be increased, and not
> waste money where the bottleneck is not.
>
Good advice. After running a vmstat and iostat, it is clear, to my mind
anyway, that the most likely bottleneck is IO, next is probably some
more RAM.
Here's the output:
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
0 0 29688 80908 128308 3315792 0 0 8 63 6 8 17
2 80 1
avg-cpu: %user %nice %sys %iowait %idle
17.18 0.00 1.93 0.81 80.08
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 14.57 66.48 506.45 58557617 446072213
sda1 0.60 0.27 4.70 235122 4136128
sda2 0.38 0.77 2.27 678754 2002576
sda3 2.37 0.49 18.61 429171 16389960
sda4 0.00 0.00 0.00 2 0
sda5 0.71 0.66 5.46 578307 4807087
sda6 0.03 0.01 0.24 6300 214196
sda7 0.02 0.00 0.19 2622 165992
sda8 60.19 64.29 474.98 56626211 418356226
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 11:16 ` Re: Hardware spec Jean-David Beyer <jeandavid8@verizon.net>
2007-09-06 11:37 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
@ 2007-09-06 14:19 ` Florian Weimer <fweimer@bfk.de>
2007-09-06 14:33 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
0 siblings, 1 reply; 12+ messages in thread
From: Florian Weimer @ 2007-09-06 14:19 UTC (permalink / raw)
To: Willo van der Merwe <willo@mirasol.co.za>; +Cc: Jean-David Beyer <jeandavid8@verizon.net>; pgsql-performance
* Willo van der Merwe:
> Good advice. After running a vmstat and iostat, it is clear, to my
> mind anyway, that the most likely bottleneck is IO, next is probably
> some more RAM.
> Here's the output:
> procs -----------memory---------- ---swap-- -----io---- --system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us
> sy id wa
> 0 0 29688 80908 128308 3315792 0 0 8 63 6 8 17
> 2 80 1
You need to run "vmstat 10" (for ten-second averages) and report a
couple of lines.
> avg-cpu: %user %nice %sys %iowait %idle
> 17.18 0.00 1.93 0.81 80.08
Same for iostat.
Your initial numbers suggest that your server isn't I/O-bound, though
(the percentage spent in iowait is much too small, and so are the tps
numbers).
--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 11:16 ` Re: Hardware spec Jean-David Beyer <jeandavid8@verizon.net>
2007-09-06 11:37 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 14:19 ` Re: Hardware spec Florian Weimer <fweimer@bfk.de>
@ 2007-09-06 14:33 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 15:21 ` Re: Hardware spec Florian Weimer <fweimer@bfk.de>
0 siblings, 1 reply; 12+ messages in thread
From: Willo van der Merwe @ 2007-09-06 14:33 UTC (permalink / raw)
To: Florian Weimer <fweimer@bfk.de>; +Cc: Jean-David Beyer <jeandavid8@verizon.net>; pgsql-performance
Florian Weimer wrote:
> You need to run "vmstat 10" (for ten-second averages) and report a
> couple of lines.
>
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
1 0 61732 47388 27908 3431344 0 0 10 65 1 4 17
2 80 1
5 0 61732 37052 28180 3431956 0 0 14 987 2320 2021 38
4 56 2
1 0 61620 43076 28356 3432256 0 0 0 367 1691 1321 28
3 67 1
3 0 61620 37620 28484 3432740 0 0 0 580 4088 6792 40
5 54 1
2 0 61596 33716 28748 3433520 0 0 24 415 2087 1890 44
4 49 2
3 0 61592 45300 28904 3416200 3 0 61 403 2282 2154 41
4 54 1
7 0 61592 30172 29092 3416964 0 0 19 358 2779 3478 31
6 63 1
1 0 61580 62948 29180 3417368 6 0 27 312 3632 4396 38
4 57 1
1 0 61444 62388 29400 3417964 0 0 6 354 2163 1918 31
4 64 1
2 0 61444 53988 29648 3417988 0 0 0 553 2095 1687 33
3 63 1
1 0 61444 63988 29832 3418348 0 0 6 352 1767 1424 22
3 73 1
1 1 61444 51148 30052 3419148 0 0 50 349 1524 834 22
3 74 2
1 0 61432 53460 30524 3419572 7 0 7 868 4434 6706 43
6 49 2
1 0 61432 58668 30628 3420148 0 0 0 284 1785 1628 27
3 69 1
iostat sda8 is the where the pg_data resides, sda3 is /var/log
avg-cpu: %user %nice %sys %iowait %idle
17.36 0.00 1.96 0.82 79.86
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda3 2.38 0.49 18.71 432395 16672800
sda8 62.34 74.46 491.74 66345555 438143794
avg-cpu: %user %nice %sys %iowait %idle
30.50 0.00 3.57 1.70 64.22
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda3 5.60 0.00 44.80 0 448
sda8 120.20 134.40 956.00 1344 9560
avg-cpu: %user %nice %sys %iowait %idle
20.68 0.00 3.43 1.35 74.54
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda3 3.30 0.00 26.40 0 264
sda8 97.90 0.00 783.20 0 7832
avg-cpu: %user %nice %sys %iowait %idle
22.31 0.00 2.75 0.68 74.27
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda3 2.10 0.00 16.78 0 168
sda8 60.34 0.80 481.92 8 4824
avg-cpu: %user %nice %sys %iowait %idle
11.65 0.00 1.60 1.03 85.72
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda3 1.70 0.00 13.61 0 136
sda8 59.36 0.00 474.87 0 4744
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 11:16 ` Re: Hardware spec Jean-David Beyer <jeandavid8@verizon.net>
2007-09-06 11:37 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 14:19 ` Re: Hardware spec Florian Weimer <fweimer@bfk.de>
2007-09-06 14:33 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
@ 2007-09-06 15:21 ` Florian Weimer <fweimer@bfk.de>
0 siblings, 0 replies; 12+ messages in thread
From: Florian Weimer @ 2007-09-06 15:21 UTC (permalink / raw)
To: Willo van der Merwe <willo@mirasol.co.za>; +Cc: Jean-David Beyer <jeandavid8@verizon.net>; pgsql-performance
* Willo van der Merwe:
> Florian Weimer wrote:
>> You need to run "vmstat 10" (for ten-second averages) and report a
>> couple of lines.
> 2 80 1
> 5 0 61732 37052 28180 3431956 0 0 14 987 2320 2021 38
> sda3 3.30 0.00 26.40 0 264
> sda8 97.90 0.00 783.20 0 7832
These values don't look I/O bound to me. CPU usage is pretty low,
too.
--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
@ 2007-09-11 20:35 ` Decibel! <decibel@decibel.org>
2007-09-12 07:01 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
1 sibling, 1 reply; 12+ messages in thread
From: Decibel! @ 2007-09-11 20:35 UTC (permalink / raw)
To: Willo van der Merwe <willo@mirasol.co.za>; +Cc: Richard Huxton <dev@archonet.com>; pgsql-performance
On Thu, Sep 06, 2007 at 11:26:46AM +0200, Willo van der Merwe wrote:
> Richard Huxton wrote:
> >Willo van der Merwe wrote:
> >>Hi guys,
> >>
> >>I'm have the rare opportunity to spec the hardware for a new database
> >>server. It's going to replace an older one, driving a social networking
> >>web application. The current server (a quad opteron with 4Gb of RAM and
> >>80Gb fast SCSI RAID10) is coping with an average load of ranging between
> >>1.5 and 3.5.
> >>
> >>The new machine spec I have so far:
> >What's the limiting factor on your current machine - disk, memory, cpup?
> I'm a bit embarrassed to admit that I'm not sure. The reason we're
> changing machines is that we might be changing ISPs and we're renting /
> leasing the machines from the ISP.
Get yourself the ability to benchmark your application. This is
invaluable^W a requirement for any kind of performance tuning.
--
Decibel!, aka Jim Nasby decibel@decibel.org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-11 20:35 ` Re: Hardware spec Decibel! <decibel@decibel.org>
@ 2007-09-12 07:01 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-12 07:17 ` Re: Hardware spec Claus Guttesen <kometen@gmail.com>
0 siblings, 1 reply; 12+ messages in thread
From: Willo van der Merwe @ 2007-09-12 07:01 UTC (permalink / raw)
To: Decibel! <decibel@decibel.org>; +Cc: Richard Huxton <dev@archonet.com>; pgsql-performance
Decibel! wrote:
> On Thu, Sep 06, 2007 at 11:26:46AM +0200, Willo van der Merwe wrote:
>
>> Richard Huxton wrote:
>>
>>> Willo van der Merwe wrote:
>>>
>>>> Hi guys,
>>>>
>>>> I'm have the rare opportunity to spec the hardware for a new database
>>>> server. It's going to replace an older one, driving a social networking
>>>> web application. The current server (a quad opteron with 4Gb of RAM and
>>>> 80Gb fast SCSI RAID10) is coping with an average load of ranging between
>>>> 1.5 and 3.5.
>>>>
>>>> The new machine spec I have so far:
>>>>
>>> What's the limiting factor on your current machine - disk, memory, cpup?
>>>
>> I'm a bit embarrassed to admit that I'm not sure. The reason we're
>> changing machines is that we might be changing ISPs and we're renting /
>> leasing the machines from the ISP.
>>
>
> Get yourself the ability to benchmark your application. This is
> invaluable^W a requirement for any kind of performance tuning.
>
I'm pretty happy with the performance of the database at this stage.
Correct me if I'm wrong, but AFAIK a load of 3.5 on a quad is not
overloading it. It also seem to scale well, so if application's demand
increases I see a minimal increase in database server load.
I was just looking for some pointers as to where to go to ITO hardware
for the future, as I can now spec a new machine. I mean is it really
worth while going for one of those RAID controllers with the battery
backed cache, for instance. If so, are there any specific ones to look
out for? Which is better RAID 5, a large RAID 10 or smaller RAID 10's?
Should I bother with RAID at all?
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Re: Hardware spec Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-11 20:35 ` Re: Hardware spec Decibel! <decibel@decibel.org>
2007-09-12 07:01 ` Re: Hardware spec Willo van der Merwe <willo@mirasol.co.za>
@ 2007-09-12 07:17 ` Claus Guttesen <kometen@gmail.com>
0 siblings, 0 replies; 12+ messages in thread
From: Claus Guttesen @ 2007-09-12 07:17 UTC (permalink / raw)
To: Willo van der Merwe <willo@mirasol.co.za>; +Cc: Decibel! <decibel@decibel.org>; Richard Huxton <dev@archonet.com>; pgsql-performance
> > Get yourself the ability to benchmark your application. This is
> > invaluable^W a requirement for any kind of performance tuning.
> >
> I'm pretty happy with the performance of the database at this stage.
> Correct me if I'm wrong, but AFAIK a load of 3.5 on a quad is not
> overloading it. It also seem to scale well, so if application's demand
> increases I see a minimal increase in database server load.
>
> I was just looking for some pointers as to where to go to ITO hardware
> for the future, as I can now spec a new machine. I mean is it really
> worth while going for one of those RAID controllers with the battery
> backed cache, for instance. If so, are there any specific ones to look
> out for? Which is better RAID 5, a large RAID 10 or smaller RAID 10's?
> Should I bother with RAID at all?
These issues have been covered before. You may want to search the
archives and get the relevant pointers.
--
regards
Claus
When lenity and cruelty play for a kingdom,
the gentlest gamester is the soonest winner.
Shakespeare
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Hardware spec]
@ 2007-09-06 15:00 Jean-David Beyer <jeandavid8@verizon.net>
0 siblings, 0 replies; 12+ messages in thread
From: Jean-David Beyer @ 2007-09-06 15:00 UTC (permalink / raw)
To: pgsql-performance
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Willo van der Merwe wrote:
> Jean-David Beyer wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Willo van der Merwe wrote:
>>
>>> Richard Huxton wrote:
>>>
>>>> Willo van der Merwe wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>> I'm have the rare opportunity to spec the hardware for a new database
>>>>> server. It's going to replace an older one, driving a social
>>>>> networking web application. The current server (a quad opteron with
>>>>> 4Gb of RAM and 80Gb fast SCSI RAID10) is coping with an average load
>>>>> of ranging between 1.5 and 3.5.
>>>>>
>>>>> The new machine spec I have so far:
>>>>>
>>>> What's the limiting factor on your current machine - disk, memory,
>>>> cpup?
>>>>
>>> I'm a bit embarrassed to admit that I'm not sure. The reason we're
>>> changing machines is that we might be changing ISPs and we're renting
>>> / leasing the machines from the ISP.
>>>
>>>
>> Before you get rid of the current ISP, better examine what is going on
>> with
>> the present setup. It would be good to know if you are memory,
>> processor, or
>> IO limited. That way you could increase what needs to be increased,
>> and not
>> waste money where the bottleneck is not.
>>
> Good advice. After running a vmstat and iostat, it is clear, to my mind
> anyway, that the most likely bottleneck is IO, next is probably some
> more RAM.
> Here's the output:
> procs -----------memory---------- ---swap-- -----io---- --system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy
> id wa
> 0 0 29688 80908 128308 3315792 0 0 8 63 6 8 17 2
> 80 1
>
>
> avg-cpu: %user %nice %sys %iowait %idle
> 17.18 0.00 1.93 0.81 80.08
>
> Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
> sda 14.57 66.48 506.45 58557617 446072213
> sda1 0.60 0.27 4.70 235122 4136128
> sda2 0.38 0.77 2.27 678754 2002576
> sda3 2.37 0.49 18.61 429171 16389960
> sda4 0.00 0.00 0.00 2 0
> sda5 0.71 0.66 5.46 578307 4807087
> sda6 0.03 0.01 0.24 6300 214196
> sda7 0.02 0.00 0.19 2622 165992
> sda8 60.19 64.29 474.98 56626211 418356226
>
>
1.) If this is when the system is heavily loaded, you have more capacity
than you need, on the average. Processors are idle, not in wait state. You
have enough memory (no swapping going on), disks not too busy (probably).
But if it is not heavily loaded, run these when it is.
2.) Did you let vmstat and iostat run just once, or are these the last of
several reports. Because these tell the average since boot for the first
report of each when they run the first time. If so, the numbers may not mean
much.
Here is iostat for my machine that is running right now. Only lines involved
in postgreSQL are shown:
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda8 409.33 0.67 3274.00 40 196440
sdb7 0.12 0.00 0.93 0 56
sdc1 44.73 0.13 357.73 8 21464
sdd1 23.43 0.00 187.47 0 11248
sde1 133.45 0.00 1067.60 0 64056
sdf1 78.25 0.00 626.00 0 37560
On sda8 is the Write-Ahead-Log.
on sdb7 are some small seldom-used relations, but also the input files that
sda and sdb are what my system uses for other stuff as well, though sda is
not too heavily used and sdb even less.
I am presently loading into the database. sdc1, sdd1, sde1, and sdf1 are the
drives reserved for database only.
I would suggest getting at least two hard drives on the new system and
perhaps putting the WAL on one and the rest on the other. My sda and sdb
drives are around 72 GBytes and the sdc, sdd, sde, and sdf are about 18
GBytes. I believe the more spindles the better (within reason) and dividing
the stuff up with the indices separate from the associated data to reduce
seeking.
$ vmstat 30
procs -----------memory---------- ---swap-- -----io---- --system--
- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy
id wa
5 1 1340 248456 248496 6652672 0 0 28 149 2 1 94 2
3 0
4 2 1340 244240 248948 6656364 0 0 0 2670 1248 11320 80 4
14 2
4 1 1340 241008 249492 6659864 0 0 0 2701 1222 11432 82 4
12 2
5 0 1340 246268 249868 6653644 0 0 0 2799 1223 11412 83 4
12 2
My machine was idle most of the time since boot (other than running BOINC
stuff), but is busy loading data into the database at the moment. See how
the first line differs from the others? I have 8GBytes RAM on this 32-bit
machine running Red Hat Enterprise Linux 5.
- --
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 10:55:01 up 28 days, 14:17, 4 users, load average: 5.36, 5.64, 5.55
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with CentOS - http://enigmail.mozdev.org
iD8DBQFG4BYFPtu2XpovyZoRArU9AJ9o3OvYNxmQVhINTcRCADy0/fv30wCfZ3oJ
WItsCN75Xxhv52AqF6AIXmk=
=3AfU
-----END PGP SIGNATURE-----
^ permalink raw reply [nested|flat] 12+ messages in thread
end of thread, other threads:[~2007-09-12 07:17 UTC | newest]
Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-09-06 06:54 Hardware spec Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 08:17 ` Richard Huxton <dev@archonet.com>
2007-09-06 09:26 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 11:16 ` Jean-David Beyer <jeandavid8@verizon.net>
2007-09-06 11:37 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 14:19 ` Florian Weimer <fweimer@bfk.de>
2007-09-06 14:33 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-06 15:21 ` Florian Weimer <fweimer@bfk.de>
2007-09-11 20:35 ` Decibel! <decibel@decibel.org>
2007-09-12 07:01 ` Willo van der Merwe <willo@mirasol.co.za>
2007-09-12 07:17 ` Claus Guttesen <kometen@gmail.com>
2007-09-06 15:00 Re: Hardware spec] Jean-David Beyer <jeandavid8@verizon.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox