pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedgeneral hardware advice
7+ messages / 4 participants
[nested] [flat]
* general hardware advice
@ 2011-02-06 18:16 Linos <info@linos.es>
0 siblings, 3 replies; 7+ messages in thread
From: Linos @ 2011-02-06 18:16 UTC (permalink / raw)
To: pgsql-performance
I am searching what would be the best hardware combination to a new server i
have to install, the new server will have a Postgresql 9.0 with a database of
about 10gb, the database part it is not the problem for me, in this size almost,
the part where i am a bit lost is that i have to share from the same server
about 20TB of data with samba or ISCSI (i have to test both ways to share when i
have the hardware) because this is to be the file-server of 8 Avid video
workstations.
I am pretty sure that the Postgresql should be installed in a raid10
configuration wigh pg_xlog in a different physical volume, i know too that the
most important things to Postgresql are memory and disks, i will get a raid card
with BBU or flash cache to be safe with write caching.
About the file-server part the files we are going to move are in the range of
1gb~4gb in mostly a sequential access nature for what i have read of how avid
works, so i think the best would be a raid 6 here, but i am not sure what
hardware to buy because i have not stored never this quantity of storage, i have
read in this mailing list other times about external enclosure boxes to attach
storage and i have read raid cards recommendations before, lsi, 3ware or areca
seems safe bets but i don't know how to put all the pieces together.
I have searched in HP Servers but i can only find HP Smart Array controllers and
i don't find a server that sill uses the best of the Smart Array P800 or Smart
Array P812, so i don't know what server to buy (it is better to stick with a
server vendor or mount my own with hardware parts), what box enclosure to buy
and how to connect the two. Could anyone please point me specific hardware parts
that he would recommend me?
I am studying too the possibility of use an OCZ Vertex 2 Pro with Flashcache or
Bcache to use it like a second level filesystem cache, any comments on that please?
Thanks,
Miguel Angel.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: general hardware advice
@ 2011-02-06 19:05 Scott Marlowe <scott.marlowe@gmail.com>
parent: Linos <info@linos.es>
2 siblings, 1 reply; 7+ messages in thread
From: Scott Marlowe @ 2011-02-06 19:05 UTC (permalink / raw)
To: Linos <info@linos.es>; +Cc: pgsql-performance
On Sun, Feb 6, 2011 at 11:16 AM, Linos <info@linos.es> wrote:
> I am searching what would be the best hardware combination to a new server i
> have to install, the new server will have a Postgresql 9.0 with a database
> of about 10gb, the database part it is not the problem for me, in this size
> almost, the part where i am a bit lost is that i have to share from the same
> server about 20TB of data with samba or ISCSI (i have to test both ways to
> share when i have the hardware) because this is to be the file-server of 8
> Avid video workstations.
>
> I am pretty sure that the Postgresql should be installed in a raid10
> configuration wigh pg_xlog in a different physical volume, i know too that
> the most important things to Postgresql are memory and disks, i will get a
> raid card with BBU or flash cache to be safe with write caching.
I'd put all of pgsql on a different card than the file share.
> About the file-server part the files we are going to move are in the range
> of 1gb~4gb in mostly a sequential access nature for what i have read of how
> avid works, so i think the best would be a raid 6 here, but i am not sure
> what hardware to buy because i have not stored never this quantity of
> storage, i have read in this mailing list other times about external
> enclosure boxes to attach storage and i have read raid cards recommendations
> before, lsi, 3ware or areca seems safe bets but i don't know how to put all
> the pieces together.
If the data on this is considered disposable, then you could look at
RAID-0 for the fastest performance. Even with SW RAID0 you'd get
incredible throughput with a few disks.
> I am studying too the possibility of use an OCZ Vertex 2 Pro with Flashcache
> or Bcache to use it like a second level filesystem cache, any comments on
> that please?
My coworkers RAVE over ZFS solaris with flash drives for cache and
spinning drives for mass storage.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: general hardware advice
@ 2011-02-06 19:24 Pierre C <lists@peufeu.com>
parent: Linos <info@linos.es>
2 siblings, 1 reply; 7+ messages in thread
From: Pierre C @ 2011-02-06 19:24 UTC (permalink / raw)
To: pgsql-performance@postgresql.org, Linos <info@linos.es>
On Sun, 06 Feb 2011 19:16:23 +0100, Linos <info@linos.es> wrote:
> I am searching what would be the best hardware combination to a new
> server i have to install, the new server will have a Postgresql 9.0 with
> a database of about 10gb, the database part it is not the problem for
> me, in this size almost, the part where i am a bit lost is that i have
> to share from the same server about 20TB of data with samba or ISCSI (i
> have to test both ways to share when i have the hardware) because this
> is to be the file-server of 8 Avid video workstations.
What is the expected load on the postgresql instance ?
Also, for multiple high throughput concurrent streams (as in AVID),
filesystem choice is critical.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: general hardware advice
@ 2011-02-06 19:31 Linos <info@linos.es>
parent: Scott Marlowe <scott.marlowe@gmail.com>
0 siblings, 0 replies; 7+ messages in thread
From: Linos @ 2011-02-06 19:31 UTC (permalink / raw)
To: Scott Marlowe <scott.marlowe@gmail.com>; +Cc: pgsql-performance
El 06/02/11 20:05, Scott Marlowe escribió:
> On Sun, Feb 6, 2011 at 11:16 AM, Linos<info@linos.es> wrote:
>> I am searching what would be the best hardware combination to a new server i
>> have to install, the new server will have a Postgresql 9.0 with a database
>> of about 10gb, the database part it is not the problem for me, in this size
>> almost, the part where i am a bit lost is that i have to share from the same
>> server about 20TB of data with samba or ISCSI (i have to test both ways to
>> share when i have the hardware) because this is to be the file-server of 8
>> Avid video workstations.
>>
>> I am pretty sure that the Postgresql should be installed in a raid10
>> configuration wigh pg_xlog in a different physical volume, i know too that
>> the most important things to Postgresql are memory and disks, i will get a
>> raid card with BBU or flash cache to be safe with write caching.
>
> I'd put all of pgsql on a different card than the file share.
This should not be a problem, i think you have reason here maybe i could use
internal disks and 1 raid card for Postgresql.
>
>> About the file-server part the files we are going to move are in the range
>> of 1gb~4gb in mostly a sequential access nature for what i have read of how
>> avid works, so i think the best would be a raid 6 here, but i am not sure
>> what hardware to buy because i have not stored never this quantity of
>> storage, i have read in this mailing list other times about external
>> enclosure boxes to attach storage and i have read raid cards recommendations
>> before, lsi, 3ware or areca seems safe bets but i don't know how to put all
>> the pieces together.
>
> If the data on this is considered disposable, then you could look at
> RAID-0 for the fastest performance. Even with SW RAID0 you'd get
> incredible throughput with a few disks.
>
It is not disposable, all the contrary, i have to take special care of the this
files :)
>> I am studying too the possibility of use an OCZ Vertex 2 Pro with Flashcache
>> or Bcache to use it like a second level filesystem cache, any comments on
>> that please?
>
> My coworkers RAVE over ZFS solaris with flash drives for cache and
> spinning drives for mass storage.
>
i think neither flashcache nor bcache are at the level of zfs with l2arc in ssd
but should perform well anyway, what it is the preferred way of use good raid
cards with zfs? jbod and configure in raid-z? i have not used zfs still.
Regards,
Miguel Angel.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: general hardware advice
@ 2011-02-06 19:34 Linos <info@linos.es>
parent: Pierre C <lists@peufeu.com>
0 siblings, 0 replies; 7+ messages in thread
From: Linos @ 2011-02-06 19:34 UTC (permalink / raw)
To: Pierre C <lists@peufeu.com>; +Cc: pgsql-performance
El 06/02/11 20:24, Pierre C escribió:
> On Sun, 06 Feb 2011 19:16:23 +0100, Linos <info@linos.es> wrote:
>
>> I am searching what would be the best hardware combination to a new server i
>> have to install, the new server will have a Postgresql 9.0 with a database of
>> about 10gb, the database part it is not the problem for me, in this size
>> almost, the part where i am a bit lost is that i have to share from the same
>> server about 20TB of data with samba or ISCSI (i have to test both ways to
>> share when i have the hardware) because this is to be the file-server of 8
>> Avid video workstations.
>
> What is the expected load on the postgresql instance ?
>
> Also, for multiple high throughput concurrent streams (as in AVID), filesystem
> choice is critical.
>
The load for the Postgresql instance it is to be low, maybe 5 or 6 simultaneous
clients with a oltp application, but it is database intensive only when
reporting not in the usual work.
About the filesystem i was thinking to use xfs, i think would be the best for
this use-case in Linux, any advice on this please?
Regards,
Miguel Angel.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: general hardware advice
@ 2011-02-06 20:38 Andy <angelflow@yahoo.com>
parent: Linos <info@linos.es>
2 siblings, 1 reply; 7+ messages in thread
From: Andy @ 2011-02-06 20:38 UTC (permalink / raw)
To: pgsql-performance@postgresql.org, Linos <info@linos.es>
--- On Sun, 2/6/11, Linos <info@linos.es> wrote:
> I am studying too the possibility of use an OCZ Vertex 2
> Pro with Flashcache or Bcache to use it like a second level
> filesystem cache, any comments on that please?
>
OCZ Vertex 2 Pro is a lot more expensive than other SSD of comparable performances because it comes with a supercapacitor that guarantees durability.
If you're just using the SSD as a cache, you don't need durability. You could save a lot of money by getting SSDs without supercapacitor, such as Corsair Force.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: general hardware advice
@ 2011-02-06 23:34 Linos <info@linos.es>
parent: Andy <angelflow@yahoo.com>
0 siblings, 0 replies; 7+ messages in thread
From: Linos @ 2011-02-06 23:34 UTC (permalink / raw)
To: Andy <angelflow@yahoo.com>; +Cc: pgsql-performance
ah ok, i was not sure about this point, thanks.
Regards,
Miguel Angel
El 06/02/11 21:38, Andy escribió:
>
> --- On Sun, 2/6/11, Linos<info@linos.es> wrote:
>
>> I am studying too the possibility of use an OCZ Vertex 2
>> Pro with Flashcache or Bcache to use it like a second level
>> filesystem cache, any comments on that please?
>>
>
> OCZ Vertex 2 Pro is a lot more expensive than other SSD of comparable performances because it comes with a supercapacitor that guarantees durability.
>
> If you're just using the SSD as a cache, you don't need durability. You could save a lot of money by getting SSDs without supercapacitor, such as Corsair Force.
>
>
>
>
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2011-02-06 23:34 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2011-02-06 18:16 general hardware advice Linos <info@linos.es>
2011-02-06 19:05 ` Scott Marlowe <scott.marlowe@gmail.com>
2011-02-06 19:31 ` Linos <info@linos.es>
2011-02-06 19:24 ` Pierre C <lists@peufeu.com>
2011-02-06 19:34 ` Linos <info@linos.es>
2011-02-06 20:38 ` Andy <angelflow@yahoo.com>
2011-02-06 23:34 ` Linos <info@linos.es>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox