pg.ddx.io pgsql-performance@postgresql.org mailing list archivehelp / color / mirror / Atom feed
new server I/O setup 3+ messages / 3 participants [nested] [flat]
* new server I/O setup @ 2010-01-14 20:03 Fernando Hevia <fhevia@ip-tel.com.ar> 0 siblings, 2 replies; 3+ messages in thread From: Fernando Hevia @ 2010-01-14 20:03 UTC (permalink / raw) To: pgsql-performance Hi all, I've just received this new server: 1 x XEON 5520 Quad Core w/ HT 8 GB RAM 1066 MHz 16 x SATA II Seagate Barracuda 7200.12 3ware 9650SE w/ 256MB BBU It will run an Ubuntu 8.04 LTS Postgres 8.4 dedicated server. Its database will be getting between 100 and 1000 inserts per second (those are call detail records of ~300 bytes each) of around 20 clients (voip gateways). Other activity is mostly read-only and some non time-critical writes generally at off peak hours. So my first choice was: 2 discs in RAID 1 for OS + pg_xlog partitioned with ext2. 12 discs in RAID 10 for postgres data, sole partition with ext3. 2 spares My second choice is: 4 discs in RAID 10 for OS + pg_xlog partitioned with ext2 10 discs in RAID 10 for postgres, ext3 2 spares. The bbu caché will be enabled for both raid volumes. I justified my first choice in that WAL writes are sequentially and OS pretty much are too, so a RAID 1 probably would hold ground against a 12 disc RAID 10 with random writes. I don't know in advance if I will manage to gather enough time to try out both setups so I wanted to know what you guys think of these 2 alternatives. Do you think a single RAID 1 will become a bottleneck? Feel free to suggest a better setup I hadn't considered, it would be most welcome. Pd: any clue if hdparm works to deactive the disks write cache even if they are behind the 3ware controller? Regards, Fernando. ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: new server I/O setup @ 2010-01-14 21:22 Scott Marlowe <scott.marlowe@gmail.com> parent: Fernando Hevia <fhevia@ip-tel.com.ar> 1 sibling, 0 replies; 3+ messages in thread From: Scott Marlowe @ 2010-01-14 21:22 UTC (permalink / raw) To: Fernando Hevia <fhevia@ip-tel.com.ar>; +Cc: pgsql-performance On Thu, Jan 14, 2010 at 1:03 PM, Fernando Hevia <fhevia@ip-tel.com.ar> wrote: > Hi all, > > I've just received this new server: > 1 x XEON 5520 Quad Core w/ HT > 8 GB RAM 1066 MHz > 16 x SATA II Seagate Barracuda 7200.12 > 3ware 9650SE w/ 256MB BBU > > It will run an Ubuntu 8.04 LTS Postgres 8.4 dedicated server. Its database > will be getting between 100 and 1000 inserts per second (those are call > detail records of ~300 bytes each) of around 20 clients (voip gateways). > Other activity is mostly read-only and some non time-critical writes > generally at off peak hours. > > So my first choice was: > > 2 discs in RAID 1 for OS + pg_xlog partitioned with ext2. > 12 discs in RAID 10 for postgres data, sole partition with ext3. > 2 spares > > > My second choice is: > > 4 discs in RAID 10 for OS + pg_xlog partitioned with ext2 > 10 discs in RAID 10 for postgres, ext3 > 2 spares. > > The bbu caché will be enabled for both raid volumes. > > I justified my first choice in that WAL writes are sequentially and OS > pretty much are too, so a RAID 1 probably would hold ground against a 12 > disc RAID 10 with random writes. I think your first choice is right. I use the same basic setup with 147G 15k5 SAS seagate drives and the pg_xlog / OS partition is almost never close to the same level of utilization, according to iostat, as the main 12 disk RAID-10 array is. We may have to buy a 16 disk array to keep up with load, and it would be all main data storage, and our pg_xlog main drive pair would be just fine. > I don't know in advance if I will manage to gather enough time to try out > both setups so I wanted to know what you guys think of these 2 > alternatives. Do you think a single RAID 1 will become a bottleneck? Feel > free to suggest a better setup I hadn't considered, it would be most > welcome. For 12 disks, most likely not. Especially since your load is mostly small randomish writes, not a bunch of big multi-megabyte records or anything, so the random access performance on the 12 disk RAID-10 should be your limiting factor. > Pd: any clue if hdparm works to deactive the disks write cache even if they > are behind the 3ware controller? Not sure, but I'm pretty sure the 3ware card already does the right thing and turns off the write caching. ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: new server I/O setup @ 2010-01-15 05:46 Greg Smith <greg@2ndquadrant.com> parent: Fernando Hevia <fhevia@ip-tel.com.ar> 1 sibling, 0 replies; 3+ messages in thread From: Greg Smith @ 2010-01-15 05:46 UTC (permalink / raw) To: Fernando Hevia <fhevia@ip-tel.com.ar>; +Cc: pgsql-performance Fernando Hevia wrote: > I justified my first choice in that WAL writes are sequentially and OS > pretty much are too, so a RAID 1 probably would hold ground against a > 12 disc RAID 10 with random writes. The problem with this theory is that when PostgreSQL does WAL writes and asks to sync the data, you'll probably discover all of the open OS writes that were sitting in the Linux write cache getting flushed before that happens. And that could lead to horrible performance--good luck if the database tries to do something after cron kicks off updatedb each night for example. I think there are two viable configurations you should be considering you haven't thought about: , but neither is quite what you're looking at: 2 discs in RAID 1 for OS 2 discs in RAID 1 for pg_xlog 10 discs in RAID 10 for postgres, ext3 2 spares. 14 discs in RAID 10 for everything 2 spares. Impossible to say which of the four possibilities here will work out better. I tend to lean toward the first one I listed above because it makes it very easy to monitor the pg_xlog activity (and the non-database activity) separately from everything else, and having no other writes going on makes it very unlikely that the pg_xlog will ever become a bottleneck. But if you've got 14 disks in there, it's unlikely to be a bottleneck anyway. The second config above will get you slightly better random I/O though, so for workloads that are really limited on that there's a good reason to prefer it. Also: the whole "use ext2 for the pg_xlog" idea is overrated far as I'm concerned. I start with ext3, and only if I get evidence that the drive is a bottleneck do I ever think of reverting to unjournaled writes just to get a little speed boost. In practice I suspect you'll see no benchmark difference, and will instead curse the decision the first time your server is restarted badly and it gets stuck at fsck. > Pd: any clue if hdparm works to deactive the disks write cache even if > they are behind the 3ware controller? You don't use hdparm for that sort of thing; you need to use 3ware's tw_cli utility. I believe that the individual drive caches are always disabled, but whether the controller cache is turned on or not depends on whether the card has a battery. The behavior here is kind of weird though--it changes if you're in RAID mode vs. JBOD mode, so be careful to look at what all the settings are. Some of these 3ware cards default to extremely aggressive background scanning for bad blocks too, you might have to tweak that downward too. -- Greg Smith 2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support greg@2ndQuadrant.com www.2ndQuadrant.com ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2010-01-15 05:46 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2010-01-14 20:03 new server I/O setup Fernando Hevia <fhevia@ip-tel.com.ar> 2010-01-14 21:22 ` Scott Marlowe <scott.marlowe@gmail.com> 2010-01-15 05:46 ` Greg Smith <greg@2ndquadrant.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox