pg.ddx.io pgsql-performance@postgresql.org mailing list archivehelp / color / mirror / Atom feed
OT (slightly) testing for data loss on an SSD drive due to power failure 2+ messages / 2 participants [nested] [flat]
* OT (slightly) testing for data loss on an SSD drive due to power failure @ 2011-04-22 14:04 John Rouillard <rouilj@renesys.com> 0 siblings, 1 reply; 2+ messages in thread From: John Rouillard @ 2011-04-22 14:04 UTC (permalink / raw) To: pgsql-performance Hi all: I realize this is slightly off topic, but is an issue of concern with the use of ssd's. We are setting up a storage server under solaris using ZFS. We have a couple of ssd's 2 x 160GB Intel X25-M MLC SATA acting as the zil (write journal) and are trying to see if it is safe to use for a power fail situation. Our testing (10 runs) hasn't shown any data loss, but I am not sure our testing has been running long enough and is valid, so I hoped the people here who have tested an ssd for data loss may have some guidance. The testing method is to copy a bunch of files over NFS to the server with the zil. When the copy is running along, pull the power to the server. The NFS client will stop and if the client got a message that block X was written safely to the zil, it will continue writing with block x+1. After the server comes backup and and the copies resume/finish the files are checksummed. If block X went missing, the checksums will fail and we will have our proof. We are looking at how to max out the writes to the SSD on the theory that we need to fill the dram buffer on the SSD and get it saturated enough such that it can't flush data to permanent storage as fast as the data is coming in. (I.E. make it a writeback with a longer delay so it's more likely to drop data.) Does anybody have any comments or testing methodologies that don't involve using an actual postgres instance? Thanks for your help. -- -- rouilj John Rouillard System Administrator Renesys Corporation 603-244-9084 (cell) 603-643-9300 x 111 ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: OT (slightly) testing for data loss on an SSD drive due to power failure @ 2011-04-23 02:48 Greg Smith <greg@2ndQuadrant.com> parent: John Rouillard <rouilj@renesys.com> 0 siblings, 0 replies; 2+ messages in thread From: Greg Smith @ 2011-04-23 02:48 UTC (permalink / raw) To: pgsql-performance On 04/22/2011 10:04 AM, John Rouillard wrote: > We have a couple of ssd's 2 x 160GB Intel X25-M MLC SATA > acting as the zil (write journal) and are trying to see if it is safe > to use for a power fail situation. > Well, the quick answer is "no". I've lost several weekends of my life to recovering information from database stored on those drivers, after they were corrupted in a crash. > The testing method is to copy a bunch of files over NFS to the server > with the zil. When the copy is running along, pull the power to the > server. The NFS client will stop and if the client got a message that > block X was written safely to the zil, it will continue writing with > block x+1. After the server comes backup and and the copies > resume/finish the files are checksummed. If block X went missing, the > checksums will fail and we will have our proof. > Interestingly, you have reinvented parts of the standard script for testing for data loss, diskchecker.pl: http://brad.livejournal.com/2116715.html You can get a few thousand commits per second using that program, which is enough to fill the drive buffer such that a power pull should sometimes lose something. I don't think you can do a proper test here using NFS; you really need something that is executing fsync calls directly in the same pattern a database server will. ZFS is more resilient than most filesystem as far as avoiding file corruption in this case. But you should still be able to find some missing transactions that are sitting in the drive cache. -- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2011-04-23 02:48 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2011-04-22 14:04 OT (slightly) testing for data loss on an SSD drive due to power failure John Rouillard <rouilj@renesys.com> 2011-04-23 02:48 ` Greg Smith <greg@2ndQuadrant.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox