pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedcompact flash disks?
14+ messages / 8 participants
[nested] [flat]
* compact flash disks?
@ 2007-03-06 22:18 James Mansion <james@mansionfamily.plus.com>
2007-03-07 02:12 ` Re: compact flash disks? Merlin Moncure <mmoncure@gmail.com>
2007-03-07 09:34 ` Re: compact flash disks? Florian Weimer <fweimer@bfk.de>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-07 16:12 ` Re: compact flash disks? cedric <cedric@over-blog.com>
0 siblings, 4 replies; 14+ messages in thread
From: James Mansion @ 2007-03-06 22:18 UTC (permalink / raw)
To: pgsql-performance
I see that one can now get compact flash to SATA connectors.
If I were to use a filesystem with noatime etc and little non-sql traffic,
does the physical update pattern tend to have hot sectors that will tend to
wear out CF?
I'm wondering about a RAID5 with data on CF drives and RAID1 for teh WAL on
a fast SATA or SAS drive pair. I'm thhinking that this would tend to have
good performance because the seek time for the data is very low, even if the
actual write speed can be slower than state of the art. 2GB CF isn't so
pricey any more.
Just wondering.
James
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.446 / Virus Database: 268.18.7/711 - Release Date: 05/03/2007
09:41
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
@ 2007-03-07 02:12 ` Merlin Moncure <mmoncure@gmail.com>
3 siblings, 0 replies; 14+ messages in thread
From: Merlin Moncure @ 2007-03-07 02:12 UTC (permalink / raw)
To: James Mansion <james@mansionfamily.plus.com>; +Cc: pgsql-performance
On 3/7/07, James Mansion <james@mansionfamily.plus.com> wrote:
> I see that one can now get compact flash to SATA connectors.
>
> If I were to use a filesystem with noatime etc and little non-sql traffic,
> does the physical update pattern tend to have hot sectors that will tend to
> wear out CF?
>
> I'm wondering about a RAID5 with data on CF drives and RAID1 for teh WAL on
> a fast SATA or SAS drive pair. I'm thhinking that this would tend to have
> good performance because the seek time for the data is very low, even if the
> actual write speed can be slower than state of the art. 2GB CF isn't so
> pricey any more.
>
> Just wondering.
me too. I think if you were going to do this I would configure as
raid 0. Sequential performance might be a problem, and traditional
hard drive failure is not. I think some of the better flash drives
spread out the writes so that life is maximized.
It's still probably cheaper buying a better motherboard and stuffing
more memory in it, and a good raid controller.
merlin
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
@ 2007-03-07 09:34 ` Florian Weimer <fweimer@bfk.de>
3 siblings, 0 replies; 14+ messages in thread
From: Florian Weimer @ 2007-03-07 09:34 UTC (permalink / raw)
To: James Mansion <james@mansionfamily.plus.com>; +Cc: pgsql-performance
* James Mansion:
> If I were to use a filesystem with noatime etc and little non-sql traffic,
> does the physical update pattern tend to have hot sectors that will tend to
> wear out CF?
Thanks to the FAT file system and its update pattern, most (if not
all) CF disks implement wear leveling nowadays. I wouldn't worry
about the issue, unless your write rates are pretty high.
--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
@ 2007-03-07 15:14 ` Ron <rjpeace@earthlink.net>
2007-03-07 15:48 ` Re: compact flash disks? Carlos Moreno <moreno_pg@mochima.com>
2007-03-08 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
3 siblings, 2 replies; 14+ messages in thread
From: Ron @ 2007-03-07 15:14 UTC (permalink / raw)
To: James Mansion <james@mansionfamily.plus.com>; +Cc: pgsql-performance
At 05:18 PM 3/6/2007, James Mansion wrote:
>I see that one can now get compact flash to SATA connectors.
>
>If I were to use a filesystem with noatime etc and little non-sql traffic,
>does the physical update pattern tend to have hot sectors that will tend to
>wear out CF?
Most flash RAMs have drivers that make sure the pattern of writes
over time is uniform across the entire device.
>I'm wondering about a RAID5 with data on CF drives and RAID1 for the WAL on
>a fast SATA or SAS drive pair. I'm thinking that this would tend to have
>good performance because the seek time for the data is very low, even if the
>actual write speed can be slower than state of the art.
WARNING: modern TOtL flash RAMs are only good for ~1.2M writes per
memory cell. and that's the =good= ones.
Using flash RAM for write heavy applications like OLTP, or for WAL,
etc can be very dangerous
Flash write speeds also stink; being ~1/2 flash's already low read speed.
Much better to use flash RAM for read heavy applications.
Even there you have to be careful that seek performance, not
throughput, is what is gating your day to day performance with those tables.
Got tables or indexes that are
a= too big to fit in RAM and
b= are write few, read many times and
c= whose pattern of access is large enough that it does not cache well?
=Those= are what should be put into flash RAMs
Side Note:
In the long run, we are going to have to seriously rethink pg's use
of WAL as the way we implement MVCC as it becomes more and more of a
performance bottleneck.
We have WAL because Stonebreaker made an assumption about the future
dominance of optical media that has turned out to be false.
...and it's been one of pg's big issues every since.
> 2GB CF isn't so
>pricey any more.
Heck =16= GB Flash only costs ~$300 US and 128GB SSDs based on flash
RAM are due out this year.
Cheers,
Ron
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
@ 2007-03-07 15:48 ` Carlos Moreno <moreno_pg@mochima.com>
2007-03-07 15:49 ` Re: compact flash disks? Csaba Nagy <nagy@ecircle-ag.com>
1 sibling, 1 reply; 14+ messages in thread
From: Carlos Moreno @ 2007-03-07 15:48 UTC (permalink / raw)
To: pgsql-performance
>
> Much better to use flash RAM for read heavy applications.
> Even there you have to be careful that seek performance, not
> throughput, is what is gating your day to day performance with those
> tables.
Isn't precisely there where Flash disks would have *the* big advantage??
I mean, access time is severely held down by the *mechanical* movement of
the heads to the right cylinder on the disk --- that's a brutally large
amount of
time compared to anything else happening on the computer (well, floppy
disks aside, and things like trying-to-break-128-bit-encryption aside
:-)).
Or are these Flash disks so slow that they compare to the HD's latency
figures?
Carlos
--
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-07 15:48 ` Re: compact flash disks? Carlos Moreno <moreno_pg@mochima.com>
@ 2007-03-07 15:49 ` Csaba Nagy <nagy@ecircle-ag.com>
2007-04-04 05:18 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
0 siblings, 1 reply; 14+ messages in thread
From: Csaba Nagy @ 2007-03-07 15:49 UTC (permalink / raw)
To: Carlos Moreno <moreno_pg@mochima.com>; +Cc: pgsql-performance
> Or are these Flash disks so slow that they compare to the HD's latency
> figures?
On sequential read speed HDs outperform flash disks... only on random
access the flash disks are better. So if your application is a DW one,
you're very likely better off using HDs.
Cheers,
Csaba.
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-07 15:48 ` Re: compact flash disks? Carlos Moreno <moreno_pg@mochima.com>
2007-03-07 15:49 ` Re: compact flash disks? Csaba Nagy <nagy@ecircle-ag.com>
@ 2007-04-04 05:18 ` James Mansion <james@mansionfamily.plus.com>
0 siblings, 0 replies; 14+ messages in thread
From: James Mansion @ 2007-04-04 05:18 UTC (permalink / raw)
To: Csaba Nagy <nagy@ecircle-ag.com>; Carlos Moreno <moreno_pg@mochima.com>; rjpeace@earthlink.net; +Cc: pgsql-performance
>On sequential read speed HDs outperform flash disks... only on random
>access the flash disks are better. So if your application is a DW one,
>you're very likely better off using HDs.
This looks likely to be a non-issue shortly, see here:
http://www.reghardware.co.uk/2007/03/27/sams_doubles_ssd_capacity/
I still think this sort of devices will become the OLTP device
of choice before too long - even if we do have to watch the wear rate.
>WARNING: modern TOtL flash RAMs are only good for ~1.2M writes per
>memory cell. and that's the =good= ones.
Well, my original question was whether the physical update pattern
of the server does have hotspots that will tend to cause a problem
in normal usage if the wear levelling (such as it is) doesn't entirely
spread the load. The sorts of application I'm interested in will not
update individual data elements very often. There's a danger that
index nodes might be rewritted frequently, but one might want to allow
that indexes persist lazily and should be recovered from a scan after
a crash that leaves them dirty, so that they can be cached and avoid
such an access pattern.
Out of interest with respect to WAL - has anyone tested to see whether
one could tune the group commit to pick up slightly bigger blocks and
write the WAL using compression, to up the *effective* write speed of
media? Once again, most data I'm interested in is far from a random
pattern and tends to compress quite well.
If the WAL write is committed to the disk platter, is it OK for
arbitrary data blocks to have failed to commit to disk so we can
recover the updates for committed transactions?
Is theer any documentation on the write barrier usage?
James
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.446 / Virus Database: 268.18.25/744 - Release Date: 03/04/2007
05:32
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
@ 2007-03-08 06:24 ` James Mansion <james@mansionfamily.plus.com>
2007-03-08 08:48 ` Re: compact flash disks? Magnus Hagander <magnus@hagander.net>
1 sibling, 1 reply; 14+ messages in thread
From: James Mansion @ 2007-03-08 06:24 UTC (permalink / raw)
To: Ron <rjpeace@earthlink.net>; +Cc: pgsql-performance
>WARNING: modern TOtL flash RAMs are only good for ~1.2M writes per
>memory cell. and that's the =good= ones.
>Using flash RAM for write heavy applications like OLTP, or for WAL,
>etc can be very dangerous
Well, that's why I suggested that the WAL would stream to a hard disk
array, where the large IO sequential write speed will be helpful.
Whether OLTP is a problem will presumably depend on the freqency of updates
and vacuum to each physical cluster of rows in a disk block.
Most rows in a trading application will have quite a long lifetime, and be
updated relatively few times (even where we writing fixings info into
trades).
>Flash write speeds also stink; being ~1/2 flash's already low read speed.
Sure - but it may still be an effective tradoff where the limiting factor
would otherwise be seek time.
>Much better to use flash RAM for read heavy applications.
Why? I can get a 'PC' server with 128GB of RAM quite easily now,
and that will mean I can cache most of not all hot data for any trading
app I've worked on. Settled trades that matured in prior periods can
be moved to tables on real disks - they are hardly ever accessed
anyway.
In the long run, we are going to have to seriously rethink pg's use
of WAL as the way we implement MVCC as it becomes more and more of a
performance bottleneck.
We have WAL because Stonebreaker made an assumption about the future
dominance of optical media that has turned out to be false.
...and it's been one of pg's big issues every since.
>> 2GB CF isn't so
>>pricey any more.
>Heck =16= GB Flash only costs ~$300 US and 128GB SSDs based on flash
>RAM are due out this year.
Quite. Suppose I have a RAID with double redundancy, then I get enough
capacity
for quite a lot of raw data, and can swap a card out every weekend and let
the
RAID rebuild it in rotation to keep them within conservative wear limits.
So long as the wear levelling works moderately well (and without needing FAT
on the disk or whatever) then I should be fine.
I think. Maybe.
James
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.446 / Virus Database: 268.18.7/713 - Release Date: 07/03/2007
09:24
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-08 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
@ 2007-03-08 08:48 ` Magnus Hagander <magnus@hagander.net>
2007-03-08 14:11 ` Re: compact flash disks? Merlin Moncure <mmoncure@gmail.com>
2007-03-09 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
0 siblings, 2 replies; 14+ messages in thread
From: Magnus Hagander @ 2007-03-08 08:48 UTC (permalink / raw)
To: James Mansion <james@mansionfamily.plus.com>; +Cc: Ron <rjpeace@earthlink.net>; pgsql-performance
On Thu, Mar 08, 2007 at 06:24:35AM -0000, James Mansion wrote:
>
> In the long run, we are going to have to seriously rethink pg's use
> of WAL as the way we implement MVCC as it becomes more and more of a
> performance bottleneck.
> We have WAL because Stonebreaker made an assumption about the future
> dominance of optical media that has turned out to be false.
> ...and it's been one of pg's big issues every since.
Uh. pg didn't *have* WAL back when Stonebreaker was working on it. It
was added in PostgreSQL 7.1, by Vadim. And it significantly increased
performance at the time, since we no longer had to sync the datafiles
after every transaction commit.
(We also didn't have MVCC back in the Stonebreaker days - it was added
in 6.5)
That said, it's certainly possible that someone can find an even better
way to do it :-)
//Magnus
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-08 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-08 08:48 ` Re: compact flash disks? Magnus Hagander <magnus@hagander.net>
@ 2007-03-08 14:11 ` Merlin Moncure <mmoncure@gmail.com>
2007-03-08 18:40 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
1 sibling, 1 reply; 14+ messages in thread
From: Merlin Moncure @ 2007-03-08 14:11 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: James Mansion <james@mansionfamily.plus.com>; Ron <rjpeace@earthlink.net>; pgsql-performance
On 3/8/07, Magnus Hagander <magnus@hagander.net> wrote:
> On Thu, Mar 08, 2007 at 06:24:35AM -0000, James Mansion wrote:
> >
> > In the long run, we are going to have to seriously rethink pg's use
> > of WAL as the way we implement MVCC as it becomes more and more of a
> > performance bottleneck.
> > We have WAL because Stonebreaker made an assumption about the future
> > dominance of optical media that has turned out to be false.
> > ...and it's been one of pg's big issues every since.
>
> Uh. pg didn't *have* WAL back when Stonebreaker was working on it. It
> was added in PostgreSQL 7.1, by Vadim. And it significantly increased
> performance at the time, since we no longer had to sync the datafiles
> after every transaction commit.
> (We also didn't have MVCC back in the Stonebreaker days - it was added
> in 6.5)
Exactly, and WAL services other purposes than minimizing the penalty
from writing to high latency media. WAL underlies PITR for example.
Near-zero latency media is coming, eventually...and I don't think the
issue is reliability (catastrophic failure is extremely unlikely) but
cost. I think the poor write performance is not an issue because you
can assemble drives in a giant raid 0 (or even 00 or 000) which will
blow away disk based raid 10 systems at virtually everything.
Solid State Drives consume less power (a big deal in server farms) and
the storage density and life-span will continue to improve. I give it
five years (maybe less) before you start to see SSD penetration in a
big way. It will simply become cheaper to build a box with SSD than
without since you won't need to buy as much RAM, draws less power, and
is much more reliable.
Disk drives will displace tape as low speed archival storage but will
probably live on in super high storage enterprise environments.
my 0.02$, as usual,
merlin
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-08 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-08 08:48 ` Re: compact flash disks? Magnus Hagander <magnus@hagander.net>
2007-03-08 14:11 ` Re: compact flash disks? Merlin Moncure <mmoncure@gmail.com>
@ 2007-03-08 18:40 ` Ron <rjpeace@earthlink.net>
2007-03-08 20:24 ` Re: compact flash disks? Merlin Moncure <mmoncure@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Ron @ 2007-03-08 18:40 UTC (permalink / raw)
To: Merlin Moncure <mmoncure@gmail.com>; Magnus Hagander <magnus@hagander.net>; +Cc: pgsql-performance
At 09:11 AM 3/8/2007, Merlin Moncure wrote:
>On 3/8/07, Magnus Hagander <magnus@hagander.net> wrote:
>>On Thu, Mar 08, 2007 at 06:24:35AM -0000, James Mansion wrote:
>> >
>> > In the long run, we are going to have to seriously rethink pg's use
>> > of WAL as the way we implement MVCC as it becomes more and more of a
>> > performance bottleneck.
>> > We have WAL because Stonebreaker made an assumption about the future
>> > dominance of optical media that has turned out to be false.
>> > ...and it's been one of pg's big issues every since.
>>
>>Uh. pg didn't *have* WAL back when Stonebreaker was working on it. It
>>was added in PostgreSQL 7.1, by Vadim. And it significantly increased
>>performance at the time, since we no longer had to sync the datafiles
>>after every transaction commit.
>>(We also didn't have MVCC back in the Stonebreaker days - it was added
>>in 6.5)
Huh. I have to go re-read my references. Seems I've misremembered history.
Thanks for correcting my mistake.
>Exactly, and WAL services other purposes than minimizing the penalty
>from writing to high latency media. WAL underlies PITR for example.
Never argued with any of this.
>Near-zero latency media is coming, eventually...and I don't think the
>issue is reliability (catastrophic failure is extremely unlikely) but
>cost. I think the poor write performance is not an issue because you
>can assemble drives in a giant raid 0 (or even 00 or 000) which will
>blow away disk based raid 10 systems at virtually everything.
Have you considered what the $cost$ of that much flash RAM would be?
>Solid State Drives consume less power (a big deal in server farms) and
>the storage density and life-span will continue to improve. I give it
>five years (maybe less) before you start to see SSD penetration in a
>big way. It will simply become cheaper to build a box with SSD than
>without since you won't need to buy as much RAM, draws less power, and
>is much more reliable.
Don't bet on it. HDs 2x in density faster than RAM does, even flash
RAM, and have a -much- lower cost per bit.
...and it's going to stay that way for the foreseeable future.
ATM, I can buy a 500GB 7200 rpm SATA II HD w/ a 5 yr warranty for
~$170 US per HD.
1TB HDs of that class cost ~$350-$400 US per.
(...and bear in mind the hybrid HDs coming out later this year that
offer the best of both HD and flash technologies at very close to
current HD costs per bit.)
The 128GB flash RAM SSDs coming out later this year are going to cost
4x - 10x those HD prices...
4+ decades of history shows that initial acquisition cost is =by far=
the primary deciding factor in IT spending.
QED: SSDs are going to remain a niche product unless or until
Something Drastic (tm) happens to the economics and storage requirements of IT.
>Disk drives will displace tape as low speed archival storage but will
>probably live on in super high storage enterprise environments.
Again, don't bet on it. Tape 2x in density even faster than HD does
and has an even lower cost per bit.
>my 0.02$, as usual,
>merlin
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-08 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-08 08:48 ` Re: compact flash disks? Magnus Hagander <magnus@hagander.net>
2007-03-08 14:11 ` Re: compact flash disks? Merlin Moncure <mmoncure@gmail.com>
2007-03-08 18:40 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
@ 2007-03-08 20:24 ` Merlin Moncure <mmoncure@gmail.com>
0 siblings, 0 replies; 14+ messages in thread
From: Merlin Moncure @ 2007-03-08 20:24 UTC (permalink / raw)
To: Ron <rjpeace@earthlink.net>; +Cc: Magnus Hagander <magnus@hagander.net>; pgsql-performance
On 3/8/07, Ron <rjpeace@earthlink.net> wrote:
> >Exactly, and WAL services other purposes than minimizing the penalty
> >from writing to high latency media. WAL underlies PITR for example.
>
> Never argued with any of this.
>
>
> >Near-zero latency media is coming, eventually...and I don't think the
> >issue is reliability (catastrophic failure is extremely unlikely) but
> >cost. I think the poor write performance is not an issue because you
> >can assemble drives in a giant raid 0 (or even 00 or 000) which will
> >blow away disk based raid 10 systems at virtually everything.
> Have you considered what the $cost$ of that much flash RAM would be?
>
>
> >Solid State Drives consume less power (a big deal in server farms) and
> >the storage density and life-span will continue to improve. I give it
> >five years (maybe less) before you start to see SSD penetration in a
> >big way. It will simply become cheaper to build a box with SSD than
> >without since you won't need to buy as much RAM, draws less power, and
> >is much more reliable.
> Don't bet on it. HDs 2x in density faster than RAM does, even flash
> RAM, and have a -much- lower cost per bit.
> ...and it's going to stay that way for the foreseeable future.
yes, but SSD drives do not have to overtake hard disks to be cost
competitive. one reason for this is that a SSD based system does not
need nearly as much ram as a hard drive based system due to 99%
reduction of cache miss penalty. Also, flash drives are falling
faster in price than hard drives. Unless there is some new
breakthrough in hard drive technology the price differential is going
to narrow a bit. Ultimately (and this may be quite some way down the
road, but maybe not), SSD will be cheaper than spinning disk due to
lower cost of materials.
> ATM, I can buy a 500GB 7200 rpm SATA II HD w/ a 5 yr warranty for
> ~$170 US per HD.
> 1TB HDs of that class cost ~$350-$400 US per.
> (...and bear in mind the hybrid HDs coming out later this year that
> offer the best of both HD and flash technologies at very close to
> current HD costs per bit.)
it is not fair to compare cost of 10ms latency hard drive to 10us
flash drive unless you add in the cost of ram to get the systems to
performance parity.
> The 128GB flash RAM SSDs coming out later this year are going to cost
> 4x - 10x those HD prices...
> 4+ decades of history shows that initial acquisition cost is =by far=
> the primary deciding factor in IT spending.
> QED: SSDs are going to remain a niche product unless or until
> Something Drastic (tm) happens to the economics and storage requirements of IT.
Historically what you are saying is true but something drastic is
indeed happening.
There is currently a power crisis in many colocation facilities in the
u.s. For example, for at least 1/3 of the cages in qwest at sterling,
va are sitting empty because there are not enough circuits to give out
(qwest is hitting environmental regs so that adding any additional
generator power they would have to re-classify as a power generating
facility). This basically puts their customers in a bidding war for
available circuits and is making power draw a very significant factor
of cost of rack rental. It is not unreasonable to factor in, say,
one year of savings of monthly power bill at a facility.
some facilities are so strapped they will only give out dc power circuits...
> >Disk drives will displace tape as low speed archival storage but will
> >probably live on in super high storage enterprise environments.
>
> Again, don't bet on it. Tape 2x in density even faster than HD does
> and has an even lower cost per bit.
this is a completely different argument but disk based backup systems
are exploding as are the companies that design and implement them.
tape has problems that go way beyond performance and price. whizzing
robot arms that stick tapes in and out of things may have looked cool
and futuristic in 1960 but its time to put this technology to bed for
good.
merlin
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 15:14 ` Re: compact flash disks? Ron <rjpeace@earthlink.net>
2007-03-08 06:24 ` Re: compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-08 08:48 ` Re: compact flash disks? Magnus Hagander <magnus@hagander.net>
@ 2007-03-09 06:24 ` James Mansion <james@mansionfamily.plus.com>
1 sibling, 0 replies; 14+ messages in thread
From: James Mansion @ 2007-03-09 06:24 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Ron <rjpeace@earthlink.net>; pgsql-performance
Isn't it likely that a single stream (or perhaps one that can be partitioned
across spindles) will tend to be fastest, since it has a nice localised
stream that a) allows for compression of reasonable blocks and b) fits with
commit aggregation?
RAM capacity on servers is going up and up, but the size of a customer
address or row on an invoice isn't. I'd like to see an emphasis on speed of
update with an assumption that most hot data is cached, most of the time.
My understanding also is that storing data columnwise is handy when its
persisted because linear scans are much faster. Saw it once with a system
modelled after APL, blew me away even on a sparc10 once the data was
organised and could be mapped.
Still, for the moment anything that helps with the existing system would be
good. Would it help to define triggers to be deferrable to commit as well
as end of statement (and per row)? Seems to me it should be, at least for
ones that raise 'some thing changed' events. And/or allow specification
that events can fold and should be very cheap (don't know if this is the
case now? Its not as well documented how this works as I'd like)
James
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.446 / Virus Database: 268.18.7/713 - Release Date: 07/03/2007
09:24
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: compact flash disks?
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
@ 2007-03-07 16:12 ` cedric <cedric@over-blog.com>
3 siblings, 0 replies; 14+ messages in thread
From: cedric @ 2007-03-07 16:12 UTC (permalink / raw)
To: pgsql-performance; +Cc: James Mansion <james@mansionfamily.plus.com>
Le mardi 6 mars 2007 23:18, James Mansion a écrit :
> I see that one can now get compact flash to SATA connectors.
I can suggest you to have a look at Gigabyte i-ram .
We use it on a website with higth traffic with lot of succes. Unfortunely, I
can not provide any benchmark...
>
> If I were to use a filesystem with noatime etc and little non-sql traffic,
> does the physical update pattern tend to have hot sectors that will tend to
> wear out CF?
>
> I'm wondering about a RAID5 with data on CF drives and RAID1 for teh WAL on
> a fast SATA or SAS drive pair. I'm thhinking that this would tend to have
> good performance because the seek time for the data is very low, even if
> the actual write speed can be slower than state of the art. 2GB CF isn't
> so pricey any more.
>
> Just wondering.
>
> James
>
> --
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.5.446 / Virus Database: 268.18.7/711 - Release Date: 05/03/2007
> 09:41
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
^ permalink raw reply [nested|flat] 14+ messages in thread
end of thread, other threads:[~2007-04-04 05:18 UTC | newest]
Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-03-06 22:18 compact flash disks? James Mansion <james@mansionfamily.plus.com>
2007-03-07 02:12 ` Merlin Moncure <mmoncure@gmail.com>
2007-03-07 09:34 ` Florian Weimer <fweimer@bfk.de>
2007-03-07 15:14 ` Ron <rjpeace@earthlink.net>
2007-03-07 15:48 ` Carlos Moreno <moreno_pg@mochima.com>
2007-03-07 15:49 ` Csaba Nagy <nagy@ecircle-ag.com>
2007-04-04 05:18 ` James Mansion <james@mansionfamily.plus.com>
2007-03-08 06:24 ` James Mansion <james@mansionfamily.plus.com>
2007-03-08 08:48 ` Magnus Hagander <magnus@hagander.net>
2007-03-08 14:11 ` Merlin Moncure <mmoncure@gmail.com>
2007-03-08 18:40 ` Ron <rjpeace@earthlink.net>
2007-03-08 20:24 ` Merlin Moncure <mmoncure@gmail.com>
2007-03-09 06:24 ` James Mansion <james@mansionfamily.plus.com>
2007-03-07 16:12 ` cedric <cedric@over-blog.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox