pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Raw devices vs. Filesystems
17+ messages / 13 participants
[nested] [flat]

* Raw devices vs. Filesystems
@ 2004-03-29 16:37 Jaime Casanova <el_vigia_ec@hotmail.com>
  2004-03-29 17:21 ` Re: Raw devices vs. Filesystems Terry Hampton <thampton@limacorp.com>
  2004-03-29 18:28 ` Re: Raw devices vs. Filesystems Christopher Browne <cbbrowne@acm.org>
  0 siblings, 2 replies; 17+ messages in thread

From: Jaime Casanova @ 2004-03-29 16:37 UTC (permalink / raw)
  To: pgsql-admin

Can you tell me (or at least guide me to a palce where i can find the 
answer) what are the benefits of filesystems over raw devices?

And what filesystem is the best for postgresql performance?

_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE* 
http://join.msn.com/?page=features/junkmail




^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-03-29 16:37 Raw devices vs. Filesystems Jaime Casanova <el_vigia_ec@hotmail.com>
@ 2004-03-29 17:21 ` Terry Hampton <thampton@limacorp.com>
  1 sibling, 0 replies; 17+ messages in thread

From: Terry Hampton @ 2004-03-29 17:21 UTC (permalink / raw)
  To: Jaime Casanova <el_vigia_ec@hotmail.com>; +Cc: pgsql-admin


	Hello  Jaime,
	
	I think you're on the right track but have gotten some
	concepts possibly confused.

	As I remember,   the original email asked if Postgres
	could be run in a "raw" mode.   Another submitter
	told us that it can not.   ( Did I read that correctly, everyone ? )

	This means run Postgress in a raw or character mode versus
	the "standard" block mode as you would handle other files
	in a filesystem.     The primary advantage for a raw mode
	is speed.  ORACLE allows this method of operation.

	So ... your last question is now rather moot.   For best
	performance assure your setup parameters and Linux
	kernel parameters are optimized.      Help with that
	is a few keystrokes from anywhere.

				Terry

		

Jaime Casanova wrote:
> Can you tell me (or at least guide me to a palce where i can find the 
> answer) what are the benefits of filesystems over raw devices?
> 
> And what filesystem is the best for postgresql performance?
> 


-- 
Terry L. Hampton
Project Manager
LimaCorp, LLC   www.limacorp.com
513.587.1874





^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-03-29 16:37 Raw devices vs. Filesystems Jaime Casanova <el_vigia_ec@hotmail.com>
@ 2004-03-29 18:28 ` Christopher Browne <cbbrowne@acm.org>
  1 sibling, 0 replies; 17+ messages in thread

From: Christopher Browne @ 2004-03-29 18:28 UTC (permalink / raw)
  To: pgsql-admin

After takin a swig o' Arrakan spice grog, el_vigia_ec@hotmail.com ("Jaime Casanova") belched out:
> Can you tell me (or at least guide me to a palce where i can find the
> answer) what are the benefits of filesystems over raw devices?

For PostgreSQL, filesystems have the merit that you can actually use
them.  PostgreSQL doesn't support use of "raw devices."

Two major benefits of using filesystems as opposed to raw devices are
that:

a) The use of raw devices is dramatically non-portable; you have to
   reimplement data access on every platform you are trying to
   support; 

b) The use of raw devices essentially mandates that you implement
   some form of generic filesystem on top of them, which adds
   considerable complexity to your code.

Two benefits to raw devices are claimed...

c) It's faster.  But that assumes that the "cooked" filesystems are
   implemented fairly badly.  That was typically true, a dozen
   years ago, but it isn't so typical now, particularly with a
   fancy cacheing controller.

d) It guarantees application control of update ordering.  Of course,
   with a cacheing controller, or disk drives that lie to one degree
   or another, those guarantees might be gone anyways.

There are other filesystem advantages, such as

e) Shifting "cooked" data around may be as simple as a "mv," whereas
   reorganizing on raw disk requires DB-specific tools...

> And what filesystem is the best for postgresql performance?

That would depend, assortedly, on what OS you are using, what kind of
hardware you are running on, what kind of usage patterns you have, as
well as on how you define the notion of "best."

Absent of any indication of any of those things, the best that can be
said is "that depends..."
-- 
(format nil "~S@~S" "cbbrowne" "acm.org")
http://cbbrowne.com/info/languages.html
TTY Message from The-XGP at MIT-AI:
The-XGP@AI 02/59/69 02:59:69
Your XGP output is startling.



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
@ 2004-04-05 19:43 Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  0 siblings, 1 reply; 17+ messages in thread

From: Gregory S. Williamson @ 2004-04-05 19:43 UTC (permalink / raw)
  To: Christopher Browne <cbbrowne@acm.org>; pgsql-admin


No point to beating a dead horse (other than the sheer joy of the thing) since postgres does not have raw device support, but ...

raw devices, at least on solaris, are about 10 times as fast as cooked file systems for Informix. This might still be a gain for postgres' performance, but the portability issues remain.

raw device use in Informix is safer in terms of data because Informix does not ever have to use the regular file system and so issues of buffering and so on go away. My understanding -- fortunately not ever tried in the real world -- is that postgres' WAL log system is as reliable as Informix writing to raw devices.

raw devices can't be copied or tampered with with regular file tools (mv, cp etc.); this changes how backups get done but also adds a layer of insulation between valuable data and users.

Greg Williamson
DBA
GlobeXplorer LLC
-----Original Message-----
From:	Christopher Browne [mailto:cbbrowne@acm.org]
Sent:	Mon 3/29/2004 10:28 AM
To:	pgsql-admin@postgresql.org
Cc:	
Subject:	Re: [ADMIN] Raw devices vs. Filesystems
After takin a swig o' Arrakan spice grog, el_vigia_ec@hotmail.com ("Jaime Casanova") belched out:
> Can you tell me (or at least guide me to a palce where i can find the
> answer) what are the benefits of filesystems over raw devices?

For PostgreSQL, filesystems have the merit that you can actually use
them.  PostgreSQL doesn't support use of "raw devices."

Two major benefits of using filesystems as opposed to raw devices are
that:

a) The use of raw devices is dramatically non-portable; you have to
   reimplement data access on every platform you are trying to
   support; 

b) The use of raw devices essentially mandates that you implement
   some form of generic filesystem on top of them, which adds
   considerable complexity to your code.

Two benefits to raw devices are claimed...

c) It's faster.  But that assumes that the "cooked" filesystems are
   implemented fairly badly.  That was typically true, a dozen
   years ago, but it isn't so typical now, particularly with a
   fancy cacheing controller.

d) It guarantees application control of update ordering.  Of course,
   with a cacheing controller, or disk drives that lie to one degree
   or another, those guarantees might be gone anyways.

There are other filesystem advantages, such as

e) Shifting "cooked" data around may be as simple as a "mv," whereas
   reorganizing on raw disk requires DB-specific tools...

> And what filesystem is the best for postgresql performance?

That would depend, assortedly, on what OS you are using, what kind of
hardware you are running on, what kind of usage patterns you have, as
well as on how you define the notion of "best."

Absent of any indication of any of those things, the best that can be
said is "that depends..."
-- 
(format nil "~S@~S" "cbbrowne" "acm.org")
http://cbbrowne.com/info/languages.html
TTY Message from The-XGP at MIT-AI:
The-XGP@AI 02/59/69 02:59:69
Your XGP output is startling.

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend






^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
@ 2004-04-06 20:57 ` Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 17+ messages in thread

From: Chris Browne @ 2004-04-06 20:57 UTC (permalink / raw)
  To: pgsql-admin

gsw@globexplorer.com ("Gregory S. Williamson") writes:
> No point to beating a dead horse (other than the sheer joy of the
> thing) since postgres does not have raw device support, but ...  raw
> devices, at least on solaris, are about 10 times as fast as cooked
> file systems for Informix. This might still be a gain for postgres'
> performance, but the portability issues remain.

That claim seems really rather remarkable.

It implies an entirely stunning degree of inefficiency in the
implementation of filesystems on Solaris.

The amount of indirection involved in walking through i-nodes and such
is something I would expect to introduce some percentage of
performance loss, but for it to introduce overhead of over 900%
presumably implies that Sun (and/or Veritas) got something really
horribly wrong.
-- 
select 'cbbrowne' || '@' || 'cbbrowne.com';
http://www.ntlug.org/~cbbrowne/nonrdbms.html
Rules of the Evil Overlord #1. "My Legions of Terror will have helmets
with   clear    plexiglass   visors,   not    face-concealing   ones."
<http://www.eviloverlord.com/;



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
@ 2004-04-07 05:26   ` Tom Lane <tgl@sss.pgh.pa.us>
  2004-04-07 07:18     ` Re: Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  2004-04-07 13:05     ` Re: Raw devices vs. Filesystems Harald Fuchs <hf320@protecting.net>
  0 siblings, 2 replies; 17+ messages in thread

From: Tom Lane @ 2004-04-07 05:26 UTC (permalink / raw)
  To: Chris Browne <cbbrowne@acm.org>; +Cc: pgsql-admin

Chris Browne <cbbrowne@acm.org> writes:
> That claim seems really rather remarkable.
> It implies an entirely stunning degree of inefficiency in the
> implementation of filesystems on Solaris.

Solaris has a reputation for having stunning degrees of inefficiency
in a number of places :-(.  On the other hand I've also heard it praised
for its ability to survive partial hardware failures (eg, N out of M
CPUs down), so maybe that's the price you gotta pay.

But to get back to the point of this discussion: to allow PG to use raw
devices instead of filesystems, we'd first have to do a ton of
portability work (since raw disk access is nowhere standard), and
abandon our principle that Postgres does not run as root (since raw disk
access is not permitted to non-root processes by any sane sysadmin).
But that last is a mighty comforting principle to have, anytime someone
complains that their el cheapo whitebox PC locks up as soon as they
start to stress the database.  I know I'd have wasted a lot more time
chasing random hardware breakages if I couldn't say "system freezes and
filesystem corruption are Clearly Not Our Fault".

After that, we get to implement our own filesystem-equivalent management
of disk space allocation, disk I/O scheduling, etc.  Are we really
smarter than all those kernel hackers doing this for a living?  I doubt it.

After that, we get to re-optimize all the existing Postgres behaviors
that are designed to sit on top of a standard Unix buffering filesystem
layer.

After that, we might reap some performance benefits.  Or maybe not.
There's not a heck of a lot of hard evidence that we would --- and
what there is traces to twenty-year-old assumptions about disk drive
and OS behavior, which are quite unlikely to still apply today.

Personally, I have a lot of more-promising projects to pursue...

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-04-07 07:18     ` Grega Bremec <gregab@noviforum.si>
  2004-04-07 16:09       ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
  1 sibling, 1 reply; 17+ messages in thread

From: Grega Bremec @ 2004-04-07 07:18 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Chris Browne <cbbrowne@acm.org>; pgsql-admin; pgsql-performance@postgresql.org

...and on Wed, Apr 07, 2004 at 01:26:02AM -0400, Tom Lane used the keyboard:
> 
> After that, we get to implement our own filesystem-equivalent management
> of disk space allocation, disk I/O scheduling, etc.  Are we really
> smarter than all those kernel hackers doing this for a living?  I doubt it.
> 
> After that, we get to re-optimize all the existing Postgres behaviors
> that are designed to sit on top of a standard Unix buffering filesystem
> layer.
> 
> After that, we might reap some performance benefits.  Or maybe not.
> There's not a heck of a lot of hard evidence that we would --- and
> what there is traces to twenty-year-old assumptions about disk drive
> and OS behavior, which are quite unlikely to still apply today.
> 
> Personally, I have a lot of more-promising projects to pursue...
> 

Has anyone tried PostgreSQL on top of OCFS? Personally, I'm not sure it
would even work, as Oracle clearly state that OCFS was _never_ meant to
be a fully fledged UNIX filesystem with POSIX features such as correct
timestamp updates, inode changes, etc., but OCFSv2 brings some features
that might lead one into thinking they're about to make it suitable for
uses beyond that of just having Oracle databases sitting on top of it.

Furthermore, this filesystem would be a blazing one stop solution for
all replication issues PostgreSQL currently suffers from, as its main
design goal was to present "a consistent file system image across the
servers in a cluster".

Now, if both goals can be achieved in one go, hell, I'm willing to try
it out myself in an attempt to extract off of it, some performance
indicators that could be compared to other database performance tests
sent to both this and the PERFORM mailing list.

So, anyone? :)

Cheers,
-- 
    Grega Bremec
    Senior Administrator
    Noviforum Ltd., Software & Media
    http://www.noviforum.si/

^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: [PERFORM] Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
  2004-04-07 07:18     ` Re: Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
@ 2004-04-07 16:09       ` Josh Berkus <josh@agliodbs.com>
  2004-04-07 16:29         ` Re: [PERFORM] Raw devices vs. Filesystems Steve Atkins <steve@blighty.com>
  2004-04-08 04:33         ` Re: [PERFORM] Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  0 siblings, 2 replies; 17+ messages in thread

From: Josh Berkus @ 2004-04-07 16:09 UTC (permalink / raw)
  To: Grega Bremec <gregab@noviforum.si>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Chris Browne <cbbrowne@acm.org>; pgsql-admin; pgsql-performance@postgresql.org

Grega,

> Furthermore, this filesystem would be a blazing one stop solution for
> all replication issues PostgreSQL currently suffers from, as its main
> design goal was to present "a consistent file system image across the
> servers in a cluster".

Does it work, though?   Without Oracle admin tools?

> Now, if both goals can be achieved in one go, hell, I'm willing to try
> it out myself in an attempt to extract off of it, some performance
> indicators that could be compared to other database performance tests
> sent to both this and the PERFORM mailing list.

Hey, any test you wanna run is fine with us.    I'm pretty sure that OCFS 
belongs to Oracle, though, patent & copyright, so we couldn't actually use it 
in practice.

If your intention in this test is to show the superiority of raw devices, let 
me give you a reality check: barring some major corporate backing getting 
involved, we can't possibly implement our own PG-FS for database support.  We 
already have a TODO list which is far too long for our developer pool, and 
implementing a custom FS either takes a large team (OCFS) or several years of 
development (Reiser).   

Now, if you know somebody who might pay for one, then great ....

-- 
Josh Berkus
Aglio Database Solutions
San Francisco



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: [PERFORM] Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
  2004-04-07 07:18     ` Re: Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  2004-04-07 16:09       ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
@ 2004-04-07 16:29         ` Steve Atkins <steve@blighty.com>
  1 sibling, 0 replies; 17+ messages in thread

From: Steve Atkins @ 2004-04-07 16:29 UTC (permalink / raw)
  To: pgsql-admin; +Cc: pgsql-performance@postgresql.org

On Wed, Apr 07, 2004 at 09:09:16AM -0700, Josh Berkus wrote:

> If your intention in this test is to show the superiority of raw devices, let 
> me give you a reality check: barring some major corporate backing getting 
> involved, we can't possibly implement our own PG-FS for database support.  We 
> already have a TODO list which is far too long for our developer pool, and 
> implementing a custom FS either takes a large team (OCFS) or several years of 
> development (Reiser).   

Is there any documentation as to what guarantees PostgreSQL requires
from the filesystem, or what posix semantics can be relaxed?

Cheers,
  Steve



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: [PERFORM] Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
  2004-04-07 07:18     ` Re: Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  2004-04-07 16:09       ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
@ 2004-04-08 04:33         ` Grega Bremec <gregab@noviforum.si>
  2004-04-09 16:02           ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
  1 sibling, 1 reply; 17+ messages in thread

From: Grega Bremec @ 2004-04-08 04:33 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-admin; pgsql-performance@postgresql.org

...and on Wed, Apr 07, 2004 at 09:09:16AM -0700, Josh Berkus used the keyboard:
> 
> Does it work, though?   Without Oracle admin tools?

Hello, Josh. :)

Well, as I said, that's why I was asking - I'm willing to give it a go
if nobody can prove me wrong. :)

> > Now, if both goals can be achieved in one go, hell, I'm willing to try
> > it out myself in an attempt to extract off of it, some performance
> > indicators that could be compared to other database performance tests
> > sent to both this and the PERFORM mailing list.
> 
> Hey, any test you wanna run is fine with us.    I'm pretty sure that OCFS 
> belongs to Oracle, though, patent & copyright, so we couldn't actually use it 
> in practice.

I thought you knew - OCFS, OCFS-Tools and OCFSv2 have not only been open-
source for quite a while now - they're released under the GPL.

    http://oss.oracle.com/projects/ocfs/
    http://oss.oracle.com/projects/ocfs-tools/
    http://oss.oracle.com/projects/ocfs2/

I don't know what that means to you (probably nothing good, as PostgreSQL
is released under the BSD license), but it most definitely can be considered
a good thing for the end user, as she can download it, compile, and set it
up on her disks, without the need to pay Oracle royalties. :)

> If your intention in this test is to show the superiority of raw devices, let 
> me give you a reality check: barring some major corporate backing getting 
> involved, we can't possibly implement our own PG-FS for database support.  We 
> already have a TODO list which is far too long for our developer pool, and 
> implementing a custom FS either takes a large team (OCFS) or several years of 
> development (Reiser).   

Not really - I was just thinking about something not-entirely-a-filesystem
and POK!, OCFS sprang to mind. It omits many POSIX features that slow down
a traditional filesystem, yet it does know the concept of inodes and most
of all, it's _really_ heavy on caching. As such, it sounded quite promising
to me, but trial, I think, is the best test.

The question does spring up though, that Steve raised in another post - just
for the record, what POSIX semantics can a postmaster live without in a
filesystem?

Cheers,
-- 
    Grega Bremec
    Senior Administrator
    Noviforum Ltd., Software & Media
    http://www.noviforum.si/

^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: [PERFORM] Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
  2004-04-07 07:18     ` Re: Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  2004-04-07 16:09       ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
  2004-04-08 04:33         ` Re: [PERFORM] Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
@ 2004-04-09 16:02           ` Josh Berkus <josh@agliodbs.com>
  2004-04-09 19:34             ` Re: [PERFORM] Raw devices vs. Filesystems Christopher Browne <cbbrowne@acm.org>
  0 siblings, 1 reply; 17+ messages in thread

From: Josh Berkus @ 2004-04-09 16:02 UTC (permalink / raw)
  To: Grega Bremec <gregab@noviforum.si>; +Cc: pgsql-admin; pgsql-performance@postgresql.org

Grega,

> Well, as I said, that's why I was asking - I'm willing to give it a go
> if nobody can prove me wrong. :)

Why not?   If you have time?

> I thought you knew - OCFS, OCFS-Tools and OCFSv2 have not only been open-
> source for quite a while now - they're released under the GPL.

Keen!   Wonder if we can make them regret it.

Seriously, if Oracle opened this stuff, it's probably becuase they used some 
GPL components in it.   It also probably means that it won't work for 
anything but Oracle ...

> I don't know what that means to you (probably nothing good, as PostgreSQL
> is released under the BSD license), 

Well, it just means that we can't ship OCFS with PostgreSQL.  

> The question does spring up though, that Steve raised in another post -
> just for the record, what POSIX semantics can a postmaster live without in
> a filesystem?

You might want to ask that question again on Hackers.  I don't know the 
answer, myself.

-- 
Josh Berkus
Aglio Database Solutions
San Francisco



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: [PERFORM] Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
  2004-04-07 07:18     ` Re: Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  2004-04-07 16:09       ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
  2004-04-08 04:33         ` Re: [PERFORM] Raw devices vs. Filesystems Grega Bremec <gregab@noviforum.si>
  2004-04-09 16:02           ` Re: [PERFORM] Raw devices vs. Filesystems Josh Berkus <josh@agliodbs.com>
@ 2004-04-09 19:34             ` Christopher Browne <cbbrowne@acm.org>
  0 siblings, 0 replies; 17+ messages in thread

From: Christopher Browne @ 2004-04-09 19:34 UTC (permalink / raw)
  To: pgsql-admin

josh@agliodbs.com (Josh Berkus) wrote:
>> Well, as I said, that's why I was asking - I'm willing to give it a go
>> if nobody can prove me wrong. :)
>
> Why not?   If you have time?

True enough.

>> I thought you knew - OCFS, OCFS-Tools and OCFSv2 have not only been
>> open- source for quite a while now - they're released under the
>> GPL.
>
> Keen!  Wonder if we can make them regret it.
>
> Seriously, if Oracle opened this stuff, it's probably becuase they
> used some GPL components in it.  It also probably means that it
> won't work for anything but Oracle ...

It could be that the experiment shows that OCFS isn't all that
helpful.  Or that it helps cover inadequacies in certain aspects of
how Oracle accesses filesystems.

If it _does_ show that it is helpful, then that may suggest a
filesystem implementation strategy useful for the BSD folks.

The main "failure case" would be if the exercise shows that using OCFS
is pretty futile.
-- 
select 'cbbrowne' || '@' || 'acm.org';
http://www3.sympatico.ca/cbbrowne/linux.html
Do you know where your towel is?



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 20:57 ` Re: Raw devices vs. Filesystems Chris Browne <cbbrowne@acm.org>
  2004-04-07 05:26   ` Re: Raw devices vs. Filesystems Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-04-07 13:05     ` Harald Fuchs <hf320@protecting.net>
  1 sibling, 0 replies; 17+ messages in thread

From: Harald Fuchs @ 2004-04-07 13:05 UTC (permalink / raw)
  To: pgsql-admin

In article <5719.1081315562@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> writes:

> But to get back to the point of this discussion: to allow PG to use raw
> devices instead of filesystems, we'd first have to do a ton of
> portability work (since raw disk access is nowhere standard), and
> abandon our principle that Postgres does not run as root (since raw disk
> access is not permitted to non-root processes by any sane sysadmin).

Why not?  In MySQL/InnoDB, you do a "chown mysql.daemon /dev/raw/raw1"
(or whatever raw disk you want to access), and that's all.

> After that, we get to implement our own filesystem-equivalent management
> of disk space allocation, disk I/O scheduling, etc.  Are we really
> smarter than all those kernel hackers doing this for a living?  I doubt it.

Ditto.  I don't have hard numbers for MySQL, but I didn't see any
noticeable improvement when messing with raw disks (at least under
Linux).




^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
@ 2004-04-06 21:23 Gregory S. Williamson <gsw@globexplorer.com>
  2004-04-06 22:46 ` Re: Raw devices vs. Filesystems scott.marlowe <scott.marlowe@ihs.com>
  2004-04-07 02:56 ` Re: Raw devices vs. Filesystems Marsh Ray <marsh-pg@mysteray.com>
  0 siblings, 2 replies; 17+ messages in thread

From: Gregory S. Williamson @ 2004-04-06 21:23 UTC (permalink / raw)
  To: Chris Browne <cbbrowne@acm.org>; pgsql-admin

Remarkable, perhaps, to you. Not in the Informix world. But irrelevant to postgres, no ?

-----Original Message-----
From: Chris Browne [mailto:cbbrowne@acm.org]
Sent: Tuesday, April 06, 2004 1:57 PM
To: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Raw devices vs. Filesystems


gsw@globexplorer.com ("Gregory S. Williamson") writes:
> No point to beating a dead horse (other than the sheer joy of the
> thing) since postgres does not have raw device support, but ...  raw
> devices, at least on solaris, are about 10 times as fast as cooked
> file systems for Informix. This might still be a gain for postgres'
> performance, but the portability issues remain.

That claim seems really rather remarkable.

It implies an entirely stunning degree of inefficiency in the
implementation of filesystems on Solaris.

The amount of indirection involved in walking through i-nodes and such
is something I would expect to introduce some percentage of
performance loss, but for it to introduce overhead of over 900%
presumably implies that Sun (and/or Veritas) got something really
horribly wrong.
-- 
select 'cbbrowne' || '@' || 'cbbrowne.com';
http://www.ntlug.org/~cbbrowne/nonrdbms.html
Rules of the Evil Overlord #1. "My Legions of Terror will have helmets
with   clear    plexiglass   visors,   not    face-concealing   ones."
<http://www.eviloverlord.com/;

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster



^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-04-06 21:23 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
@ 2004-04-06 22:46 ` scott.marlowe <scott.marlowe@ihs.com>
  1 sibling, 0 replies; 17+ messages in thread

From: scott.marlowe @ 2004-04-06 22:46 UTC (permalink / raw)
  To: Gregory S. Williamson <gsw@globexplorer.com>; +Cc: Chris Browne <cbbrowne@acm.org>; pgsql-admin

Note that the innefficiency could well lie with Informix's file system 
interfacing as easily as it could lie with the operating system.  Do they 
charge extra for being able to access raw devices or somehow make more 
money by supporting them?  If so, there could be a clear business case for 
lots of uwaits() in the code path that handles file systems.

I'm just saying it's a possibility.

On Tue, 6 Apr 2004, Gregory S. Williamson wrote:

> Remarkable, perhaps, to you. Not in the Informix world. But irrelevant to postgres, no ?
> 
> -----Original Message-----
> From: Chris Browne [mailto:cbbrowne@acm.org]
> Sent: Tuesday, April 06, 2004 1:57 PM
> To: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] Raw devices vs. Filesystems
> 
> 
> gsw@globexplorer.com ("Gregory S. Williamson") writes:
> > No point to beating a dead horse (other than the sheer joy of the
> > thing) since postgres does not have raw device support, but ...  raw
> > devices, at least on solaris, are about 10 times as fast as cooked
> > file systems for Informix. This might still be a gain for postgres'
> > performance, but the portability issues remain.
> 
> That claim seems really rather remarkable.
> 
> It implies an entirely stunning degree of inefficiency in the
> implementation of filesystems on Solaris.
> 
> The amount of indirection involved in walking through i-nodes and such
> is something I would expect to introduce some percentage of
> performance loss, but for it to introduce overhead of over 900%
> presumably implies that Sun (and/or Veritas) got something really
> horribly wrong.
> 




^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
  2004-04-06 21:23 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
@ 2004-04-07 02:56 ` Marsh Ray <marsh-pg@mysteray.com>
  1 sibling, 0 replies; 17+ messages in thread

From: Marsh Ray @ 2004-04-07 02:56 UTC (permalink / raw)
  To: Gregory S. Williamson <gsw@globexplorer.com>; +Cc: Chris Browne <cbbrowne@acm.org>; pgsql-admin

 > gsw@globexplorer.com ("Gregory S. Williamson") writes:
 >>No point to beating a dead horse (other than the sheer joy of the
 >>thing) since postgres does not have raw device support, but ...  raw
 >>devices, at least on solaris, are about 10 times as fast as cooked
 >>file systems for Informix. This might still be a gain for postgres'
 >>performance, but the portability issues remain.

 > From: Chris Browne [mailto:cbbrowne@acm.org]
 > That claim seems really rather remarkable.
 > It implies an entirely stunning degree of inefficiency in the
 > implementation of filesystems on Solaris.
 > The amount of indirection involved in walking through i-nodes and such
 > is something I would expect to introduce some percentage of
 > performance loss, but for it to introduce overhead of over 900%
 > presumably implies that Sun (and/or Veritas) got something really
 > horribly wrong.

Gregory S. Williamson wrote:
> Remarkable, perhaps, to you. Not in the Informix world. But
 > irrelevant to postgres, no ?

I too am a little surprised by those numbers, but I think the potential 
for a performance gain of that order is relevant.

As I once heard someone remark: "When show up at a pool hall talking 
those kind of odds, well, people start making phone calls."

- Marsh




^ permalink  raw  reply  [nested|flat] 17+ messages in thread

* Re: Raw devices vs. Filesystems
@ 2004-04-07 16:46 Murthy Kambhampaty <murthy.kambhampaty@goeci.com>
  0 siblings, 0 replies; 17+ messages in thread

From: Murthy Kambhampaty @ 2004-04-07 16:46 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; Chris Browne <cbbrowne@acm.org>; +Cc: pgsql-admin

On Wednesday, April 07, 2004 1:26 AM Tom Lane wrote:

> 
> But to get back to the point of this discussion: to allow PG 
> to use raw devices instead of filesystems, we'd first have to do a ton of
> portability work 
...

[The following is said in a low, tentative voice :) ]

I wonder if writing the postgresql data structures as HDF5 data structures (http://hdf.ncsa.uiuc.edu/whatishdf5.html) within a single HDF5 file (perhaps the WAL files would still reside elsewhere) would improve performance while allowing HDF5 to handle portability, and other useful features, is a better solution than the relying on filesystem features.

HDF5 actually provides an added portability advantage that postgresql does not currently enjoy:
"a completely portable file format, so that a file can be written on any system and read on any other"
(See http://hdf.ncsa.uiuc.edu/HDF5/RD100-2002/All_About_HDF5.pdf).
The HDF5 "distribution" includes tools for dumping data structures, etc. so if you're hooked on filesystem level operations, you have the ability to inspect postgresql data structures within the HDF5 file, i.e., "outside postgresql".

HDF5's is also designed for clustered/grid computing systems:
"The HDF5 format and library provide a powerful means of organizing and accessing data in a manner that allows scientists to share, process, and manipulate data in today's heterogeneous and quickly-evolving high-performance computational environment, including the emerging computational GRIDs." (http://hdf.ncsa.uiuc.edu/HDF5/RD100-2002/All_About_HDF5.pdf, p. 3).
So, the main purpose of this post is to suggest that HDF5's design moves a postgresql version built on a HDF5 datastore that much closer to being ready for cluster-computing environments, with respect to the datastore (there's still the shared memory, etc., that need to be addressed, but ...).

We're playing with HDF5 from Python (see the pytables project) for our "analytics" work, but that requires moving data out of postgresql. I suspect that an SQL interface to HDF5 data structures using postgresql would be a lot more convenient, and that postgresql would gain multiple benefits from having all its data structures in a single HDF5 file. OTOH, maybe us analytics types are better off with Python over HDF5 and "postgresql on HDF5" is not a net win for postgresql. Still, there seems to a great advantage to having rich data structures to operate on rather than just "files", and allowing the HDF5 library to deal with portability, I/O efficiency, and clustering.

Hope my $0.02 worth was.

Cheers,
	Murthy



^ permalink  raw  reply  [nested|flat] 17+ messages in thread


end of thread, other threads:[~2004-04-09 19:34 UTC | newest]

Thread overview: 17+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-03-29 16:37 Raw devices vs. Filesystems Jaime Casanova <el_vigia_ec@hotmail.com>
2004-03-29 17:21 ` Terry Hampton <thampton@limacorp.com>
2004-03-29 18:28 ` Christopher Browne <cbbrowne@acm.org>
2004-04-05 19:43 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
2004-04-06 20:57 ` Chris Browne <cbbrowne@acm.org>
2004-04-07 05:26   ` Tom Lane <tgl@sss.pgh.pa.us>
2004-04-07 07:18     ` Grega Bremec <gregab@noviforum.si>
2004-04-07 16:09       ` Josh Berkus <josh@agliodbs.com>
2004-04-07 16:29         ` Steve Atkins <steve@blighty.com>
2004-04-08 04:33         ` Grega Bremec <gregab@noviforum.si>
2004-04-09 16:02           ` Josh Berkus <josh@agliodbs.com>
2004-04-09 19:34             ` Christopher Browne <cbbrowne@acm.org>
2004-04-07 13:05     ` Harald Fuchs <hf320@protecting.net>
2004-04-06 21:23 Re: Raw devices vs. Filesystems Gregory S. Williamson <gsw@globexplorer.com>
2004-04-06 22:46 ` scott.marlowe <scott.marlowe@ihs.com>
2004-04-07 02:56 ` Marsh Ray <marsh-pg@mysteray.com>
2004-04-07 16:46 Re: Raw devices vs. Filesystems Murthy Kambhampaty <murthy.kambhampaty@goeci.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox