pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedRunning on an NFS Mounted Directory
19+ messages / 7 participants
[nested] [flat]
* Running on an NFS Mounted Directory
@ 2006-04-27 02:06 Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
0 siblings, 1 reply; 19+ messages in thread
From: Ketema Harris @ 2006-04-27 02:06 UTC (permalink / raw)
To: pgsql-performance
I was wondering if there were any performance issues with having a data
directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
anyone done this before?
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 02:35 ` Steve Wampler <swampler@noao.edu>
2006-04-27 04:43 ` Re: Running on an NFS Mounted Directory Dan Gorman <dgorman@hi5.com>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
0 siblings, 2 replies; 19+ messages in thread
From: Steve Wampler @ 2006-04-27 02:35 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: pgsql-performance
On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
> I was wondering if there were any performance issues with having a data
> directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
> anyone done this before?
My understanding is that NFS is pretty poor in performance in general,
so I would expect it to be particularly bad for a DB. You might run
some (non-DB) performance tests to get a feel for how bad it might me.
(Someone once told me that NFS topped out at around 12MB/s, but I don't
know if that's really true [they were trying to sell a competitive
networked filesystem]).
In any event, you're at least limited by ethernet speeds, if not more.
--
Steve Wampler -- swampler@noao.edu
The gods that smiled on your birth are now laughing out loud.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
@ 2006-04-27 04:43 ` Dan Gorman <dgorman@hi5.com>
1 sibling, 0 replies; 19+ messages in thread
From: Dan Gorman @ 2006-04-27 04:43 UTC (permalink / raw)
To: swampler@noao.edu; +Cc: Ketema Harris <ketema@gmail.com>; pgsql-performance
We have gotten very good performance from netapp and postgres 7.4.11 .
I was able to push about 100MB/s over gigE, but that was limited by
our netapp.
DAS will generally always be faster, but if for example you have 2
disks vs. 100 NFS mounted ,NFS will be faster.
NFS is very reliable and I would stay away from iscsi.
Regards,
Dan Gorman
On Apr 26, 2006, at 7:35 PM, Steve Wampler wrote:
> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
>> I was wondering if there were any performance issues with having a
>> data
>> directory that was an nfs mounted drive? Say like a SAN or NAS
>> device? Has
>> anyone done this before?
>
> My understanding is that NFS is pretty poor in performance in general,
> so I would expect it to be particularly bad for a DB. You might run
> some (non-DB) performance tests to get a feel for how bad it might me.
> (Someone once told me that NFS topped out at around 12MB/s, but I
> don't
> know if that's really true [they were trying to sell a competitive
> networked filesystem]).
>
> In any event, you're at least limited by ethernet speeds, if not more.
>
> --
> Steve Wampler -- swampler@noao.edu
> The gods that smiled on your birth are now laughing out loud.
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
@ 2006-04-27 04:55 ` Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-05-02 14:58 ` Re: Running on an NFS Mounted Directory Fortuitous Technologies <null@fortuitous.dot.nul>
1 sibling, 2 replies; 19+ messages in thread
From: Jim C. Nasby @ 2006-04-27 04:55 UTC (permalink / raw)
To: Steve Wampler <swampler@noao.edu>; +Cc: Ketema Harris <ketema@gmail.com>; pgsql-performance
On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote:
> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
> > I was wondering if there were any performance issues with having a data
> > directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
> > anyone done this before?
>
> My understanding is that NFS is pretty poor in performance in general,
> so I would expect it to be particularly bad for a DB. You might run
> some (non-DB) performance tests to get a feel for how bad it might me.
> (Someone once told me that NFS topped out at around 12MB/s, but I don't
> know if that's really true [they were trying to sell a competitive
> networked filesystem]).
>
> In any event, you're at least limited by ethernet speeds, if not more.
More importantly, the latency involved will kill commit performance. If
it doesn't then it's likely that fsync isn't being obeyed, which means 0
data integrity.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
@ 2006-04-27 12:38 ` Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
1 sibling, 1 reply; 19+ messages in thread
From: Ketema Harris @ 2006-04-27 12:38 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; Steve Wampler <swampler@noao.edu>; +Cc: pgsql-performance
I am looking for the best solution to have a large amount of disk storage
attached to my PostgreSQL 8.1 server. I was thinking of having a san or nas
attached device be mounted by the pg server over nfs, hence the question
about nfs performance. What other options/protocols are there to get high
performance and data integrity while having the benefit of not having the
physical storage attached to the db server?
On 4/27/06 12:55 AM, "Jim C. Nasby" <jnasby@pervasive.com> wrote:
> On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote:
>> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
>>> I was wondering if there were any performance issues with having a data
>>> directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
>>> anyone done this before?
>>
>> My understanding is that NFS is pretty poor in performance in general,
>> so I would expect it to be particularly bad for a DB. You might run
>> some (non-DB) performance tests to get a feel for how bad it might me.
>> (Someone once told me that NFS topped out at around 12MB/s, but I don't
>> know if that's really true [they were trying to sell a competitive
>> networked filesystem]).
>>
>> In any event, you're at least limited by ethernet speeds, if not more.
>
> More importantly, the latency involved will kill commit performance. If
> it doesn't then it's likely that fsync isn't being obeyed, which means 0
> data integrity.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 12:44 ` Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Michael Stone @ 2006-04-27 12:44 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: pgsql-performance
On Thu, Apr 27, 2006 at 08:38:55AM -0400, Ketema Harris wrote:
>I am looking for the best solution to have a large amount of disk storage
>attached to my PostgreSQL 8.1 server.
>What other options/protocols are there to get high performance and data
>integrity while having the benefit of not having the physical storage
>attached to the db server?
These are two distinct requirements. Are both really requirements or is
one "nice to have"? The "best" solution for "a large amount of disk
storage" isn't "not having the physical storage attached to the db
server". If you use non-local storage it will be slower and more
expensive, quite likely by a large margin. There may be other advantages
to doing so, but you haven't mentioned any of those as requirements.
Mike Stone
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
@ 2006-04-27 12:57 ` Ketema Harris <ketema@gmail.com>
2006-04-27 13:05 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:09 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 13:24 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
0 siblings, 3 replies; 19+ messages in thread
From: Ketema Harris @ 2006-04-27 12:57 UTC (permalink / raw)
To: Michael Stone <mstone+postgres@mathom.us>; +Cc: pgsql-performance
OK. My thought process was that having non local storage as say a big raid
5 san ( I am talking 5 TB with expansion capability up to 10 ) would allow
me to have redundancy, expandability, and hopefully still retain decent
performance from the db. I also would hopefully then not have to do
periodic backups from the db server to some other type of storage. Is this
not a good idea? How bad of a performance hit are we talking about? Also,
in regards to the commit data integrity, as far as the db is concerned once
the data is sent to the san or nas isn't it "written"? The storage may have
that write in cache, but from my reading and understanding of how these
various storage devices work that is how they keep up performance. I would
expect my bottleneck if any to be the actual Ethernet transfer to the
storage, and I am going to try and compensate for that with a full gigabit
backbone.
On 4/27/06 8:44 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
> On Thu, Apr 27, 2006 at 08:38:55AM -0400, Ketema Harris wrote:
>> I am looking for the best solution to have a large amount of disk storage
>> attached to my PostgreSQL 8.1 server.
>
>> What other options/protocols are there to get high performance and data
>> integrity while having the benefit of not having the physical storage
>> attached to the db server?
>
> These are two distinct requirements. Are both really requirements or is
> one "nice to have"? The "best" solution for "a large amount of disk
> storage" isn't "not having the physical storage attached to the db
> server". If you use non-local storage it will be slower and more
> expensive, quite likely by a large margin. There may be other advantages
> to doing so, but you haven't mentioned any of those as requirements.
>
> Mike Stone
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 13:05 ` Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:06 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2 siblings, 1 reply; 19+ messages in thread
From: Bruno Wolff III @ 2006-04-27 13:05 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
On Thu, Apr 27, 2006 at 08:57:51 -0400,
Ketema Harris <ketema@gmail.com> wrote:
> performance from the db. I also would hopefully then not have to do
> periodic backups from the db server to some other type of storage. Is this
> not a good idea? How bad of a performance hit are we talking about? Also,
You always need to do backups if you care about your data. What if someone
accidental deletes a lot of data? What if someone blows up your data
center (or there is a flood)?
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:05 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
@ 2006-04-27 13:06 ` Ketema Harris <ketema@gmail.com>
2006-04-27 13:31 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
0 siblings, 1 reply; 19+ messages in thread
From: Ketema Harris @ 2006-04-27 13:06 UTC (permalink / raw)
To: Bruno Wolff III <bruno@wolff.to>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
Yes, your right, I meant not have to do the backups from the db server
itself. I can do that within the storage device now, by allocating space
for it, and letting the device copy the data files on some periodic basis.
On 4/27/06 9:05 AM, "Bruno Wolff III" <bruno@wolff.to> wrote:
> On Thu, Apr 27, 2006 at 08:57:51 -0400,
> Ketema Harris <ketema@gmail.com> wrote:
>> performance from the db. I also would hopefully then not have to do
>> periodic backups from the db server to some other type of storage. Is this
>> not a good idea? How bad of a performance hit are we talking about? Also,
>
> You always need to do backups if you care about your data. What if someone
> accidental deletes a lot of data? What if someone blows up your data
> center (or there is a flood)?
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:05 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:06 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 13:31 ` Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:42 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Bruno Wolff III @ 2006-04-27 13:31 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
On Thu, Apr 27, 2006 at 09:06:48 -0400,
Ketema Harris <ketema@gmail.com> wrote:
> Yes, your right, I meant not have to do the backups from the db server
> itself. I can do that within the storage device now, by allocating space
> for it, and letting the device copy the data files on some periodic basis.
Only if the database server isn't running or your SAN provides a way to
provide a snapshot of the data at a particular instant in time.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:05 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:06 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:31 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
@ 2006-04-27 13:42 ` Ketema Harris <ketema@gmail.com>
2006-04-27 18:58 ` Re: Running on an NFS Mounted Directory Dan Gorman <dgorman@hi5.com>
0 siblings, 1 reply; 19+ messages in thread
From: Ketema Harris @ 2006-04-27 13:42 UTC (permalink / raw)
To: Bruno Wolff III <bruno@wolff.to>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
The SAN has the snapshot capability.
On 4/27/06 9:31 AM, "Bruno Wolff III" <bruno@wolff.to> wrote:
> On Thu, Apr 27, 2006 at 09:06:48 -0400,
> Ketema Harris <ketema@gmail.com> wrote:
>> Yes, your right, I meant not have to do the backups from the db server
>> itself. I can do that within the storage device now, by allocating space
>> for it, and letting the device copy the data files on some periodic basis.
>
> Only if the database server isn't running or your SAN provides a way to
> provide a snapshot of the data at a particular instant in time.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:05 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:06 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:31 ` Re: Running on an NFS Mounted Directory Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:42 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 18:58 ` Dan Gorman <dgorman@hi5.com>
0 siblings, 0 replies; 19+ messages in thread
From: Dan Gorman @ 2006-04-27 18:58 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: Bruno Wolff III <bruno@wolff.to>; Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
So do NAS's
Dan
On Apr 27, 2006, at 6:42 AM, Ketema Harris wrote:
> The SAN has the snapshot capability.
>
>
> On 4/27/06 9:31 AM, "Bruno Wolff III" <bruno@wolff.to> wrote:
>
>> On Thu, Apr 27, 2006 at 09:06:48 -0400,
>> Ketema Harris <ketema@gmail.com> wrote:
>>> Yes, your right, I meant not have to do the backups from the db
>>> server
>>> itself. I can do that within the storage device now, by
>>> allocating space
>>> for it, and letting the device copy the data files on some
>>> periodic basis.
>>
>> Only if the database server isn't running or your SAN provides a
>> way to
>> provide a snapshot of the data at a particular instant in time.
>
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 13:09 ` Steve Wampler <swampler@noao.edu>
2 siblings, 0 replies; 19+ messages in thread
From: Steve Wampler @ 2006-04-27 13:09 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: Michael Stone <mstone+postgres@mathom.us>; pgsql-performance
On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote:
> OK. My thought process was that having non local storage as say a big raid
> 5 san ( I am talking 5 TB with expansion capability up to 10 ) would allow
> me to have redundancy, expandability, and hopefully still retain decent
> performance from the db. I also would hopefully then not have to do
> periodic backups from the db server to some other type of storage. Is this
> not a good idea? How bad of a performance hit are we talking about? Also,
> in regards to the commit data integrity, as far as the db is concerned once
> the data is sent to the san or nas isn't it "written"? The storage may have
> that write in cache, but from my reading and understanding of how these
> various storage devices work that is how they keep up performance. I would
> expect my bottleneck if any to be the actual Ethernet transfer to the
> storage, and I am going to try and compensate for that with a full gigabit
> backbone.
Well, if you have to have both the best performance and remote attach
storage, I think you'll find that a fibre-channel SAN is still the king
of the hill. 4Gb FC switches are common now, though finding a 4Gb
HBA for your computer might be a trick. 2Gb HBAs are everywhere in
FC land. That's a premium price solution, however, and I don't know
anything about how well PG would perform with a FC SAN. We use our
SAN for bulk science data and leave the PGDB on a separate machine
with local disk.
--
Steve Wampler -- swampler@noao.edu
The gods that smiled on your birth are now laughing out loud.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 13:24 ` Michael Stone <mstone+postgres@mathom.us>
2006-04-27 13:41 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2 siblings, 1 reply; 19+ messages in thread
From: Michael Stone @ 2006-04-27 13:24 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: pgsql-performance
On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote:
>OK. My thought process was that having non local storage as say a big raid
>5 san ( I am talking 5 TB with expansion capability up to 10 )
That's two disk trays for a cheap slow array. (Versus a more expensive
solution with more spindles and better seek performance.)
>would allow
>me to have redundancy, expandability, and hopefully still retain decent
>performance from the db. I also would hopefully then not have to do
>periodic backups from the db server to some other type of storage.
No, backups are completely unrelated to your storage type; you need them
either way. On a SAN you can use a SAN backup solution to back multiple
systems up with a single backup unit without involving the host CPUs.
This is fairly useless if you aren't amortizing the cost over a large
environment.
>Is this not a good idea?
It really depends on what you're hoping to get. As described, it's not
clear. (I don't know what you mean by "redundancy, expandability" or
"decent performance".)
>How bad of a performance hit are we talking about?
Way too many factors for an easy answer. Consider the case of NAS vs
SCSI direct attach storage. You're probably in that case comparing a
single 125MB/s (peak) gigabit ethernet channel to (potentially several)
320MB/s (peak) SCSI channels. With a high-end NAS you might get 120MB/s
off that GBE. With a (more realistic) mid-range unit you're more likely
to get 40-60MB/s. Getting 200MB/s off the SCSI channel isn't a stretch,
and you can fairly easily stripe across multiple SCSI channels. (You can
also bond multiple GBEs, but then your cost & complexity start going way
up, and you're never going to scale as well.) If you have an environment
where you're doing a lot of sequential scans it isn't even a contest.
You can also substitute SATA for SCSI, etc.
For a FC SAN the peformance numbers are a lot better, but the costs &
complexity are a lot higher. An iSCSI SAN is somewhere in the middle.
>Also, in regards to the commit data integrity, as far as the db is
>concerned once the data is sent to the san or nas isn't it "written"?
>The storage may have that write in cache, but from my reading and
>understanding of how these various storage devices work that is how
>they keep up performance.
Depends on the configuration, but yes, most should be able to report
back a "write" once the data is in a non-volatile cache. You can do the
same with a direct-attached array and eliminate the latency inherent in
accessing the remote storage.
>I would expect my bottleneck if any to be the actual Ethernet transfer
>to the storage, and I am going to try and compensate for that with a
>full gigabit backbone.
see above.
The advantages of a NAS or SAN are in things you haven't really touched
on. Is the filesystem going to be accessed by several systems? Do you
need the ability to do snapshots? (You may be able to do this with
direct-attach also, but doing it on a remote storage device tends to be
simpler.) Do you want to share one big, expensive, reliable unit between
multiple systems? Will you be doing failover? (Note that failover
requires software to go with the hardware, and can be done in a
different way with local storage also.) In some environments the answers
to those questions are yes, and the price premium & performance
implications are well worth it. For a single DB server the answer is
almost certainly "no".
Mike Stone
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:24 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
@ 2006-04-27 13:41 ` Ketema Harris <ketema@gmail.com>
2006-04-27 14:04 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
0 siblings, 1 reply; 19+ messages in thread
From: Ketema Harris @ 2006-04-27 13:41 UTC (permalink / raw)
To: Michael Stone <mstone+postgres@mathom.us>; +Cc: pgsql-performance
First, I appreciate all of your input.
>No, backups are completely unrelated to your storage type; you need them
> either way.
Please another post. I meant the storage would do the back ups.
>redundancy, expandability
What I mean by these stupid flavor words is:
Redundancy : raid 5.
Expandability : the ability to stick another drive in my array and get more
storage and not have to turn of the db.
>Do you
> need the ability to do snapshots?
Yes.
>Do you want to share one big, expensive, reliable unit between
> multiple systems? Will you be doing failover?
Yes, and Yes. Really on one other system, a phone system, but it is the
crux of my business and will be writing a lot of recorded phone calls. I am
working with a storage company now to set up the failover, I want the db and
phone systems to never no if the storage switched over.
You have given me a lot to think about. The performance concerns me and I
will have to find some way to test. Perhaps spending a little less on the
storage system and more on the actual servers is the way to go? Then
utilize some combination off pg_backup, and the archive_command directive
with a periodic script.
Thank You all. I will keep researching this and the more input the better.
Thank You.
On 4/27/06 9:24 AM, "Michael Stone" <mstone+postgres@mathom.us> wrote:
> On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote:
>> OK. My thought process was that having non local storage as say a big raid
>> 5 san ( I am talking 5 TB with expansion capability up to 10 )
>
> That's two disk trays for a cheap slow array. (Versus a more expensive
> solution with more spindles and better seek performance.)
>
>> would allow
>> me to have redundancy, expandability, and hopefully still retain decent
>> performance from the db. I also would hopefully then not have to do
>> periodic backups from the db server to some other type of storage.
>
> No, backups are completely unrelated to your storage type; you need them
> either way. On a SAN you can use a SAN backup solution to back multiple
> systems up with a single backup unit without involving the host CPUs.
> This is fairly useless if you aren't amortizing the cost over a large
> environment.
>
>> Is this not a good idea?
>
> It really depends on what you're hoping to get. As described, it's not
> clear. (I don't know what you mean by "redundancy, expandability" or
> "decent performance".)
>
>> How bad of a performance hit are we talking about?
>
> Way too many factors for an easy answer. Consider the case of NAS vs
> SCSI direct attach storage. You're probably in that case comparing a
> single 125MB/s (peak) gigabit ethernet channel to (potentially several)
> 320MB/s (peak) SCSI channels. With a high-end NAS you might get 120MB/s
> off that GBE. With a (more realistic) mid-range unit you're more likely
> to get 40-60MB/s. Getting 200MB/s off the SCSI channel isn't a stretch,
> and you can fairly easily stripe across multiple SCSI channels. (You can
> also bond multiple GBEs, but then your cost & complexity start going way
> up, and you're never going to scale as well.) If you have an environment
> where you're doing a lot of sequential scans it isn't even a contest.
> You can also substitute SATA for SCSI, etc.
>
> For a FC SAN the peformance numbers are a lot better, but the costs &
> complexity are a lot higher. An iSCSI SAN is somewhere in the middle.
>
>> Also, in regards to the commit data integrity, as far as the db is
>> concerned once the data is sent to the san or nas isn't it "written"?
>> The storage may have that write in cache, but from my reading and
>> understanding of how these various storage devices work that is how
>> they keep up performance.
>
> Depends on the configuration, but yes, most should be able to report
> back a "write" once the data is in a non-volatile cache. You can do the
> same with a direct-attached array and eliminate the latency inherent in
> accessing the remote storage.
>
>> I would expect my bottleneck if any to be the actual Ethernet transfer
>> to the storage, and I am going to try and compensate for that with a
>> full gigabit backbone.
>
> see above.
>
> The advantages of a NAS or SAN are in things you haven't really touched
> on. Is the filesystem going to be accessed by several systems? Do you
> need the ability to do snapshots? (You may be able to do this with
> direct-attach also, but doing it on a remote storage device tends to be
> simpler.) Do you want to share one big, expensive, reliable unit between
> multiple systems? Will you be doing failover? (Note that failover
> requires software to go with the hardware, and can be done in a
> different way with local storage also.) In some environments the answers
> to those questions are yes, and the price premium & performance
> implications are well worth it. For a single DB server the answer is
> almost certainly "no".
>
> Mike Stone
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:24 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 13:41 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
@ 2006-04-27 14:04 ` Michael Stone <mstone+postgres@mathom.us>
2006-04-27 17:50 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
0 siblings, 1 reply; 19+ messages in thread
From: Michael Stone @ 2006-04-27 14:04 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; +Cc: pgsql-performance
On Thu, Apr 27, 2006 at 09:41:21AM -0400, Ketema Harris wrote:
>>No, backups are completely unrelated to your storage type; you need them
>> either way.
>Please another post. I meant the storage would do the back ups.
Which isn't a backup. Even expensive storage arrays can break or burn
down.
>>redundancy, expandability
>What I mean by these stupid flavor words is:
>Redundancy : raid 5.
You can get that without external storage.
>Expandability : the ability to stick another drive in my array and get more
>storage and not have to turn of the db.
You can also get that without external storage assuming you choose a
platform with a volume manager.
>>Do you
>> need the ability to do snapshots?
>Yes.
If that's a hard requirement you'll have to eat the cost & performance
problems of an external solution or choose a platform that will let you
do that with direct-attach storage. (Something with a volume manager.)
>>Do you want to share one big, expensive, reliable unit between
>> multiple systems? Will you be doing failover?
>Yes, and Yes. Really on one other system, a phone system, but it is the
>crux of my business and will be writing a lot of recorded phone calls. I am
>working with a storage company now to set up the failover, I want the db and
>phone systems to never no if the storage switched over.
If you actually have a couple of systems you're trying to fail over, a
FC SAN may be a reasonable solution. Depending on your reliability
requirement you can have multiple interfaces & FC switches to get
redundant paths and a much higher level of storage reliability than you
could get with direct attach storage. OTOH, if the DB server itself
breaks you're still out of luck. :) You might compare that sort of
solution with a solution that has redundant servers and implements the
failover in software instead of hardware.
Mike Stone
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:24 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 13:41 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 14:04 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
@ 2006-04-27 17:50 ` Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 17:57 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
0 siblings, 1 reply; 19+ messages in thread
From: Jim C. Nasby @ 2006-04-27 17:50 UTC (permalink / raw)
To: Ketema Harris <ketema@gmail.com>; pgsql-performance
On Thu, Apr 27, 2006 at 10:04:19AM -0400, Michael Stone wrote:
> >>redundancy, expandability
> >What I mean by these stupid flavor words is:
> >Redundancy : raid 5.
>
> You can get that without external storage.
Yes, but some dedicated storage devices actually provide good
performance with RAID5. Most simpler solutions give pretty abysmal write
performance.
> >>Do you
> >>need the ability to do snapshots?
> >Yes.
>
> If that's a hard requirement you'll have to eat the cost & performance
> problems of an external solution or choose a platform that will let you
> do that with direct-attach storage. (Something with a volume manager.)
I'm wondering if PITR would suffice. Or maybe even Slony.
> >>Do you want to share one big, expensive, reliable unit between
> >>multiple systems? Will you be doing failover?
> >Yes, and Yes. Really on one other system, a phone system, but it is the
> >crux of my business and will be writing a lot of recorded phone calls. I am
> >working with a storage company now to set up the failover, I want the db
> >and
> >phone systems to never no if the storage switched over.
>
> If you actually have a couple of systems you're trying to fail over, a
> FC SAN may be a reasonable solution. Depending on your reliability
> requirement you can have multiple interfaces & FC switches to get
> redundant paths and a much higher level of storage reliability than you
> could get with direct attach storage. OTOH, if the DB server itself
> breaks you're still out of luck. :) You might compare that sort of
> solution with a solution that has redundant servers and implements the
> failover in software instead of hardware.
BTW, I know a company here in Austin that does capacity planning for
complex systems like this; contact me off-list if you're interested in
talking to them.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 13:24 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 13:41 ` Re: Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 14:04 ` Re: Running on an NFS Mounted Directory Michael Stone <mstone+postgres@mathom.us>
2006-04-27 17:50 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
@ 2006-04-27 17:57 ` Michael Stone <mstone+postgres@mathom.us>
0 siblings, 0 replies; 19+ messages in thread
From: Michael Stone @ 2006-04-27 17:57 UTC (permalink / raw)
To: pgsql-performance
On Thu, Apr 27, 2006 at 12:50:16PM -0500, Jim C. Nasby wrote:
>Yes, but some dedicated storage devices actually provide good
>performance with RAID5. Most simpler solutions give pretty abysmal write
>performance.
dedicated storage device != SAN != NAS. You can get good performance in
a dedicated direct-attach device without paying for the SAN/NAS
infrastructure if you don't need it; you don't have to go right from EMC
to PERC with nothing in the middle.
Mike Stone
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Running on an NFS Mounted Directory
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Re: Running on an NFS Mounted Directory Steve Wampler <swampler@noao.edu>
2006-04-27 04:55 ` Re: Running on an NFS Mounted Directory Jim C. Nasby <jnasby@pervasive.com>
@ 2006-05-02 14:58 ` Fortuitous Technologies <null@fortuitous.dot.nul>
1 sibling, 0 replies; 19+ messages in thread
From: Fortuitous Technologies @ 2006-05-02 14:58 UTC (permalink / raw)
To: pgsql-performance
On Wed, 26 Apr 2006 23:55:24 -0500, Jim C. Nasby wrote:
> On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote:
>> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
>> > I was wondering if there were any performance issues with having a data
>> > directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
>> > anyone done this before?
>>
>> My understanding is that NFS is pretty poor in performance in general,
NFS is not a good choice for several reasons. First, NFS takes
priority in the system kernel, and will slow down all other
operations. Your best choice, as pointed out by others, is a DAS
solutions. If you must use NFS, you should consider putting it on
a fast dedicated network by itself.
^ permalink raw reply [nested|flat] 19+ messages in thread
end of thread, other threads:[~2006-05-02 14:58 UTC | newest]
Thread overview: 19+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-04-27 02:06 Running on an NFS Mounted Directory Ketema Harris <ketema@gmail.com>
2006-04-27 02:35 ` Steve Wampler <swampler@noao.edu>
2006-04-27 04:43 ` Dan Gorman <dgorman@hi5.com>
2006-04-27 04:55 ` Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 12:38 ` Ketema Harris <ketema@gmail.com>
2006-04-27 12:44 ` Michael Stone <mstone+postgres@mathom.us>
2006-04-27 12:57 ` Ketema Harris <ketema@gmail.com>
2006-04-27 13:05 ` Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:06 ` Ketema Harris <ketema@gmail.com>
2006-04-27 13:31 ` Bruno Wolff III <bruno@wolff.to>
2006-04-27 13:42 ` Ketema Harris <ketema@gmail.com>
2006-04-27 18:58 ` Dan Gorman <dgorman@hi5.com>
2006-04-27 13:09 ` Steve Wampler <swampler@noao.edu>
2006-04-27 13:24 ` Michael Stone <mstone+postgres@mathom.us>
2006-04-27 13:41 ` Ketema Harris <ketema@gmail.com>
2006-04-27 14:04 ` Michael Stone <mstone+postgres@mathom.us>
2006-04-27 17:50 ` Jim C. Nasby <jnasby@pervasive.com>
2006-04-27 17:57 ` Michael Stone <mstone+postgres@mathom.us>
2006-05-02 14:58 ` Fortuitous Technologies <null@fortuitous.dot.nul>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox