Running on an NFS Mounted Directory

From ketema@gmail.com Thu Apr 27 02:07:10 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 3F5A311F60B0 for ; Wed, 26 Apr 2006 23:07:10 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 38296-08-2 for ; Wed, 26 Apr 2006 23:07:01 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.233]) by postgresql.org (Postfix) with ESMTP id 7666511F6920 for ; Wed, 26 Apr 2006 23:06:59 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id i32so1507619wra for ; Wed, 26 Apr 2006 19:07:00 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:user-agent:date:subject:from:to:message-id:thread-topic:thread-index:mime-version:content-type; b=oF6MVNfhLbxoE/w6x3TIzvr1SOaGYsBc4uZ7bGGS9JCOOGfRIUtbjTUFsq1+7yHM2HL2L6w6niBGbOqiRrcsAigQSrUunjG75bhC53J26LhDr/07VJnm3tf1jrNQ5DAnTy3ce/pFZzVzXkMAdhowXF8UaHCr/PzH3WPhnm7RfoE= Received: by 10.54.125.3 with SMTP id x3mr219924wrc; Wed, 26 Apr 2006 19:07:00 -0700 (PDT) Received: from ?192.168.0.12? ( [72.156.236.193]) by mx.gmail.com with ESMTP id 26sm121898wra.2006.04.26.19.07.00; Wed, 26 Apr 2006 19:07:00 -0700 (PDT) User-Agent: Microsoft-Entourage/11.2.3.060209 Date: Wed, 26 Apr 2006 22:06:58 -0400 Subject: Running on an NFS Mounted Directory From: Ketema Harris To: Message-ID: Thread-Topic: Running on an NFS Mounted Directory Thread-Index: AcZpn0OCglyb5tWSEdqtugARJHlfcA== Mime-version: 1.0 Content-type: multipart/alternative; boundary="B_3228934019_1019469" X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/621 X-Sequence-Number: 18705 > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3228934019_1019469 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit I was wondering if there were any performance issues with having a data directory that was an nfs mounted drive? Say like a SAN or NAS device? Has anyone done this before? --B_3228934019_1019469 Content-type: text/html; charset="US-ASCII" Content-transfer-encoding: quoted-printable Running on an NFS Mounted Directory I was wondering if the= re were any performance issues with having a data directory that was an nfs = mounted drive? Say like a SAN or NAS device? Has anyone done this befo= re?
--B_3228934019_1019469-- From swampler@noao.edu Thu Apr 27 03:36:05 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 1630711F60E8 for ; Thu, 27 Apr 2006 00:36:05 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 67794-05 for ; Thu, 27 Apr 2006 00:35:51 -0300 (ADT) X-Greylist: delayed 01:00:08.878658 by SQLgrey- Received: from noao.edu (noao.edu [140.252.1.54]) by postgresql.org (Postfix) with ESMTP id 2619211F60E5 for ; Thu, 27 Apr 2006 00:35:50 -0300 (ADT) X-TFF-CGPSA-Version: 1.4f1 X-TFF-CGPSA-Filter: Scanned Received: from weaver.tuc.noao.edu ([140.252.14.8] verified) by noao.edu (CommuniGate Pro SMTP 5.0.9) with ESMTPS id 25881248; Wed, 26 Apr 2006 19:35:42 -0700 Received: from weaver.tuc.noao.edu (localhost.localdomain [127.0.0.1]) by weaver.tuc.noao.edu (8.13.1/8.13.1) with ESMTP id k3R2Zg0p014715; Wed, 26 Apr 2006 19:35:42 -0700 Received: (from swampler@localhost) by weaver.tuc.noao.edu (8.13.1/8.13.1/Submit) id k3R2ZgDg014714; Wed, 26 Apr 2006 19:35:42 -0700 Date: Wed, 26 Apr 2006 19:35:42 -0700 From: Steve Wampler To: Ketema Harris Cc: pgsql-performance@postgresql.org Subject: Re: Running on an NFS Mounted Directory Message-ID: <20060427023542.GA14672@weaver.tuc.noao.edu> Reply-To: swampler@noao.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/624 X-Sequence-Number: 18708 On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote: > I was wondering if there were any performance issues with having a data > directory that was an nfs mounted drive? Say like a SAN or NAS device? Has > anyone done this before? My understanding is that NFS is pretty poor in performance in general, so I would expect it to be particularly bad for a DB. You might run some (non-DB) performance tests to get a feel for how bad it might me. (Someone once told me that NFS topped out at around 12MB/s, but I don't know if that's really true [they were trying to sell a competitive networked filesystem]). In any event, you're at least limited by ethernet speeds, if not more. -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud. From dgorman@hi5.com Thu Apr 27 05:02:50 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id C791D11F68FC for ; Thu, 27 Apr 2006 02:02:50 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 89207-09 for ; Thu, 27 Apr 2006 02:02:42 -0300 (ADT) X-Greylist: delayed 00:18:44.681278 by SQLgrey- Received: from svr4.postgresql.org (svr4.postgresql.org [66.98.251.159]) by postgresql.org (Postfix) with ESMTP id D0A8711F68DA for ; Thu, 27 Apr 2006 02:02:41 -0300 (ADT) Received: from corpmail.hi5.com (corpmail.hi5.com [204.13.49.5]) by svr4.postgresql.org (Postfix) with ESMTP id 718B25B2E5D for ; Thu, 27 Apr 2006 04:43:56 +0000 (GMT) Received: from [192.168.15.101] by hi5.com (MDaemon PRO v9.0.0 trial) with ESMTP id md50002838323.msg for ; Wed, 26 Apr 2006 21:43:26 -0700 In-Reply-To: <20060427023542.GA14672@weaver.tuc.noao.edu> References: <20060427023542.GA14672@weaver.tuc.noao.edu> Mime-Version: 1.0 (Apple Message framework v749.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Cc: Ketema Harris , pgsql-performance@postgresql.org Content-Transfer-Encoding: 7bit From: Dan Gorman Subject: Re: Running on an NFS Mounted Directory Date: Wed, 26 Apr 2006 21:43:26 -0700 To: swampler@noao.edu X-Mailer: Apple Mail (2.749.3) X-Authenticated-Sender: dgorman@hi5.com X-Spam-Processed: corpmail, Wed, 26 Apr 2006 21:43:26 -0700 (not processed: message from valid local sender) X-Lookup-Warning: EHLO lookup on [192.168.15.101] does not match 65.254.152.26 X-MDRemoteIP: 65.254.152.26 X-Return-Path: dgorman@hi5.com X-MDaemon-Deliver-To: pgsql-performance@postgresql.org X-MDAV-Processed: corpmail, Wed, 26 Apr 2006 21:43:26 -0700 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/626 X-Sequence-Number: 18710 We have gotten very good performance from netapp and postgres 7.4.11 . I was able to push about 100MB/s over gigE, but that was limited by our netapp. DAS will generally always be faster, but if for example you have 2 disks vs. 100 NFS mounted ,NFS will be faster. NFS is very reliable and I would stay away from iscsi. Regards, Dan Gorman On Apr 26, 2006, at 7:35 PM, Steve Wampler wrote: > On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote: >> I was wondering if there were any performance issues with having a >> data >> directory that was an nfs mounted drive? Say like a SAN or NAS >> device? Has >> anyone done this before? > > My understanding is that NFS is pretty poor in performance in general, > so I would expect it to be particularly bad for a DB. You might run > some (non-DB) performance tests to get a feel for how bad it might me. > (Someone once told me that NFS topped out at around 12MB/s, but I > don't > know if that's really true [they were trying to sell a competitive > networked filesystem]). > > In any event, you're at least limited by ethernet speeds, if not more. > > -- > Steve Wampler -- swampler@noao.edu > The gods that smiled on your birth are now laughing out loud. > > ---------------------------(end of > broadcast)--------------------------- > TIP 2: Don't 'kill -9' the postmaster From jnasby@pervasive.com Thu Apr 27 04:55:44 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id DE8D311F65A1 for ; Thu, 27 Apr 2006 01:55:44 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 82983-06 for ; Thu, 27 Apr 2006 01:55:28 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by postgresql.org (Postfix) with ESMTP id 7429511F65AD for ; Thu, 27 Apr 2006 01:55:27 -0300 (ADT) Received: by noel.decibel.org (Postfix, from userid 1001) id 07E0D56423; Wed, 26 Apr 2006 23:55:27 -0500 (CDT) Received: (hashcash-sendmail, from uid 1001); Wed, 26 Apr 2006 23:55:24 -0500 Date: Wed, 26 Apr 2006 23:55:24 -0500 From: "Jim C. Nasby" To: Steve Wampler Cc: Ketema Harris , pgsql-performance@postgresql.org Subject: Re: Running on an NFS Mounted Directory Message-ID: <20060427045524.GV97354@pervasive.com> References: <20060427023542.GA14672@weaver.tuc.noao.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060427023542.GA14672@weaver.tuc.noao.edu> X-Operating-System: FreeBSD 6.0-RELEASE-p4 amd64 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.11 X-Hashcash: 1:20:060427:swampler@noao.edu::il9/UeObdk7aPmFz:00000000000000000000 0000000000000000000000006Rie X-Hashcash: 1:20:060427:ketema@gmail.com::AijIYmKXQl9dxi1M:0DWdr X-Hashcash: 1:20:060427:pgsql-performance@postgresql.org::xAyxho/ARBavvnB6:00000 000000000000000000000000CviL X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/625 X-Sequence-Number: 18709 On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote: > On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote: > > I was wondering if there were any performance issues with having a data > > directory that was an nfs mounted drive? Say like a SAN or NAS device? Has > > anyone done this before? > > My understanding is that NFS is pretty poor in performance in general, > so I would expect it to be particularly bad for a DB. You might run > some (non-DB) performance tests to get a feel for how bad it might me. > (Someone once told me that NFS topped out at around 12MB/s, but I don't > know if that's really true [they were trying to sell a competitive > networked filesystem]). > > In any event, you're at least limited by ethernet speeds, if not more. More importantly, the latency involved will kill commit performance. If it doesn't then it's likely that fsync isn't being obeyed, which means 0 data integrity. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 From ketema@gmail.com Thu Apr 27 12:39:09 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id BE16611F60E5 for ; Thu, 27 Apr 2006 09:39:09 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 13993-01 for ; Thu, 27 Apr 2006 09:38:58 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.233]) by postgresql.org (Postfix) with ESMTP id 0A00711F60A2 for ; Thu, 27 Apr 2006 09:38:56 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id i32so1583805wra for ; Thu, 27 Apr 2006 05:38:57 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:user-agent:date:subject:from:to:cc:message-id:thread-topic:thread-index:in-reply-to:mime-version:content-type:content-transfer-encoding; b=lrWGE1aTRkzzukhc7v8gPlN8BmT/6Hox1hDWWLsnwG316kmQ+jDuPH06mEBgl85sp8wG3QDNuAE8QMYuzF8Bn0b5+Pbd+SH9t+5cz9sB2awSfjGZhxVK0s43yoz+cPmLsEP34Yg4FnxJnK853+VcViFqQQsokfKeJ/4i8VqXqxg= Received: by 10.54.140.11 with SMTP id n11mr2168287wrd; Thu, 27 Apr 2006 05:38:57 -0700 (PDT) Received: from ?192.168.0.12? ( [72.156.236.193]) by mx.gmail.com with ESMTP id 28sm802435wrl.2006.04.27.05.38.56; Thu, 27 Apr 2006 05:38:57 -0700 (PDT) User-Agent: Microsoft-Entourage/11.2.3.060209 Date: Thu, 27 Apr 2006 08:38:55 -0400 Subject: Re: Running on an NFS Mounted Directory From: Ketema Harris To: "Jim C. Nasby" , Steve Wampler CC: Message-ID: Thread-Topic: [PERFORM] Running on an NFS Mounted Directory Thread-Index: AcZp94vNypahiNXqEdqi1AARJHlfcA== In-Reply-To: <20060427045524.GV97354@pervasive.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/633 X-Sequence-Number: 18717 I am looking for the best solution to have a large amount of disk storage attached to my PostgreSQL 8.1 server. I was thinking of having a san or nas attached device be mounted by the pg server over nfs, hence the question about nfs performance. What other options/protocols are there to get high performance and data integrity while having the benefit of not having the physical storage attached to the db server? On 4/27/06 12:55 AM, "Jim C. Nasby" wrote: > On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote: >> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote: >>> I was wondering if there were any performance issues with having a data >>> directory that was an nfs mounted drive? Say like a SAN or NAS device? Has >>> anyone done this before? >> >> My understanding is that NFS is pretty poor in performance in general, >> so I would expect it to be particularly bad for a DB. You might run >> some (non-DB) performance tests to get a feel for how bad it might me. >> (Someone once told me that NFS topped out at around 12MB/s, but I don't >> know if that's really true [they were trying to sell a competitive >> networked filesystem]). >> >> In any event, you're at least limited by ethernet speeds, if not more. > > More importantly, the latency involved will kill commit performance. If > it doesn't then it's likely that fsync isn't being obeyed, which means 0 > data integrity. From mstone+postgres@mathom.us Thu Apr 27 12:45:13 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 1011B11F60A2 for ; Thu, 27 Apr 2006 09:45:13 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 13295-03 for ; Thu, 27 Apr 2006 09:45:02 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from vms044pub.verizon.net (vms044pub.verizon.net [206.46.252.44]) by postgresql.org (Postfix) with ESMTP id 0BD3711F60E5 for ; Thu, 27 Apr 2006 09:45:00 -0300 (ADT) Received: from osgiliath.mathom.us ([70.108.132.172]) by vms044.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0IYD00H9DSQZ2TS4@vms044.mailsrvcs.net> for pgsql-performance@postgresql.org; Thu, 27 Apr 2006 07:45:00 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by osgiliath.mathom.us (Postfix) with ESMTP id BB31C6E6C0; Thu, 27 Apr 2006 08:44:58 -0400 (EDT) Received: from osgiliath.mathom.us ([127.0.0.1]) by localhost (osgiliath.home.mathom.us [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 16366-02-2; Thu, 27 Apr 2006 08:44:57 -0400 (EDT) Received: by osgiliath.mathom.us (Postfix, from userid 1000) id AFD346E7A7; Thu, 27 Apr 2006 08:44:57 -0400 (EDT) Date: Thu, 27 Apr 2006 08:44:57 -0400 From: Michael Stone Subject: Re: Running on an NFS Mounted Directory In-reply-to: To: Ketema Harris Cc: pgsql-performance@postgresql.org Mail-followup-to: Ketema Harris , pgsql-performance@postgresql.org Message-id: <20060427124455.GF31328@mathom.us> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-disposition: inline X-Pgp-Fingerprint: 53 FF 38 00 E7 DD 0A 9C 84 52 84 C5 EE DF 7C 88 X-Virus-Scanned: Debian amavisd-new at mathom.us References: <20060427045524.GV97354@pervasive.com> User-Agent: Mutt/1.5.11+cvs20060403 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/634 X-Sequence-Number: 18718 On Thu, Apr 27, 2006 at 08:38:55AM -0400, Ketema Harris wrote: >I am looking for the best solution to have a large amount of disk storage >attached to my PostgreSQL 8.1 server. >What other options/protocols are there to get high performance and data >integrity while having the benefit of not having the physical storage >attached to the db server? These are two distinct requirements. Are both really requirements or is one "nice to have"? The "best" solution for "a large amount of disk storage" isn't "not having the physical storage attached to the db server". If you use non-local storage it will be slower and more expensive, quite likely by a large margin. There may be other advantages to doing so, but you haven't mentioned any of those as requirements. Mike Stone From ketema@gmail.com Thu Apr 27 12:58:09 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 3117511F61F8 for ; Thu, 27 Apr 2006 09:58:09 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 10537-09 for ; Thu, 27 Apr 2006 09:57:54 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.231]) by postgresql.org (Postfix) with ESMTP id B9DD311F60E5 for ; Thu, 27 Apr 2006 09:57:52 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id i32so1586928wra for ; Thu, 27 Apr 2006 05:57:52 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:user-agent:date:subject:from:to:cc:message-id:thread-topic:thread-index:in-reply-to:mime-version:content-type:content-transfer-encoding; b=PnOr0dVc4lqa+PVR+tZHZAYk+i7hk+BMyCUAd1Q7KfKnzqRcIwTzkne/xkTgFb0yvPzUSTs3uq3th6/sM73nobitQ7dUE616XCEoHgyQNaeWCJes0dYrjrjFDEssP2hg3mUqMoPHSiJ0SYHl72qQryxVluhxl08j1auSEY6XctU= Received: by 10.54.147.19 with SMTP id u19mr286203wrd; Thu, 27 Apr 2006 05:57:52 -0700 (PDT) Received: from ?192.168.0.12? ( [72.156.236.193]) by mx.gmail.com with ESMTP id 29sm496872wrl.2006.04.27.05.57.52; Thu, 27 Apr 2006 05:57:52 -0700 (PDT) User-Agent: Microsoft-Entourage/11.2.3.060209 Date: Thu, 27 Apr 2006 08:57:51 -0400 Subject: Re: Running on an NFS Mounted Directory From: Ketema Harris To: Michael Stone CC: Message-ID: Thread-Topic: [PERFORM] Running on an NFS Mounted Directory Thread-Index: AcZp+jDpb2ae0dXtEdqi1AARJHlfcA== In-Reply-To: <20060427124455.GF31328@mathom.us> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/635 X-Sequence-Number: 18719 OK. My thought process was that having non local storage as say a big raid 5 san ( I am talking 5 TB with expansion capability up to 10 ) would allow me to have redundancy, expandability, and hopefully still retain decent performance from the db. I also would hopefully then not have to do periodic backups from the db server to some other type of storage. Is this not a good idea? How bad of a performance hit are we talking about? Also, in regards to the commit data integrity, as far as the db is concerned once the data is sent to the san or nas isn't it "written"? The storage may have that write in cache, but from my reading and understanding of how these various storage devices work that is how they keep up performance. I would expect my bottleneck if any to be the actual Ethernet transfer to the storage, and I am going to try and compensate for that with a full gigabit backbone. On 4/27/06 8:44 AM, "Michael Stone" wrote: > On Thu, Apr 27, 2006 at 08:38:55AM -0400, Ketema Harris wrote: >> I am looking for the best solution to have a large amount of disk storage >> attached to my PostgreSQL 8.1 server. > >> What other options/protocols are there to get high performance and data >> integrity while having the benefit of not having the physical storage >> attached to the db server? > > These are two distinct requirements. Are both really requirements or is > one "nice to have"? The "best" solution for "a large amount of disk > storage" isn't "not having the physical storage attached to the db > server". If you use non-local storage it will be slower and more > expensive, quite likely by a large margin. There may be other advantages > to doing so, but you haven't mentioned any of those as requirements. > > Mike Stone From bruno@wolff.to Thu Apr 27 13:02:55 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 11CC911F60B1 for ; Thu, 27 Apr 2006 10:02:55 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 15132-08 for ; Thu, 27 Apr 2006 10:02:46 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from wolff.to (wolff.to [66.93.197.194]) by postgresql.org (Postfix) with SMTP id 8FFC911F60E7 for ; Thu, 27 Apr 2006 10:02:44 -0300 (ADT) Received: (qmail 15627 invoked by uid 500); 27 Apr 2006 13:05:55 -0000 Date: Thu, 27 Apr 2006 08:05:55 -0500 From: Bruno Wolff III To: Ketema Harris Cc: Michael Stone , pgsql-performance@postgresql.org Subject: Re: Running on an NFS Mounted Directory Message-ID: <20060427130555.GE6159@wolff.to> Mail-Followup-To: Bruno Wolff III , Ketema Harris , Michael Stone , pgsql-performance@postgresql.org References: <20060427124455.GF31328@mathom.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/636 X-Sequence-Number: 18720 On Thu, Apr 27, 2006 at 08:57:51 -0400, Ketema Harris wrote: > performance from the db. I also would hopefully then not have to do > periodic backups from the db server to some other type of storage. Is this > not a good idea? How bad of a performance hit are we talking about? Also, You always need to do backups if you care about your data. What if someone accidental deletes a lot of data? What if someone blows up your data center (or there is a flood)? From ketema@gmail.com Thu Apr 27 13:07:01 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id D85C911F60B1 for ; Thu, 27 Apr 2006 10:07:01 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 18560-02 for ; Thu, 27 Apr 2006 10:06:52 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.229]) by postgresql.org (Postfix) with ESMTP id DC77711F65AD for ; Thu, 27 Apr 2006 10:06:50 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id i32so1588617wra for ; Thu, 27 Apr 2006 06:06:51 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:user-agent:date:subject:from:to:cc:message-id:thread-topic:thread-index:in-reply-to:mime-version:content-type:content-transfer-encoding; b=ZhhaknAllm3eoL5vCd1FenmQTMJWnSLXwqXTJ6xrycCdub1Llai0EHNQJMQN+0iu7hP0ILLoiQbYEMENO4YEyi4EHF2TLsB+H0127tHYsEXCkW5o/amOxUAOGLpANRJyGVDjkFRqBSzVpj8y80mLCYc5pfXGw97OB1YMUxajK+E= Received: by 10.54.107.17 with SMTP id f17mr6096236wrc; Thu, 27 Apr 2006 06:06:51 -0700 (PDT) Received: from ?192.168.0.12? ( [72.156.236.193]) by mx.gmail.com with ESMTP id 7sm488320wrl.2006.04.27.06.06.50; Thu, 27 Apr 2006 06:06:50 -0700 (PDT) User-Agent: Microsoft-Entourage/11.2.3.060209 Date: Thu, 27 Apr 2006 09:06:48 -0400 Subject: Re: Running on an NFS Mounted Directory From: Ketema Harris To: Bruno Wolff III CC: Michael Stone , Message-ID: Thread-Topic: Running on an NFS Mounted Directory Thread-Index: AcZp+3D9r6PWJtXuEdqi1AARJHlfcA== In-Reply-To: <20060427130555.GE6159@wolff.to> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/637 X-Sequence-Number: 18721 Yes, your right, I meant not have to do the backups from the db server itself. I can do that within the storage device now, by allocating space for it, and letting the device copy the data files on some periodic basis. On 4/27/06 9:05 AM, "Bruno Wolff III" wrote: > On Thu, Apr 27, 2006 at 08:57:51 -0400, > Ketema Harris wrote: >> performance from the db. I also would hopefully then not have to do >> periodic backups from the db server to some other type of storage. Is this >> not a good idea? How bad of a performance hit are we talking about? Also, > > You always need to do backups if you care about your data. What if someone > accidental deletes a lot of data? What if someone blows up your data > center (or there is a flood)? From swampler@noao.edu Thu Apr 27 13:10:14 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id D5D1D11F6780 for ; Thu, 27 Apr 2006 10:10:14 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 13991-06 for ; Thu, 27 Apr 2006 10:09:58 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noao.edu (noao.edu [140.252.1.54]) by postgresql.org (Postfix) with ESMTP id B27F911F65C6 for ; Thu, 27 Apr 2006 10:09:56 -0300 (ADT) X-TFF-CGPSA-Version: 1.4f1 X-TFF-CGPSA-Filter: Scanned Received: from weaver.tuc.noao.edu ([140.252.14.8] verified) by noao.edu (CommuniGate Pro SMTP 5.0.9) with ESMTPS id 25895456; Thu, 27 Apr 2006 06:09:51 -0700 Received: from weaver.tuc.noao.edu (localhost.localdomain [127.0.0.1]) by weaver.tuc.noao.edu (8.13.1/8.13.1) with ESMTP id k3RD9pAn021314; Thu, 27 Apr 2006 06:09:51 -0700 Received: (from swampler@localhost) by weaver.tuc.noao.edu (8.13.1/8.13.1/Submit) id k3RD9ot0021313; Thu, 27 Apr 2006 06:09:50 -0700 Date: Thu, 27 Apr 2006 06:09:50 -0700 From: Steve Wampler To: Ketema Harris Cc: Michael Stone , pgsql-performance@postgresql.org Subject: Re: Running on an NFS Mounted Directory Message-ID: <20060427130950.GA21200@weaver.tuc.noao.edu> Reply-To: swampler@noao.edu References: <20060427124455.GF31328@mathom.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/638 X-Sequence-Number: 18722 On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote: > OK. My thought process was that having non local storage as say a big raid > 5 san ( I am talking 5 TB with expansion capability up to 10 ) would allow > me to have redundancy, expandability, and hopefully still retain decent > performance from the db. I also would hopefully then not have to do > periodic backups from the db server to some other type of storage. Is this > not a good idea? How bad of a performance hit are we talking about? Also, > in regards to the commit data integrity, as far as the db is concerned once > the data is sent to the san or nas isn't it "written"? The storage may have > that write in cache, but from my reading and understanding of how these > various storage devices work that is how they keep up performance. I would > expect my bottleneck if any to be the actual Ethernet transfer to the > storage, and I am going to try and compensate for that with a full gigabit > backbone. Well, if you have to have both the best performance and remote attach storage, I think you'll find that a fibre-channel SAN is still the king of the hill. 4Gb FC switches are common now, though finding a 4Gb HBA for your computer might be a trick. 2Gb HBAs are everywhere in FC land. That's a premium price solution, however, and I don't know anything about how well PG would perform with a FC SAN. We use our SAN for bulk science data and leave the PGDB on a separate machine with local disk. -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud. From mstone+postgres@mathom.us Thu Apr 27 13:25:14 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 8CEE111F65C9 for ; Thu, 27 Apr 2006 10:25:14 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 17431-10 for ; Thu, 27 Apr 2006 10:25:01 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from vms048pub.verizon.net (vms048pub.verizon.net [206.46.252.48]) by postgresql.org (Postfix) with ESMTP id 47E0111F65AD for ; Thu, 27 Apr 2006 10:24:58 -0300 (ADT) Received: from osgiliath.mathom.us ([70.108.132.172]) by vms048.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0IYD004EGULLJC25@vms048.mailsrvcs.net> for pgsql-performance@postgresql.org; Thu, 27 Apr 2006 08:24:59 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by osgiliath.mathom.us (Postfix) with ESMTP id D72676E6C0; Thu, 27 Apr 2006 09:24:55 -0400 (EDT) Received: from osgiliath.mathom.us ([127.0.0.1]) by localhost (osgiliath.home.mathom.us [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 16702-02; Thu, 27 Apr 2006 09:24:55 -0400 (EDT) Received: by osgiliath.mathom.us (Postfix, from userid 1000) id A7C996E945; Thu, 27 Apr 2006 09:24:54 -0400 (EDT) Date: Thu, 27 Apr 2006 09:24:54 -0400 From: Michael Stone Subject: Re: Running on an NFS Mounted Directory In-reply-to: To: Ketema Harris Cc: pgsql-performance@postgresql.org Mail-followup-to: Ketema Harris , pgsql-performance@postgresql.org Message-id: <20060427132452.GG31328@mathom.us> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-disposition: inline X-Pgp-Fingerprint: 53 FF 38 00 E7 DD 0A 9C 84 52 84 C5 EE DF 7C 88 X-Virus-Scanned: Debian amavisd-new at mathom.us References: <20060427124455.GF31328@mathom.us> User-Agent: Mutt/1.5.11+cvs20060403 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/639 X-Sequence-Number: 18723 On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote: >OK. My thought process was that having non local storage as say a big raid >5 san ( I am talking 5 TB with expansion capability up to 10 ) That's two disk trays for a cheap slow array. (Versus a more expensive solution with more spindles and better seek performance.) >would allow >me to have redundancy, expandability, and hopefully still retain decent >performance from the db. I also would hopefully then not have to do >periodic backups from the db server to some other type of storage. No, backups are completely unrelated to your storage type; you need them either way. On a SAN you can use a SAN backup solution to back multiple systems up with a single backup unit without involving the host CPUs. This is fairly useless if you aren't amortizing the cost over a large environment. >Is this not a good idea? It really depends on what you're hoping to get. As described, it's not clear. (I don't know what you mean by "redundancy, expandability" or "decent performance".) >How bad of a performance hit are we talking about? Way too many factors for an easy answer. Consider the case of NAS vs SCSI direct attach storage. You're probably in that case comparing a single 125MB/s (peak) gigabit ethernet channel to (potentially several) 320MB/s (peak) SCSI channels. With a high-end NAS you might get 120MB/s off that GBE. With a (more realistic) mid-range unit you're more likely to get 40-60MB/s. Getting 200MB/s off the SCSI channel isn't a stretch, and you can fairly easily stripe across multiple SCSI channels. (You can also bond multiple GBEs, but then your cost & complexity start going way up, and you're never going to scale as well.) If you have an environment where you're doing a lot of sequential scans it isn't even a contest. You can also substitute SATA for SCSI, etc. For a FC SAN the peformance numbers are a lot better, but the costs & complexity are a lot higher. An iSCSI SAN is somewhere in the middle. >Also, in regards to the commit data integrity, as far as the db is >concerned once the data is sent to the san or nas isn't it "written"? >The storage may have that write in cache, but from my reading and >understanding of how these various storage devices work that is how >they keep up performance. Depends on the configuration, but yes, most should be able to report back a "write" once the data is in a non-volatile cache. You can do the same with a direct-attached array and eliminate the latency inherent in accessing the remote storage. >I would expect my bottleneck if any to be the actual Ethernet transfer >to the storage, and I am going to try and compensate for that with a >full gigabit backbone. see above. The advantages of a NAS or SAN are in things you haven't really touched on. Is the filesystem going to be accessed by several systems? Do you need the ability to do snapshots? (You may be able to do this with direct-attach also, but doing it on a remote storage device tends to be simpler.) Do you want to share one big, expensive, reliable unit between multiple systems? Will you be doing failover? (Note that failover requires software to go with the hardware, and can be done in a different way with local storage also.) In some environments the answers to those questions are yes, and the price premium & performance implications are well worth it. For a single DB server the answer is almost certainly "no". Mike Stone From bruno@wolff.to Thu Apr 27 13:28:30 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 2955A11F65C6 for ; Thu, 27 Apr 2006 10:28:30 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 19963-08 for ; Thu, 27 Apr 2006 10:28:20 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from wolff.to (wolff.to [66.93.197.194]) by postgresql.org (Postfix) with SMTP id C276211F60E5 for ; Thu, 27 Apr 2006 10:28:18 -0300 (ADT) Received: (qmail 19756 invoked by uid 500); 27 Apr 2006 13:31:30 -0000 Date: Thu, 27 Apr 2006 08:31:30 -0500 From: Bruno Wolff III To: Ketema Harris Cc: Michael Stone , pgsql-performance@postgresql.org Subject: Re: Running on an NFS Mounted Directory Message-ID: <20060427133130.GA19320@wolff.to> Mail-Followup-To: Bruno Wolff III , Ketema Harris , Michael Stone , pgsql-performance@postgresql.org References: <20060427130555.GE6159@wolff.to> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/640 X-Sequence-Number: 18724 On Thu, Apr 27, 2006 at 09:06:48 -0400, Ketema Harris wrote: > Yes, your right, I meant not have to do the backups from the db server > itself. I can do that within the storage device now, by allocating space > for it, and letting the device copy the data files on some periodic basis. Only if the database server isn't running or your SAN provides a way to provide a snapshot of the data at a particular instant in time. From ketema@gmail.com Thu Apr 27 13:41:39 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 71DDD11F65A1 for ; Thu, 27 Apr 2006 10:41:39 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 24260-04 for ; Thu, 27 Apr 2006 10:41:25 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.231]) by postgresql.org (Postfix) with ESMTP id 3DC6D11F65AD for ; Thu, 27 Apr 2006 10:41:22 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id i32so1595590wra for ; Thu, 27 Apr 2006 06:41:23 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:user-agent:date:subject:from:to:cc:message-id:thread-topic:thread-index:in-reply-to:mime-version:content-type:content-transfer-encoding; b=ewLTScQD79gylz9P7vv4WF2DBT64eLCw8W6gUrmPYm9LtJOBX/R9lmEU9C4wpA+qftA3JOLw/M/wFZ11jSTyM7BbowOKdeE1lvBmjE4ZCDsNe244H8+dccOLvxsK8k/CGhtXFlxJCAuiIKUzaM4xSTcKnqf7jUOtqLbGblmR744= Received: by 10.54.128.20 with SMTP id a20mr4287472wrd; Thu, 27 Apr 2006 06:41:23 -0700 (PDT) Received: from ?192.168.0.12? ( [72.156.236.193]) by mx.gmail.com with ESMTP id 13sm854035wrl.2006.04.27.06.41.22; Thu, 27 Apr 2006 06:41:23 -0700 (PDT) User-Agent: Microsoft-Entourage/11.2.3.060209 Date: Thu, 27 Apr 2006 09:41:21 -0400 Subject: Re: Running on an NFS Mounted Directory From: Ketema Harris To: Michael Stone CC: Message-ID: Thread-Topic: [PERFORM] Running on an NFS Mounted Directory Thread-Index: AcZqAESYgxTUytXzEdqi1AARJHlfcA== In-Reply-To: <20060427132452.GG31328@mathom.us> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/641 X-Sequence-Number: 18725 First, I appreciate all of your input. >No, backups are completely unrelated to your storage type; you need them > either way. Please another post. I meant the storage would do the back ups. >redundancy, expandability What I mean by these stupid flavor words is: Redundancy : raid 5. Expandability : the ability to stick another drive in my array and get more storage and not have to turn of the db. >Do you > need the ability to do snapshots? Yes. >Do you want to share one big, expensive, reliable unit between > multiple systems? Will you be doing failover? Yes, and Yes. Really on one other system, a phone system, but it is the crux of my business and will be writing a lot of recorded phone calls. I am working with a storage company now to set up the failover, I want the db and phone systems to never no if the storage switched over. You have given me a lot to think about. The performance concerns me and I will have to find some way to test. Perhaps spending a little less on the storage system and more on the actual servers is the way to go? Then utilize some combination off pg_backup, and the archive_command directive with a periodic script. Thank You all. I will keep researching this and the more input the better. Thank You. On 4/27/06 9:24 AM, "Michael Stone" wrote: > On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote: >> OK. My thought process was that having non local storage as say a big raid >> 5 san ( I am talking 5 TB with expansion capability up to 10 ) > > That's two disk trays for a cheap slow array. (Versus a more expensive > solution with more spindles and better seek performance.) > >> would allow >> me to have redundancy, expandability, and hopefully still retain decent >> performance from the db. I also would hopefully then not have to do >> periodic backups from the db server to some other type of storage. > > No, backups are completely unrelated to your storage type; you need them > either way. On a SAN you can use a SAN backup solution to back multiple > systems up with a single backup unit without involving the host CPUs. > This is fairly useless if you aren't amortizing the cost over a large > environment. > >> Is this not a good idea? > > It really depends on what you're hoping to get. As described, it's not > clear. (I don't know what you mean by "redundancy, expandability" or > "decent performance".) > >> How bad of a performance hit are we talking about? > > Way too many factors for an easy answer. Consider the case of NAS vs > SCSI direct attach storage. You're probably in that case comparing a > single 125MB/s (peak) gigabit ethernet channel to (potentially several) > 320MB/s (peak) SCSI channels. With a high-end NAS you might get 120MB/s > off that GBE. With a (more realistic) mid-range unit you're more likely > to get 40-60MB/s. Getting 200MB/s off the SCSI channel isn't a stretch, > and you can fairly easily stripe across multiple SCSI channels. (You can > also bond multiple GBEs, but then your cost & complexity start going way > up, and you're never going to scale as well.) If you have an environment > where you're doing a lot of sequential scans it isn't even a contest. > You can also substitute SATA for SCSI, etc. > > For a FC SAN the peformance numbers are a lot better, but the costs & > complexity are a lot higher. An iSCSI SAN is somewhere in the middle. > >> Also, in regards to the commit data integrity, as far as the db is >> concerned once the data is sent to the san or nas isn't it "written"? >> The storage may have that write in cache, but from my reading and >> understanding of how these various storage devices work that is how >> they keep up performance. > > Depends on the configuration, but yes, most should be able to report > back a "write" once the data is in a non-volatile cache. You can do the > same with a direct-attached array and eliminate the latency inherent in > accessing the remote storage. > >> I would expect my bottleneck if any to be the actual Ethernet transfer >> to the storage, and I am going to try and compensate for that with a >> full gigabit backbone. > > see above. > > The advantages of a NAS or SAN are in things you haven't really touched > on. Is the filesystem going to be accessed by several systems? Do you > need the ability to do snapshots? (You may be able to do this with > direct-attach also, but doing it on a remote storage device tends to be > simpler.) Do you want to share one big, expensive, reliable unit between > multiple systems? Will you be doing failover? (Note that failover > requires software to go with the hardware, and can be done in a > different way with local storage also.) In some environments the answers > to those questions are yes, and the price premium & performance > implications are well worth it. For a single DB server the answer is > almost certainly "no". > > Mike Stone From ketema@gmail.com Thu Apr 27 13:42:22 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 5B91F11F65C6 for ; Thu, 27 Apr 2006 10:42:22 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 27073-02 for ; Thu, 27 Apr 2006 10:42:13 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.225]) by postgresql.org (Postfix) with ESMTP id 81A0711F65A1 for ; Thu, 27 Apr 2006 10:42:11 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id i32so1595737wra for ; Thu, 27 Apr 2006 06:42:12 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:user-agent:date:subject:from:to:cc:message-id:thread-topic:thread-index:in-reply-to:mime-version:content-type:content-transfer-encoding; b=GSHswkVL1/WCBdNS4E7895xtEyr94U83iWbDMMHomBre2HquZYygNUCiKx4y18xsXVTDf0E7TV2BdFlRMK6qCO6rqrtl9Y5XtaGoX8bkMgcjhBdpbevIdvtqwvmxbw0h/LP7W7kpNMb6MHBqUtZwCabFJp+ZLuOS7s8bK3wwR0U= Received: by 10.54.123.12 with SMTP id v12mr2526556wrc; Thu, 27 Apr 2006 06:42:12 -0700 (PDT) Received: from ?192.168.0.12? ( [72.156.236.193]) by mx.gmail.com with ESMTP id 11sm826720wrl.2006.04.27.06.42.11; Thu, 27 Apr 2006 06:42:11 -0700 (PDT) User-Agent: Microsoft-Entourage/11.2.3.060209 Date: Thu, 27 Apr 2006 09:42:10 -0400 Subject: Re: Running on an NFS Mounted Directory From: Ketema Harris To: Bruno Wolff III CC: Michael Stone , Message-ID: Thread-Topic: Running on an NFS Mounted Directory Thread-Index: AcZqAGHMoC+A0tXzEdqi1AARJHlfcA== In-Reply-To: <20060427133130.GA19320@wolff.to> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/642 X-Sequence-Number: 18726 The SAN has the snapshot capability. On 4/27/06 9:31 AM, "Bruno Wolff III" wrote: > On Thu, Apr 27, 2006 at 09:06:48 -0400, > Ketema Harris wrote: >> Yes, your right, I meant not have to do the backups from the db server >> itself. I can do that within the storage device now, by allocating space >> for it, and letting the device copy the data files on some periodic basis. > > Only if the database server isn't running or your SAN provides a way to > provide a snapshot of the data at a particular instant in time. From mstone+postgres@mathom.us Thu Apr 27 14:04:37 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id D8AC711F60AE for ; Thu, 27 Apr 2006 11:04:37 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 28476-07 for ; Thu, 27 Apr 2006 11:04:27 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from vms042pub.verizon.net (vms042pub.verizon.net [206.46.252.42]) by postgresql.org (Postfix) with ESMTP id A233E11F60A2 for ; Thu, 27 Apr 2006 11:04:25 -0300 (ADT) Received: from osgiliath.mathom.us ([70.108.132.172]) by vms042.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0IYD008GGWFBV2K0@vms042.mailsrvcs.net> for pgsql-performance@postgresql.org; Thu, 27 Apr 2006 09:04:24 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by osgiliath.mathom.us (Postfix) with ESMTP id 1170F6E6A8; Thu, 27 Apr 2006 10:04:21 -0400 (EDT) Received: from osgiliath.mathom.us ([127.0.0.1]) by localhost (osgiliath.home.mathom.us [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 16908-03; Thu, 27 Apr 2006 10:04:20 -0400 (EDT) Received: by osgiliath.mathom.us (Postfix, from userid 1000) id CCBE56E945; Thu, 27 Apr 2006 10:04:19 -0400 (EDT) Date: Thu, 27 Apr 2006 10:04:19 -0400 From: Michael Stone Subject: Re: Running on an NFS Mounted Directory In-reply-to: To: Ketema Harris Cc: pgsql-performance@postgresql.org Mail-followup-to: Ketema Harris , pgsql-performance@postgresql.org Message-id: <20060427140417.GH31328@mathom.us> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-disposition: inline X-Pgp-Fingerprint: 53 FF 38 00 E7 DD 0A 9C 84 52 84 C5 EE DF 7C 88 X-Virus-Scanned: Debian amavisd-new at mathom.us References: <20060427132452.GG31328@mathom.us> User-Agent: Mutt/1.5.11+cvs20060403 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/643 X-Sequence-Number: 18727 On Thu, Apr 27, 2006 at 09:41:21AM -0400, Ketema Harris wrote: >>No, backups are completely unrelated to your storage type; you need them >> either way. >Please another post. I meant the storage would do the back ups. Which isn't a backup. Even expensive storage arrays can break or burn down. >>redundancy, expandability >What I mean by these stupid flavor words is: >Redundancy : raid 5. You can get that without external storage. >Expandability : the ability to stick another drive in my array and get more >storage and not have to turn of the db. You can also get that without external storage assuming you choose a platform with a volume manager. >>Do you >> need the ability to do snapshots? >Yes. If that's a hard requirement you'll have to eat the cost & performance problems of an external solution or choose a platform that will let you do that with direct-attach storage. (Something with a volume manager.) >>Do you want to share one big, expensive, reliable unit between >> multiple systems? Will you be doing failover? >Yes, and Yes. Really on one other system, a phone system, but it is the >crux of my business and will be writing a lot of recorded phone calls. I am >working with a storage company now to set up the failover, I want the db and >phone systems to never no if the storage switched over. If you actually have a couple of systems you're trying to fail over, a FC SAN may be a reasonable solution. Depending on your reliability requirement you can have multiple interfaces & FC switches to get redundant paths and a much higher level of storage reliability than you could get with direct attach storage. OTOH, if the DB server itself breaks you're still out of luck. :) You might compare that sort of solution with a solution that has redundant servers and implements the failover in software instead of hardware. Mike Stone From jnasby@pervasive.com Thu Apr 27 17:50:55 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id C1BE111F620C for ; Thu, 27 Apr 2006 14:50:55 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 97833-03 for ; Thu, 27 Apr 2006 14:50:34 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by postgresql.org (Postfix) with ESMTP id 44F1911F60E5 for ; Thu, 27 Apr 2006 14:50:32 -0300 (ADT) Received: by noel.decibel.org (Postfix, from userid 1001) id 54E8456423; Thu, 27 Apr 2006 12:50:17 -0500 (CDT) Received: (hashcash-sendmail, from uid 1001); Thu, 27 Apr 2006 12:50:16 -0500 Date: Thu, 27 Apr 2006 12:50:16 -0500 From: "Jim C. Nasby" To: Ketema Harris , pgsql-performance@postgresql.org Subject: Re: Running on an NFS Mounted Directory Message-ID: <20060427175016.GH97354@pervasive.com> References: <20060427132452.GG31328@mathom.us> <20060427140417.GH31328@mathom.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060427140417.GH31328@mathom.us> X-Operating-System: FreeBSD 6.0-RELEASE-p4 amd64 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.11 X-Hashcash: 1:20:060427:ketema@gmail.com::OUtvWD/w4nKEJmi0:04ZzH X-Hashcash: 1:20:060427:pgsql-performance@postgresql.org::gjmrtUjNLQgQCcmU:00000 0000000000000000000000001Fqw X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/650 X-Sequence-Number: 18734 On Thu, Apr 27, 2006 at 10:04:19AM -0400, Michael Stone wrote: > >>redundancy, expandability > >What I mean by these stupid flavor words is: > >Redundancy : raid 5. > > You can get that without external storage. Yes, but some dedicated storage devices actually provide good performance with RAID5. Most simpler solutions give pretty abysmal write performance. > >>Do you > >>need the ability to do snapshots? > >Yes. > > If that's a hard requirement you'll have to eat the cost & performance > problems of an external solution or choose a platform that will let you > do that with direct-attach storage. (Something with a volume manager.) I'm wondering if PITR would suffice. Or maybe even Slony. > >>Do you want to share one big, expensive, reliable unit between > >>multiple systems? Will you be doing failover? > >Yes, and Yes. Really on one other system, a phone system, but it is the > >crux of my business and will be writing a lot of recorded phone calls. I am > >working with a storage company now to set up the failover, I want the db > >and > >phone systems to never no if the storage switched over. > > If you actually have a couple of systems you're trying to fail over, a > FC SAN may be a reasonable solution. Depending on your reliability > requirement you can have multiple interfaces & FC switches to get > redundant paths and a much higher level of storage reliability than you > could get with direct attach storage. OTOH, if the DB server itself > breaks you're still out of luck. :) You might compare that sort of > solution with a solution that has redundant servers and implements the > failover in software instead of hardware. BTW, I know a company here in Austin that does capacity planning for complex systems like this; contact me off-list if you're interested in talking to them. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 From mstone+postgres@mathom.us Thu Apr 27 17:57:22 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 3210011F620E for ; Thu, 27 Apr 2006 14:57:22 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 00111-03 for ; Thu, 27 Apr 2006 14:57:12 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from vms046pub.verizon.net (vms046pub.verizon.net [206.46.252.46]) by postgresql.org (Postfix) with ESMTP id 6653A11F61EE for ; Thu, 27 Apr 2006 14:57:10 -0300 (ADT) Received: from osgiliath.mathom.us ([70.108.132.172]) by vms046.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0IYE00GBF77AO6Z5@vms046.mailsrvcs.net> for pgsql-performance@postgresql.org; Thu, 27 Apr 2006 12:57:10 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by osgiliath.mathom.us (Postfix) with ESMTP id 4FB8E6E6C0 for ; Thu, 27 Apr 2006 13:57:08 -0400 (EDT) Received: from osgiliath.mathom.us ([127.0.0.1]) by localhost (osgiliath.home.mathom.us [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 18998-01 for ; Thu, 27 Apr 2006 13:57:07 -0400 (EDT) Received: by osgiliath.mathom.us (Postfix, from userid 1000) id BE6E76E945; Thu, 27 Apr 2006 13:57:06 -0400 (EDT) Date: Thu, 27 Apr 2006 13:57:06 -0400 From: Michael Stone Subject: Re: Running on an NFS Mounted Directory In-reply-to: <20060427175016.GH97354@pervasive.com> To: pgsql-performance@postgresql.org Mail-followup-to: pgsql-performance@postgresql.org Message-id: <20060427175704.GI31328@mathom.us> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-disposition: inline X-Pgp-Fingerprint: 53 FF 38 00 E7 DD 0A 9C 84 52 84 C5 EE DF 7C 88 X-Virus-Scanned: Debian amavisd-new at mathom.us References: <20060427132452.GG31328@mathom.us> <20060427140417.GH31328@mathom.us> <20060427175016.GH97354@pervasive.com> User-Agent: Mutt/1.5.11+cvs20060403 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/651 X-Sequence-Number: 18735 On Thu, Apr 27, 2006 at 12:50:16PM -0500, Jim C. Nasby wrote: >Yes, but some dedicated storage devices actually provide good >performance with RAID5. Most simpler solutions give pretty abysmal write >performance. dedicated storage device != SAN != NAS. You can get good performance in a dedicated direct-attach device without paying for the SAN/NAS infrastructure if you don't need it; you don't have to go right from EMC to PERC with nothing in the middle. Mike Stone From dgorman@hi5.com Thu Apr 27 19:00:06 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 9D50D11F6750 for ; Thu, 27 Apr 2006 16:00:06 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 14449-03 for ; Thu, 27 Apr 2006 15:59:56 -0300 (ADT) X-Greylist: delayed 14:15:58.359001 by SQLgrey- Received: from corpmail.hi5.com (corpmail.hi5.com [204.13.49.5]) by postgresql.org (Postfix) with ESMTP id 88B7811F66E7 for ; Thu, 27 Apr 2006 15:59:55 -0300 (ADT) Received: from [192.168.1.133] by hi5.com (MDaemon PRO v9.0.0 trial) with ESMTP id md50002978775.msg for ; Thu, 27 Apr 2006 11:59:06 -0700 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v749.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Cc: Bruno Wolff III , Michael Stone , Content-Transfer-Encoding: 7bit From: Dan Gorman Subject: Re: Running on an NFS Mounted Directory Date: Thu, 27 Apr 2006 11:58:59 -0700 To: Ketema Harris X-Mailer: Apple Mail (2.749.3) X-Authenticated-Sender: dgorman@hi5.com X-Spam-Processed: corpmail, Thu, 27 Apr 2006 11:59:06 -0700 (not processed: message from valid local sender) X-MDRemoteIP: 66.237.43.131 X-Return-Path: dgorman@hi5.com X-MDaemon-Deliver-To: pgsql-performance@postgresql.org X-MDAV-Processed: corpmail, Thu, 27 Apr 2006 11:59:06 -0700 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/655 X-Sequence-Number: 18739 So do NAS's Dan On Apr 27, 2006, at 6:42 AM, Ketema Harris wrote: > The SAN has the snapshot capability. > > > On 4/27/06 9:31 AM, "Bruno Wolff III" wrote: > >> On Thu, Apr 27, 2006 at 09:06:48 -0400, >> Ketema Harris wrote: >>> Yes, your right, I meant not have to do the backups from the db >>> server >>> itself. I can do that within the storage device now, by >>> allocating space >>> for it, and letting the device copy the data files on some >>> periodic basis. >> >> Only if the database server isn't running or your SAN provides a >> way to >> provide a snapshot of the data at a particular instant in time. > > > > ---------------------------(end of > broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org From null@fortuitous.dot.nul Tue May 2 14:59:04 2006 X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id A4A7B9FA4AF for ; Tue, 2 May 2006 11:59:04 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 29359-03 for ; Tue, 2 May 2006 11:58:55 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from floppy.pyrenet.fr (news.pyrenet.fr [194.116.145.2]) by postgresql.org (Postfix) with ESMTP id 309199F9D4D for ; Tue, 2 May 2006 11:58:54 -0300 (ADT) Received: by floppy.pyrenet.fr (Postfix, from userid 106) id CA81C30CDC; Tue, 2 May 2006 16:58:52 +0200 (MET DST) From: Fortuitous Technologies Subject: Re: Running on an NFS Mounted Directory User-Agent: Pan/0.14.2.91 (As She Crawled Across the Table (Debian GNU/Linux)) Message-Id: X-Newsgroups: pgsql.performance References: <20060427023542.GA14672@weaver.tuc.noao.edu> <20060427045524.GV97354@pervasive.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Lines: 15 Date: Tue, 02 May 2006 14:58:51 GMT X-Complaints-To: abuse@earthlink.net Organization: EarthLink Inc. -- http://www.EarthLink.net To: pgsql-performance@postgresql.org X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200605/74 X-Sequence-Number: 18861 On Wed, 26 Apr 2006 23:55:24 -0500, Jim C. Nasby wrote: > On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote: >> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote: >> > I was wondering if there were any performance issues with having a data >> > directory that was an nfs mounted drive? Say like a SAN or NAS device? Has >> > anyone done this before? >> >> My understanding is that NFS is pretty poor in performance in general, NFS is not a good choice for several reasons. First, NFS takes priority in the system kernel, and will slow down all other operations. Your best choice, as pointed out by others, is a DAS solutions. If you must use NFS, you should consider putting it on a fast dedicated network by itself.