Received: from localhost (unknown [200.46.204.182]) by postgresql.org (Postfix) with ESMTP id C67C49FB67E for ; Fri, 7 Sep 2007 15:21:59 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.182]) (amavisd-maia, port 10024) with ESMTP id 02441-03 for ; Fri, 7 Sep 2007 15:21:30 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from smtp.janestcapital.com (www.janestcapital.com [66.155.124.107]) by postgresql.org (Postfix) with ESMTP id 697649FB5E3 for ; Fri, 7 Sep 2007 15:21:54 -0300 (ADT) Received: from [172.25.129.161] [38.96.172.125] by janestcapital.com with ESMTP (SMTPD-9.10) id A6DE0348; Fri, 07 Sep 2007 14:22:22 -0400 Message-ID: <46E196C0.5020508@janestcapital.com> Date: Fri, 07 Sep 2007 14:21:52 -0400 From: Brian Hurt User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bryan Murphy CC: pgsql-performance@postgresql.org Subject: Re: SAN vs Internal Disks References: <20070907100446.GE1795@mathom.us> <20070907103341.GC20896@oppetid.no> <46E18886.7060502@aptalaska.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.298 tagged_above=0 required=5 tests=AWL=-0.833, BAYES_50=0.001, DNS_FROM_OPENWHOIS=1.13 X-Spam-Level: X-Archive-Number: 200709/133 X-Sequence-Number: 26678 Bryan Murphy wrote: >Our database server connects to the san via iSCSI over Gig/E using >jumbo frames. File system is XFS (noatime). > > > > ... >Throughput, however, kinda sucks. I just can't get the kind of >throughput to it I was hoping to get. When our memory cache is blown, >the database can be downright painful for the next few minutes as >everything gets paged back into the cache. > > > Remember that Gig/E is bandwidth limited to about 100 Mbyte/sec. Maybe a little faster than that downhill with a tailwind, but not much. You're going to get much better bandwidth connecting to a local raid card talking to local disks simply due to not having the ethernet as a bottleneck. iSCSI is easy to set up and manage, but it's slow. This is the big advantage Fibre Channel has- serious performance. You can have multiple channels on a single fibre channel card- IIRC, QLogic's cards have a default of 4 channels- each pumping 400 Mbyte/sec. At which point the local bus rapidly becomes the bottleneck. Of course, this comes at the cost of a signifigant increase in complexity. Brian