Received: from maia.hub.org (unknown [200.46.208.211]) by mail.postgresql.org (Postfix) with ESMTP id 3310F634623 for ; Tue, 6 Apr 2010 04:39:13 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.208.211]) (amavisd-maia, port 10024) with ESMTP id 71616-09 for ; Tue, 6 Apr 2010 07:38:58 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from yoda.tecnocon.com (mail.technocon.com [62.132.195.121]) by mail.postgresql.org (Postfix) with ESMTP id 1DFE46342BE for ; Tue, 6 Apr 2010 04:39:01 -0300 (ADT) Received: from yoda (localhost [127.0.0.1]) by yoda.tecnocon.com (Postfix) with ESMTP id 4D1DC4B694A; Tue, 6 Apr 2010 09:39:00 +0200 (CEST) Received: from localhost ([127.0.0.1]) by yoda with SMTP ID 116; Tue, 6 Apr 2010 09:39:00 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by yoda.tecnocon.com (Postfix) with ESMTP id E5D834B6954; Tue, 6 Apr 2010 09:38:59 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at technocon.com Received: from yoda.tecnocon.com ([127.0.0.1]) by localhost (yoda.dhcp.tecnocon.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jj95n92XVMW9; Tue, 6 Apr 2010 09:38:54 +0200 (CEST) Received: from yoda.dhcp.tecnocon.com (localhost [127.0.0.1]) by yoda.tecnocon.com (Postfix) with ESMTP id 276D24B694A; Tue, 6 Apr 2010 09:38:54 +0200 (CEST) Subject: Re: Using high speed swap to improve performance? From: =?windows-1252?Q?Christiaan_Willemsen?= To: =?windows-1252?Q?Scott_Marlowe?= Cc: pgsql-performance@postgresql.org Date: Tue, 6 Apr 2010 09:38:54 +0200 Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=_ekKKrUjatW5HHlVtdfGYXu-TZnGVbkUCJkVerRHLt5qwH2ey" X-Priority: 3 (Normal) X-Mailer: Zarafa 6.20.11-17604 Message-Id: X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-2.598 tagged_above=-10 required=5 tests=BAYES_00=-2.599, HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 201004/46 X-Sequence-Number: 38296 This is a multi-part message in MIME format. Your mail reader does not understand MIME message format. --=_ekKKrUjatW5HHlVtdfGYXu-TZnGVbkUCJkVerRHLt5qwH2ey Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable =0D=0AHi Scott,=0D=0A=0D=0A=A0=0D=0A=0D=0AThat sound like a usefull thing= to do, but the big advantage of the SAN is that in case the physical mac= hine goes down, I can quickly startup a virtual machine using the same da= tabase files to act as a fallback. It will have less memory, and less CPU= 's but it will do fine for some time.=0D=0A=0D=0A=A0=0D=0A=0D=0ASo when p= utting fast tables on local storage, I losse those tables when the machin= e goes down.=0D=0A=0D=0A=A0=0D=0A=0D=0APutting indexes on there however m= ight me intresting.. What will Postgresql do when it is started on the ba= ckupmachine, and it finds out the index files are missing=3F Will it recr= eate those files, or will it panic and not start at all, or can we just m= anually reindex=3F=0D=0A=0D=0A=A0=0D=0A=0D=0AKind regards,=0D=0A=0D=0A=A0= =0D=0A=0D=0AChristiaan=0D=0A=A0=0D=0A-----Original message-----=0D=0AFrom= : Scott Marlowe =0D=0ASent: Sun 04-04-2010 23:08= =0D=0ATo: Christiaan Willemsen ;=20=0D=0ACC: pg= sql-performance@postgresql.org;=20=0D=0ASubject: Re: [PERFORM] Using high= speed swap to improve performance=3F=0D=0A=0D=0AOn Fri, Apr 2, 2010 at 1= :15 PM, Christiaan Willemsen=0D=0A wrote:=0D=0A= > Hi there,=0D=0A>=0D=0A> About a year ago we setup a machine with sixtee= n 15k disk spindles on=0D=0A> Solaris using ZFS. Now that Oracle has take= n Sun, and is closing up Solaris,=0D=0A> we want to move away (we are mor= e familiar with Linux anyway).=0D=0A>=0D=0A> So the plan is to move to Li= nux and put the data on a SAN using iSCSI (two=0D=0A> or four network int= erfaces). This however leaves us with with 16 very nice=0D=0A> disks dooi= ng nothing. Sound like a wast of time. If we were to use Solaris,=0D=0A> = ZFS would have a solution: use it as L2ARC. But there is no Linux filesys= tem=0D=0A> with those features (ZFS on fuse it not really an option).=0D=0A= >=0D=0A> So I was thinking: Why not make a big fat array using 14 disks (= raid 1, 10=0D=0A> or 5), and make this a big and fast swap disk. Latency = will be lower than=0D=0A> the SAN can provide, and throughput will also b= e better, and it will relief=0D=0A> the SAN from a lot of read iops.=0D=0A= >=0D=0A> So I could create a 1TB swap disk, and put it onto the OS next t= o the 64GB=0D=0A> of memory. Then I can set Postgres to use more than the= RAM size so it will=0D=0A> start swapping. It would appear to postgres t= hat the complete database will=0D=0A> fit into memory. The question is: w= ill this do any good=3F And if so: what=0D=0A> will happen=3F=0D=0A=0D=0A= I'd make a couple of RAID-10s out of it and use them for highly used=0D=0A= tables and / or indexes etc...=0D=0A=0D=0A --=_ekKKrUjatW5HHlVtdfGYXu-TZnGVbkUCJkVerRHLt5qwH2ey Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable =0A=0A =0A =0A RE: = Re: [PERFORM] Using high speed swap to improve performance=3F=0A = =0A=0A=0A=

Hi Scott,

 

That sound like a usefull thing to do, bu= t the big advantage of the SAN is that in case the physical machine goes = down, I can quickly startup a virtual machine using the same database fil= es to act as a fallback. It will have less memory, and less CPU's but= it will do fine for some time.

 

So when putting fast t= ables on local storage, I losse those tables when the machine goes down.<= /p>

 

Putting indexes on there however might me intresting..= What will Postgresql do when it is started on the backupmachine, and it = finds out the index files are missing=3F Will it recreate those files, or= will it panic and not start at all, or can we just manually reindex=3F

 

Kind regards,

 

Christiaan
 =

-----Original message-----
From: Scott Marlowe <scott.marlowe@gmail.com>
Sent: Sun 04-04-2010 23:08
To: Christiaan Willemsen &l= t;cwillemsen@technocon.com>;
CC: pgsql-performa= nce@postgresql.org;
Subject: Re: [PERFORM] Using h= igh speed swap to improve performance=3F

On Fri, Apr 2, 2010 a= t 1:15 PM, Christiaan Willemsen
<cwillemsen@technocon.com> wro= te:
> Hi there,
>
> About a year ago we setup a m= achine with sixteen 15k disk spindles on
> Solaris using ZFS. Now= that Oracle has taken Sun, and is closing up Solaris,
> we want = to move away (we are more familiar with Linux anyway).
>
>= ; So the plan is to move to Linux and put the data on a SAN using iSCSI (= two
> or four network interfaces). This however leaves us with wi= th 16 very nice
> disks dooing nothing. Sound like a wast of time= =2E If we were to use Solaris,
> ZFS would have a solution: use i= t as L2ARC. But there is no Linux filesystem
> with those feature= s (ZFS on fuse it not really an option).
>
> So I was thi= nking: Why not make a big fat array using 14 disks (raid 1, 10
> = or 5), and make this a big and fast swap disk. Latency will be lower than=
> the SAN can provide, and throughput will also be better, and i= t will relief
> the SAN from a lot of read iops.
>
&= gt; So I could create a 1TB swap disk, and put it onto the OS next to the= 64GB
> of memory. Then I can set Postgres to use more than the R= AM size so it will
> start swapping. It would appear to postgres = that the complete database will
> fit into memory. The question i= s: will this do any good=3F And if so: what
> will happen=3F

I'd make a couple of RAID-10s out of it and use them for highl= y used
tables and / or indexes etc...

=0A=0A --=_ekKKrUjatW5HHlVtdfGYXu-TZnGVbkUCJkVerRHLt5qwH2ey--