X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 57DE7DAD2C for ; Wed, 16 Nov 2005 02:12:49 -0400 (AST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 87854-10 for ; Wed, 16 Nov 2005 06:12:48 +0000 (GMT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from mail.mi8.com (d01gw04.mi8.com [63.240.6.44]) by svr1.postgresql.org (Postfix) with ESMTP id 557D6D6837 for ; Wed, 16 Nov 2005 02:12:45 -0400 (AST) Received: from 172.16.1.110 by mail.mi8.com with ESMTP (- Welcome to Mi8 Corporation www.Mi8.com (D4)); Wed, 16 Nov 2005 01:12:11 -0500 X-Server-Uuid: C8FB4D43-1108-484A-A898-3CBCC7906230 Received: from MI8NYCMAIL06.Mi8.com ([172.16.1.175]) by D01SMTP01.Mi8.com with Microsoft SMTPSVC(6.0.3790.211); Wed, 16 Nov 2005 01:11:42 -0500 Received: from 69.181.100.71 ([69.181.100.71]) by MI8NYCMAIL06.Mi8.com ( [172.16.1.219]) via Exchange Front-End Server mi8owa.mi8.com ( [172.16.1.104]) with Microsoft Exchange Server HTTP-DAV ; Wed, 16 Nov 2005 01:11:41 -0500 User-Agent: Microsoft-Entourage/11.2.1.051004 Date: Tue, 15 Nov 2005 22:11:39 -0800 Subject: Re: Hardware/OS recommendations for large databases ( From: "Luke Lonergan" To: "James Mello" , "William Yu" , pgsql-performance@postgresql.org Message-ID: Thread-Topic: [PERFORM] Hardware/OS recommendations for large databases ( 5TB) Thread-Index: AcXqF1J+4GMu4DJdRoSAkms/8lt9bwAADWwwABdEvjs= In-Reply-To: <775F825B46B551499DC75DC4E92F80AB0E3623@jupiter.ad.haydrian.com> MIME-Version: 1.0 X-OriginalArrivalTime: 16 Nov 2005 06:11:42.0485 (UTC) FILETIME=[9D39E450:01C5EA74] X-WSS-ID: 6F64123131S12018823-01-01 Content-Type: multipart/alternative; boundary=B_3214937500_22997028 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=1.254 required=5 tests=[HTML_MESSAGE=0.001, RCVD_NUMERIC_HELO=1.253] X-Spam-Score: 1.254 X-Spam-Level: * X-Archive-Number: 200511/266 X-Sequence-Number: 15523 > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3214937500_22997028 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable James, On 11/15/05 11:07 AM, "James Mello" wrote: > Unless there was a way to guarantee consistency, it would be hard at > best to make this work. Convergence on large data sets across boxes is > non-trivial, and diffing databases is difficult at best. Unless there > was some form of automated way to ensure consistency, going 8 ways into > separate boxes is *very* hard. I do suppose that if you have fancy > storage (EMC, Hitachi) you could do BCV or Shadow copies. But in terms > of commodity stuff, I'd have to agree with Merlin. It=B9s a matter of good software that handles the distribution / parallel query optimization / distributed transactions and management features. Combine that with a gigabit ethernet switch and it works =AD we routinely get 50x speedup over SMP on OLAP / Decision Support workloads. Regards, - Luke --B_3214937500_22997028 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)= James= ,


On 11/15/05 11:07 AM, "James Mello" <james@haydrian.com> wr= ote:

Unless there was a way to guarantee consistency, it wou= ld be hard at
best to make this work. Convergence on large data sets across boxes is
non-trivial, and diffing databases is difficult at best. Unless there
was some form of automated way to ensure consistency, going 8 ways into
separate boxes is *very* hard. I do suppose that if you have fancy
storage (EMC, Hitachi) you could do BCV or Shadow copies. But in terms
of commodity stuff, I'd have to agree with Merlin.

It’s a matter of good software that handles the distribution / parall= el query optimization / distributed transactions and management features. &n= bsp;Combine that with a gigabit ethernet switch and it works – we rout= inely get 50x speedup over SMP on OLAP / Decision Support workloads.

Regards,

- Luke --B_3214937500_22997028--