Received: from localhost (maia-5.hub.org [200.46.204.182]) by postgresql.org (Postfix) with ESMTP id 343FB9FB25B; Thu, 18 Jan 2007 17:53:04 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.182]) (amavisd-new, port 10024) with ESMTP id 49057-01; Thu, 18 Jan 2007 17:52:51 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey-1.7.4 X-Greylist: from auto-whitelisted by SQLgrey-1.7.4 Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id 5DF309FB1D2; Thu, 18 Jan 2007 17:53:00 -0400 (AST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id l0ILqwMd006594; Thu, 18 Jan 2007 16:52:58 -0500 (EST) To: Brian Hurt cc: pgsql-performance@postgresql.org, pgsql-advocacy@postgresql.org Subject: Re: [PERFORM] Postgres and really huge tables In-reply-to: <45AFD927.9050605@janestcapital.com> References: <4939.1164740825@sss.pgh.pa.us> <200611282344.kASNijt09203@momjian.us> <456CE066.50803@cheapcomplexdevices.com> <456D8A65.3080400@janestcapital.com> <1164812624.9690.500.camel@archimedes> <456DA476.4000606@janestcapital.com> <456DB8C9.7020206@cheapcomplexdevices.com> <45AFD927.9050605@janestcapital.com> Comments: In-reply-to Brian Hurt message dated "Thu, 18 Jan 2007 15:31:35 -0500" Date: Thu, 18 Jan 2007 16:52:58 -0500 Message-ID: <6593.1169157178@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200701/57 X-Sequence-Number: 10539 Brian Hurt writes: > Is there any experience with Postgresql and really huge tables? I'm > talking about terabytes (plural) here in a single table. The 2MASS sky survey point-source catalog http://www.ipac.caltech.edu/2mass/releases/allsky/doc/sec2_2a.html is 470 million rows by 60 columns; I don't have it loaded up but a very conservative estimate would be a quarter terabyte. (I've got a copy of the data ... 5 double-sided DVDs, gzipped ...) I haven't heard from Rae Stiening recently but I know he's been using Postgres to whack that data around since about 2001 (PG 7.1 or so, which is positively medieval compared to current releases). So at least for static data, it's certainly possible to get useful results. What are your processing requirements? regards, tom lane