Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id DEBA49FB41A for ; Wed, 4 Oct 2006 18:00:29 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 58811-08 for ; Wed, 4 Oct 2006 18:00:29 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from mail.mi8.com (d01gw04.mi8.com [63.240.6.44]) by postgresql.org (Postfix) with ESMTP id 2ACB99FB203 for ; Wed, 4 Oct 2006 18:00:28 -0300 (ADT) Received: from 172.16.1.239 by mail.mi8.com with ESMTP (- Welcome to Mi8 Corporation www.Mi8.com (D4)); Wed, 04 Oct 2006 17:00:08 -0400 X-Server-Uuid: C8FB4D43-1108-484A-A898-3CBCC7906230 Received: from MI8NYCMAIL06.Mi8.com ([172.16.1.175]) by d01smtp06.Mi8.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 4 Oct 2006 17:00:02 -0400 Received: from 63.80.24.130 ([63.80.24.130]) by MI8NYCMAIL06.Mi8.com ( [172.16.1.219]) via Exchange Front-End Server mi8owa.mi8.com ( [172.16.1.106]) with Microsoft Exchange Server HTTP-DAV ; Wed, 4 Oct 2006 21:00:02 +0000 User-Agent: Microsoft-Entourage/11.2.5.060620 Date: Wed, 04 Oct 2006 14:00:00 -0700 Subject: Re: Netflix Prize data From: "Luke Lonergan" To: "Mark Woodward" , pg@mohawksoft.com, pgsql-hackers@postgresql.org Message-ID: Thread-Topic: [HACKERS] Netflix Prize data Thread-Index: Acbn+A4ITJZMblPrEdutkQAWy4o9DA== In-Reply-To: <18350.24.91.171.78.1159994622.squirrel@mail.mohawksoft.com> MIME-Version: 1.0 X-OriginalArrivalTime: 04 Oct 2006 21:00:03.0074 (UTC) FILETIME=[0FDDD620:01C6E7F8] X-WSS-ID: 693AFF412US19297135-04-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200610/248 X-Sequence-Number: 92145 Mark, On 10/4/06 1:43 PM, "Mark Woodward" wrote: > markw@snoopy:~/netflix$ time psql netflix -c "select count(*) from ratings" > count > ----------- > 100480507 > (1 row) > > > real 2m6.270s > user 0m0.004s > sys 0m0.005s I think you are getting about 40MB/s on your sequential scan of about 5GB of heap data in this case. I calculate the size of the data as: 3 Integers (12 bytes), one text date field (10 bytes ?) and tuple overhead (24 bytes) = 46 bytes per row 100 million rows x 46 bytes / row = 4.6 Gbytes - Luke