X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 0305ED94F8; Mon, 3 Oct 2005 18:52:37 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 47710-03; Mon, 3 Oct 2005 21:52:33 +0000 (GMT) Received: from mail.Mi8.com (d01gw01.mi8.com [63.240.6.47]) by svr1.postgresql.org (Postfix) with ESMTP id 9CF35D9ED1; Mon, 3 Oct 2005 18:52:32 -0300 (ADT) Received: from 172.16.1.110 by mail.Mi8.com with ESMTP (- Welcome to Mi8 Corporation www.Mi8.com (D1)); Mon, 03 Oct 2005 17:52:28 -0400 X-Server-Uuid: 241911D6-425B-44B9-A073-E3FE0F8FC774 Received: from MI8NYCMAIL06.Mi8.com ([172.16.1.175]) by D01SMTP01.Mi8.com with Microsoft SMTPSVC(6.0.3790.211); Mon, 3 Oct 2005 17:52:27 -0400 Received: from 67.103.45.218 ([67.103.45.218]) by MI8NYCMAIL06.Mi8.com ( [172.16.1.219]) via Exchange Front-End Server mi8owa.mi8.com ( [172.16.1.104]) with Microsoft Exchange Server HTTP-DAV ; Mon, 3 Oct 2005 17:52:27 -0500 User-Agent: Microsoft-Entourage/11.2.0.050811 Date: Mon, 03 Oct 2005 14:52:27 -0700 Subject: Re: [HACKERS] A Better External Sort? From: "Luke Lonergan" To: "Hannu Krosing" , "Josh Berkus" Cc: "Jeffrey W. Baker" , pgsql-hackers@postgresql.org, "Michael Stone" , pgsql-performance@postgresql.org Message-ID: Thread-Topic: [HACKERS] [PERFORM] A Better External Sort? Thread-Index: AcXIZL6a/Q9jgTRXEdq40gANk63kWA== In-Reply-To: <1128375790.5882.18.camel@fuji.krosing.net> MIME-Version: 1.0 X-OriginalArrivalTime: 03 Oct 2005 21:52:27.0957 (UTC) FILETIME=[BF2C6650:01C5C864] X-WSS-ID: 6F5F77964JK176011-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=1.574 required=5 tests=[AWL=-0.007, FORGED_RCVD_HELO=0.05, RCVD_NUMERIC_HELO=1.531] X-Spam-Level: * X-Archive-Number: 200510/33 X-Sequence-Number: 14801 Hannu, On 10/3/05 2:43 PM, "Hannu Krosing" wrote: > Just FYI, I run a count(*) on a 15.6GB table on a lightly loaded db and > it run in 163 sec. (Dual opteron 2.6GHz, 6GB RAM, 6 x 74GB 15k disks in > RAID10, reiserfs). A little less than 100MB sec. This confirms our findings - sequential scan is CPU limited at about 120MB/s per single threaded executor. This is too slow for fast file systems like we're discussing here. Bizgres MPP gets 250MB/s by running multiple scanners, but we still chew up unnecessary amounts of CPU. > After this I ran count(*) over a 2.4GB file from another tablespace on > another device (4x142GB 10k disks in RAID10) and it run 22.5 sec on > first run and 12.5 on second. You're getting caching effects here. - Luke