X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id A61565280F for ; Mon, 16 May 2005 14:38:56 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 07072-10 for ; Mon, 16 May 2005 17:38:46 +0000 (GMT) Received: from svr2.postgresql.org (svr2.postgresql.org [65.19.161.25]) by svr1.postgresql.org (Postfix) with ESMTP id E1E9552883 for ; Mon, 16 May 2005 14:38:15 -0300 (ADT) Received: from mta10.nationwireless.net (mta10.nationwireless.net [63.163.128.10]) by svr2.postgresql.org (Postfix) with ESMTP id E21C2F0DF3 for ; Mon, 16 May 2005 15:21:03 +0100 (BST) Received: from jfradkin [63.162.100.127] by mta10.nationwireless.net with ESMTP (SMTPD-8.20) id A5B006E4; Mon, 16 May 2005 08:44:32 -0400 From: "Joel Fradkin" To: , Subject: Re: Prefetch Date: Mon, 16 May 2005 08:45:01 -0400 Message-ID: <001301c55a15$13647190$797ba8c0@jfradkin> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable In-Reply-To: <200505091910.26407.molson@oceanconsulting.com> X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=1.913 tagged_above=0 required=5 tests=AWL, MSGID_DOLLARS X-Spam-Level: * X-Archive-Number: 200505/303 X-Sequence-Number: 12467 My only comment is what is the layout of your data (just one table with indexes?). I found on my date with dozens of joins my view speed was not good for = me to use, so I made a flat file with no joins and it flies. Joel Fradkin =20 Wazagua, Inc. 2520 Trailmate Dr Sarasota, Florida 34243 Tel. 941-753-7111 ext 305 =20 jfradkin@wazagua.com www.wazagua.com Powered by Wazagua Providing you with the latest Web-based technology & advanced tools. C 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc This email message is for the use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized = review, use, disclosure or distribution is prohibited. If you are not the = intended recipient, please contact the sender by reply email and delete and = destroy all copies of the original message, including attachments. =20 =20 -----Original Message----- From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Matt Olson Sent: Monday, May 09, 2005 9:10 PM To: pgsql-performance@postgresql.org Subject: [PERFORM] Prefetch I wanted to get some opinions about row prefetching. AFAIK, there is no = prefetching done by PostgreSQL; all prefetching is delegated to the operating=20 system. =20 The hardware (can't say enough good things about it): Athlon 64, dual channel 4GB ram 240GB usable 4 disk raid5 (ATA133) Fedora Core 3 PostgreSQL 7.4.7 I have what is essentially a data warehouse of stock data. Each day has = around 30,000 records (tickers). A typical operation is to get the 200 = day=20 simple moving average (of price) for each ticker and write the result to = a=20 summary table. In running this process (Perl/DBI), it is typical to see = 70-80% I/O wait time with postgres running a about 8-9%. If I run the = next day's date, the postgres cache and file cache is now populated with 199 = days of the needed data, postgres runs 80-90% of CPU and total run time is greatly=20 reduced. My conclusion is that this is a high cache hit rate in action. = =20 I've done other things that make sense, like using indexes, playing with = the planner constants and turning up the postgres cache buffers. =20 Even playing with extream hdparm read-ahead numbers (i.e. 64738), there = is no=20 apparent difference in database performance. The random nature of the = I/O=20 drops disk reads down to about 1MB/sec for the array. A linear table = scan=20 can easily yield 70-80MB/sec on this system. Total table size is = usually=20 around 1GB and with indexes should be able to fit completely in main = memory. Other databases like Oracle and DB2 implement some sort of row prefetch. Has=20 there been serious consideration of implementing something like a = prefetch=20 subsystem? Does anyone have any opinions as to why this would be a bad = idea for postgres? =20 Postges is great for a multiuser environment and OLTP applications. However,=20 in this set up, a data warehouse, the observed performance is not what I = would hope for. =20 Regards, Matt Olson Ocean Consulting http://www.oceanconsulting.com/ ---------------------------(end of broadcast)--------------------------- TIP 8: explain analyze is your friend