X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id F139E329C82 for ; Tue, 14 Sep 2004 01:51:30 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 24605-05 for ; Tue, 14 Sep 2004 00:51:23 +0000 (GMT) Received: from mproxy.gmail.com (rproxy.gmail.com [64.233.170.197]) by svr1.postgresql.org (Postfix) with ESMTP id 8424F329E7F for ; Tue, 14 Sep 2004 01:51:25 +0100 (BST) Received: by mproxy.gmail.com with SMTP id v18so449157rnb for ; Mon, 13 Sep 2004 17:51:22 -0700 (PDT) Received: by 10.38.8.74 with SMTP id 74mr1923039rnh; Mon, 13 Sep 2004 17:51:22 -0700 (PDT) Received: by 10.38.8.4 with HTTP; Mon, 13 Sep 2004 17:51:22 -0700 (PDT) Message-ID: <3f71fdf104091317515c756512@mail.gmail.com> Date: Mon, 13 Sep 2004 19:51:22 -0500 From: Stephen Crowley Reply-To: Stephen Crowley To: pgsql-performance@postgresql.org Subject: Large # of rows in query extremely slow, not using index Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200409/95 X-Sequence-Number: 8223 Does postgres cache the entire result set before it begins returning data to the client? I have a table with ~8 million rows and I am executing a query which should return about ~800,000 rows. The problem is that as soon as I execute the query it absolutely kills my machine and begins swapping for 5 or 6 minutes before it begins returning results. Is postgres trying to load the whole query into memory before returning anything? Also, why would it choose not to use the index? It is properly estimating the # of rows returned. If I set enable_seqscan to off it is just as slow. Running postgres 8.0 beta2 dev2 explain select * from island_history where date='2004-09-07' and stock='QQQ'; QUERY PLAN --------------------------------------------------------------------------- Seq Scan on island_history (cost=0.00..266711.23 rows=896150 width=83) Filter: ((date = '2004-09-07'::date) AND ((stock)::text = 'QQQ'::text)) (2 rows) Any help would be appreciated --Stephen Table "public.island_history" Column | Type | Modifiers ------------------+------------------------+----------- date | date | not null stock | character varying(6) | time | time without time zone | not null reference_number | numeric(9,0) | not null message_type | character(1) | not null buy_sell_ind | character(1) | shares | numeric(6,0) | remaining_shares | numeric(6,0) | price | numeric(10,4) | display | character(1) | match_number | numeric(9,0) | not null Indexes: "island_history_pkey" PRIMARY KEY, btree (date, reference_number, message_type, "time", match_number) "island_history_date_stock_time" btree (date, stock, "time") "island_history_oid" btree (oid)