X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 13BA53A46D0 for ; Thu, 18 Nov 2004 17:14:05 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 62838-09 for ; Thu, 18 Nov 2004 17:14:02 +0000 (GMT) Received: from net2.micro-automation.com (net2.micro-automation.com [64.7.141.29]) by svr1.postgresql.org (Postfix) with SMTP id 0BFD33A46D3 for ; Thu, 18 Nov 2004 17:14:02 +0000 (GMT) Received: (qmail 5382 invoked from network); 18 Nov 2004 17:13:56 -0000 Received: from dcdsl.ebox.com (HELO ?192.168.1.36?) (davec@64.7.143.116) by net2.micro-automation.com with SMTP; 18 Nov 2004 17:13:56 -0000 Message-ID: <419CD858.1000201@fastcrypt.com> Date: Thu, 18 Nov 2004 12:14:00 -0500 From: Dave Cramer Reply-To: pg@fastcrypt.com Organization: Postgres International User-Agent: Mozilla Thunderbird 0.9 (X11/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Andrew Janian Cc: pgsql-performance@postgresql.org Subject: Re: Query Performance and IOWait References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200411/290 X-Sequence-Number: 9244 Andrew, Dell's aren't well known for their disk performance, apparently most of the perc controllers sold with dell's are actually adaptec controllers. Also apparently they do not come with the battery required to use the battery backed up write cache ( In fact according to some Dell won't even sell the battery to you). Also Dell's monitoring software is quite a memory hog. Have you looked at top ?, and also hdparm -Tt /dev/sd? Dave Andrew Janian wrote: >Hello All, > >I have a setup with a Dell Poweredge 2650 with Red Hat and Postgres 7.4.5 with a database with about 27GB of data. The table in question has about 35 million rows. > >I am running the following query: > >SELECT * >FROM mb_fix_message >WHERE msg_client_order_id IN ( > SELECT msg_client_order_id > FROM mb_fix_message > WHERE msg_log_time >= '2004-06-01' > AND msg_log_time < '2004-06-01 13:30:00.000' > AND msg_message_type IN ('D','G') > AND mb_ord_type = '1' > ) > AND msg_log_time > '2004-06-01' > AND msg_log_time < '2004-06-01 23:59:59.999' > AND msg_message_type = '8' > AND (mb_raw_text LIKE '%39=1%' OR mb_raw_text LIKE '%39=2%'); > >with the following plan: > > QUERY PLAN >Nested Loop IN Join (cost=0.00..34047.29 rows=1 width=526) > -> Index Scan using mfi_log_time on mb_fix_message (cost=0.00..22231.31 rows=2539 width=526) > Index Cond: ((msg_log_time > '2004-06-01 00:00:00'::timestamp without time zone) AND (msg_log_time < '2004-06-01 23:59:59.999'::timestamp without time zone)) > Filter: (((msg_message_type)::text = '8'::text) AND (((mb_raw_text)::text ~~ '%39=1%'::text) OR ((mb_raw_text)::text ~~ '%39=2%'::text))) > -> Index Scan using mfi_client_ordid on mb_fix_message (cost=0.00..445.56 rows=1 width=18) > Index Cond: (("outer".msg_client_order_id)::text = (mb_fix_message.msg_client_order_id)::text) > Filter: ((msg_log_time >= '2004-06-01 00:00:00'::timestamp without time zone) AND (msg_log_time < '2004-06-01 13:30:00'::timestamp without time zone) AND ((msg_message_type)::text = 'D'::text) OR ((msg_message_type)::text = 'G'::text)) AND ((mb_ord_type)::text = '1'::text)) > >While running, this query produces 100% iowait usage on its processor and takes a ungodly amount of time (about an hour). > >The postgres settings are as follows: > >shared_buffers = 32768 # min 16, at least max_connections*2, 8KB each >sort_mem = 262144 # min 64, size in KB > >And the /etc/sysctl.conf has: >kernel.shmall = 274235392 >kernel.shmmax = 274235392 > >The system has 4GB of RAM. > >I am pretty sure of these settings, but only from my reading of the docs and others' recommendations online. > >Thanks, > >Andrew Janian >OMS Development >Scottrade Financial Services >(314) 965-1555 x 1513 >Cell: (314) 369-2083 > >---------------------------(end of broadcast)--------------------------- >TIP 7: don't forget to increase your free space map settings > > -- Dave Cramer http://www.postgresintl.com 519 939 0336 ICQ#14675561