X-Original-To: pgsql-general-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 3450E5E4722 for ; Tue, 7 Sep 2004 17:18:10 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 58833-03 for ; Tue, 7 Sep 2004 16:18:05 +0000 (GMT) Received: from silmaril.syscor.priv (h24-108-208-253.sbm.shawcable.net [24.108.208.253]) by svr1.postgresql.org (Postfix) with ESMTP id B00685E4813 for ; Tue, 7 Sep 2004 17:18:01 +0100 (BST) Received: from syscor.com (strider.syscor.priv [192.168.1.3]) by silmaril.syscor.priv (8.12.8/8.12.8) with ESMTP id i87GI0Ye010908 for ; Tue, 7 Sep 2004 09:18:00 -0700 Message-ID: <413DDFB7.1060006@syscor.com> Date: Tue, 07 Sep 2004 09:20:07 -0700 From: Ron St-Pierre User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: pgsql-general Subject: Re: [PERFORM] Table UPDATE is too slow References: <4134BF36.3000504@syscor.com> In-Reply-To: <4134BF36.3000504@syscor.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200409/271 X-Sequence-Number: 65477 Ron St-Pierre wrote: > We have a web based application with data that is updated daily. The > biggest bottleneck occurs when we try to update > one of the tables. This table contains 58,000 rows and 62 columns, and > EVERY column is indexed. Every column is > queryable (?) by the users through the web interface so we are > reluctant to remove the indexes (recreating them would > be time consuming too). The primary key is an INT and the rest of the > columns are a mix of NUMERIC, TEXT, and DATEs. > A typical update is: > UPDATE dataTable SET field01=44.5, field02=44.5, field03='Bob', > field04='foo', ... , field60='2004-08-30', field61='2004-08-29' > WHERE id = 1234; > > Also of note is that the update is run about 10 times per day; we get > blocks of data from 10 different sources, so we pre-process the > data and then update the table. We also run VACUUM FULL ANALYZE on a > nightly basis. It now appears that VACUUM wasn't running properly. A manual VACUUM FULL ANALYZE VEBOSE told us that approximately 275000 total pages were needed. I increased the max_fsm_pages to 300000, VACUUMED, renamed the database and re-created it from backup, vacuumed numerous times, and the total fsm_pages needed continued to remain in the 235000 -> 270000 range. This morning I deleted the original (renamed) database, and a VACUUM FULL ANALYZE VEBOSE now says that only about 9400 pages are needed. One question about redirecting VACUUMs output to file though. When I run: psql -d imperial -c "vacuum full verbose analyze;" > vac.info vac.info contains only the following line: VACUUM I've been unable to capture the VERBOSE output to file. Any suggestions? > Also, thanks for everyone's input about my original posting, I am investigating some of the options mentioned to further increase performance. Ron