X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 2131D3A477A for ; Thu, 18 Nov 2004 20:29:48 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 27289-05 for ; Thu, 18 Nov 2004 20:29:44 +0000 (GMT) Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by svr1.postgresql.org (Postfix) with SMTP id C4BC33A47D6 for ; Thu, 18 Nov 2004 20:29:45 +0000 (GMT) Received: from unknown (HELO jupiter.black-lion.info) (janwieck@68.80.245.191 with login) by smtp014.mail.yahoo.com with SMTP; 18 Nov 2004 20:29:44 -0000 Received: from [172.21.8.18] ([192.168.192.101]) (authenticated bits=0) by jupiter.black-lion.info (8.12.10/8.12.9) with ESMTP id iAIKTc94040434; Thu, 18 Nov 2004 15:29:39 -0500 (EST) (envelope-from JanWieck@Yahoo.com) Message-ID: <419CF81B.9020406@Yahoo.com> Date: Thu, 18 Nov 2004 14:29:31 -0500 From: Jan Wieck User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.1) Gecko/20040707 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tom Lane Cc: David Parker , pgsql-hackers@postgreSQL.org, "Matthew T. O'Connor" Subject: Re: Timing of pgstats updates References: <07FDEE0ED7455A48AC42AC2070EDFF7C26BAC3@corpsrv2.tazznetworks.com> <17519.1100796192@sss.pgh.pa.us> In-Reply-To: <17519.1100796192@sss.pgh.pa.us> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200411/648 X-Sequence-Number: 61220 On 11/18/2004 11:43 AM, Tom Lane wrote: > "David Parker" writes: >> What I think is happening with the missing pg_statistic entries: >> The install of our application involves a lot of data importing (via >> JDBC) in one large transaction, which can take up to 30 minutes. (I >> realize I left out this key piece of info in my original post...) > >> The pg_autovacuum logic is relying on data from pg_stat_all_tables to >> make the decision about running analyze. As far as I can tell, the data >> in this view gets updated outside of the transaction, because I saw the >> numbers growing while I was importing. I saw pg_autovacuum log messages >> for running analyze on several tables, but no statistics data showed up >> for these, I assume because the actual data in the table wasn't yet >> visible to pg_autovacuum because the import transaction had not finished >> yet. > >> When the import finished, not all of the tables affected by the import >> were re-visited because they had not bumped up over the threshold again, >> even though the analyze run for those tables had not generated any stats >> because of the still-open transaction. > > Bingo. The per-table activity stats are sent to the collector whenever > the backend waits for a client command. Given a moderately long > transaction block doing updates, it's not hard at all to imagine that > autovacuum would kick off vacuum and/or analyze while the updating > transaction is still in progress. The resulting operation is of course > a waste of time. > > It'd be trivial to adjust postgres.c so that per-table stats are > only transmitted when we exit the transaction (basically move the > pgstat_report_tabstat call down a couple lines so it's not called if > IsTransactionOrTransactionBlock). > > This seems like a good change to me. Does anyone not like it? > > regards, tom lane Sounds reasonable here. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #