Message-ID: <4647309D.4090805@hagander.net>
Date: Sun, 13 May 2007 17:37:01 +0200
From: Magnus Hagander <magnus@hagander.net>
User-Agent: Thunderbird 2.0.0.4pre (Windows/20070512)
MIME-Version: 1.0
To: Heikki Linnakangas <heikki@enterprisedb.com>
CC: "Jim C. Nasby" <decibel@decibel.org>, Greg Smith <gsmith@gregsmith.com>,
	pgsql-hackers@postgresql.org
Subject: Re: Performance monitoring
References: <Pine.GSO.4.64.0704010051050.14920@westnet.com>
	<200704030814.33999.peter_e@gmx.net>
	<Pine.GSO.4.64.0704030927180.5732@westnet.com>
	<46443F02.6060300@enterprisedb.com>
	<Pine.GSO.4.64.0705111009080.18685@westnet.com>
	<46448DC8.4050102@enterprisedb.com>
	<20070512211948.GA58786@nasby.net>
	<4646B61C.1010103@enterprisedb.com>
In-Reply-To: <4646B61C.1010103@enterprisedb.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Heikki Linnakangas wrote:
>>> Yeah, if we have the summary line we don't need the other lines and
>>> vice versa. I have sympathy for parsing log files, I've done that a
>>> lot in the past and I can see what you mean. Having the individual
>>> lines is nice when you're monitoring a running system; you don't get
>>> the summary line until the checkpoint is finished. I suppose we can
>>> have both the individual lines and the summary, the extra lines
>>> shouldn't hurt anyone, and you won't get them unless you turn on the
>>> new log_checkpoints parameter anyway.
>>
>> Not to beat a dead horse, but do we really want to force folks to be
>> parsing logs for performance monitoring? Especially if that log parsing
>> is just going to result in data being inserted into a table anyway?
>>
>> I know there's concern about performance of the stats system and maybe
>> that needs to be addressed, but pushing users to log parsing is a lot of
>> extra effort, non-standard, likely to be overlooked, and doesn't play
>> well with other tools. It also conflicts with all the existing
>> statistics framework.
> 
> There is two counters for checkpoints in pgstats, the number of timed
> (triggered by checkpoint_timeout) and requested (triggered by
> checkpoint_segments) checkpoints.
> 
> Maybe we should improve the stats system so that we can collect events
> with timestamps and durations, but in my experience log files actually
> are the most reliable and universal way to collect real-time performance
> information. Any serious tool has a generic log parser. The other
> alternative is SNMP. I welcome the efforts on pgsnmpd..

pgsnmpd can't provide any information that's not in the backend. Unless
we'd turn it into a log parser, which is not really something I think is
a good idea.

Log files are great for one kind of thing, "live statistics" through
SNMP or the statistics collector for another kind. It only gets wrong
when you put them in the wrong place. Things you poll regularly makes a
lot more sense in some kind of live view than in a log file.

//Magnus