From: Ed L. <pgsql@bluepolka.net>
To: George Pavlov <gpavlov@mynewplace.com>
Cc: pgsql-general@postgresql.org
Subject: Re: query log corrupted-looking entries
Date: Fri, 1 Jun 2007 15:20:19 -0600
Message-ID: <200706011520.19227.pgsql@bluepolka.net> (raw)
In-Reply-To: <8C5B026B51B6854CBE88121DBF097A86CDD9A4@ehost010-33.exch010.intermedia.net>
References: <29860.1161202644@sss.pgh.pa.us>
<200705291119.55567.pgsql@bluepolka.net>
<8C5B026B51B6854CBE88121DBF097A86CDD9A4@ehost010-33.exch010.intermedia.net>
On Friday 01 June 2007 3:09 pm, George Pavlov wrote:
> On 5/29/2007 10:19 AM, Ed L. wrote:
> > On Wednesday 23 May 2007 1:04 pm, George Pavlov wrote:
> > FWIW, I've also been seeing this sort of query log
> > corruption for as long as I can remember, 7.1 through 8.2,
> > HPUX (parisc, ia64), Linux on intel, amd...
>
> Do you have any tricks for dealing with the problem from a
> query analyzer perspective? That is, if you use something like
> pgfouine do you have any quick and easy way to remove those
> lines (and the affected lines around them)? Or someway to
> "fix" the corrupted lines? I'd say that lately ~40% of my
> daily query logs are suffering from this problem making query
> analysis very hard.
Not really. I filter by perl regular expressions, but I wouldn't
say that is reliable. However, our query volumes are high
enough that losing a few here and there is no big deal in
analysis. I long ago realized that reliably replaying query
logs was infeasible due in part to this issue. Regex filtering
is problematic and error-prone (for example, how do you reliably
tell the difference between log timestamp and DB data?).
Perhaps others have better answers.
Ed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pgsql@bluepolka.net, gpavlov@mynewplace.com, pgsql-general@postgresql.org
Subject: Re: query log corrupted-looking entries
In-Reply-To: <200706011520.19227.pgsql@bluepolka.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox