pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Martijn van Oosterhout <kleptog@svana.org>
To: George Pavlov <gpavlov@mynewplace.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-general@postgresql.org, "Ed L." <pgsql@bluepolka.net>
Cc: Alvaro Herrera <alvherre@commandprompt.com>
Subject: Re: query log corrupted-looking entries
Date: Fri, 8 Jun 2007 19:21:37 +0200
Message-ID: <20070608172137.GC9818@svana.org> (raw)
In-Reply-To: <8C5B026B51B6854CBE88121DBF097A86DEA7AE@ehost010-33.exch010.intermedia.net>
References: <8C5B026B51B6854CBE88121DBF097A86C39FC2@ehost010-33.exch010.intermedia.net>
	<200705291119.55567.pgsql@bluepolka.net>
	<8C5B026B51B6854CBE88121DBF097A86CDD9A4@ehost010-33.exch010.intermedia.net>
	<24195.1180733773@sss.pgh.pa.us>
	<8C5B026B51B6854CBE88121DBF097A86CDD9D2@ehost010-33.exch010.intermedia.net>
	<24885.1180736713@sss.pgh.pa.us>
	<8C5B026B51B6854CBE88121DBF097A86CDDA32@ehost010-33.exch010.intermedia.net>
	<8C5B026B51B6854CBE88121DBF097A86DEA716@ehost010-33.exch010.intermedia.net>
	<11458.1181265461@sss.pgh.pa.us>
	<8C5B026B51B6854CBE88121DBF097A86DEA7AE@ehost010-33.exch010.intermedia.net>

On Fri, Jun 08, 2007 at 09:30:21AM -0700, George Pavlov wrote:
> As to the full length of the entries that get
> interrupted they do seem to be all on the long side--I can't say with
> total certainty, but the couple of dozen that I looked at were all >
> 4096 when all the interruptions are taken out. So I think I can say that
> I see corruptions happen within "long" entries, but they can happen
> anywhere within that long entry, and one can have multiple interruptions
> within one entry.

That sounds about right. There's a requirement to be atomic for writes
less than PIPE_BUF, but for larger there are no guarentees at all. I
imagine the kernel noted the write was large and proceeded to fill the
last block of the buffer upto a round 4k. That would basically mean
they could get chopped into peices of any size.

Incidently, on systems I've worked with logging went through syslog and
it seems to split long lines at 80 characters or so:

[528-1] SELECT fo...
[528-2] ...
etc...

I always assumed it was postgres doing the splitting but perhaps it was
syslog all along.

Have a nice day,
-- 
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Attachments:

  signature.asc (application/pgp-signature, 188B) download

view thread (30+ messages)

Message-ID: <20070608172137.GC9818@svana.org>
Permalink:  ../20070608172137.GC9818@svana.org/
Also on:    postgresql.org/message-id/20070608172137.GC9818@svana.org

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: kleptog@svana.org, gpavlov@mynewplace.com, tgl@sss.pgh.pa.us, pgsql@bluepolka.net, alvherre@commandprompt.com
  Subject: Re: query log corrupted-looking entries
  In-Reply-To: <20070608172137.GC9818@svana.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox