Received: from localhost (maia-1.hub.org [200.46.204.191])
	by postgresql.org (Postfix) with ESMTP id 27AEC9FB642
	for <pgsql-general-postgresql.org@postgresql.org>;
	Fri,  8 Jun 2007 14:22:01 -0300 (ADT)
Received: from postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.204.191]) (amavisd-maia, port 10024)
	with ESMTP id 23625-01 for
	<pgsql-general-postgresql.org@postgresql.org>;
	Fri,  8 Jun 2007 14:21:56 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.5
Received: from svana.org (svana.org [203.190.233.187])
	by postgresql.org (Postfix) with ESMTP id 7F25D9FB5EF
	for <pgsql-general@postgresql.org>;
	Fri,  8 Jun 2007 14:21:56 -0300 (ADT)
Received: from kleptog by svana.org with local (Exim 4.50)
	id 1Hwi9Z-00048a-Bf; Sat, 09 Jun 2007 03:21:37 +1000
Date: Fri, 8 Jun 2007 19:21:37 +0200
From: Martijn van Oosterhout <kleptog@svana.org>
To: George Pavlov <gpavlov@mynewplace.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-general@postgresql.org,
	"Ed L." <pgsql@bluepolka.net>, Alvaro Herrera <alvherre@commandprompt.com>
Subject: Re: query log corrupted-looking entries
Message-ID: <20070608172137.GC9818@svana.org>
Reply-To: Martijn van Oosterhout <kleptog@svana.org>
Mail-Followup-To: George Pavlov <gpavlov@mynewplace.com>,
	Tom Lane <tgl@sss.pgh.pa.us>, pgsql-general@postgresql.org,
	"Ed L." <pgsql@bluepolka.net>,
	Alvaro Herrera <alvherre@commandprompt.com>
References: 
 <8C5B026B51B6854CBE88121DBF097A86C39FC2@ehost010-33.exch010.intermedia.net>
	<200705291119.55567.pgsql@bluepolka.net>
	<8C5B026B51B6854CBE88121DBF097A86CDD9A4@ehost010-33.exch010.intermedia.net>
	<24195.1180733773@sss.pgh.pa.us>
	<8C5B026B51B6854CBE88121DBF097A86CDD9D2@ehost010-33.exch010.intermedia.net>
	<24885.1180736713@sss.pgh.pa.us>
	<8C5B026B51B6854CBE88121DBF097A86CDDA32@ehost010-33.exch010.intermedia.net>
	<8C5B026B51B6854CBE88121DBF097A86DEA716@ehost010-33.exch010.intermedia.net>
	<11458.1181265461@sss.pgh.pa.us>
	<8C5B026B51B6854CBE88121DBF097A86DEA7AE@ehost010-33.exch010.intermedia.net>
Mime-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="1ccMZA6j1vT5UqiK"
Content-Disposition: inline
In-Reply-To: 
 <8C5B026B51B6854CBE88121DBF097A86DEA7AE@ehost010-33.exch010.intermedia.net>
X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6
X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522  48A7 6394 F08A 0DC6 7BE6
X-PGP-Key-URL: <http://svana.org/kleptog/0DC67BE6.pgp.asc>
User-Agent: Mutt/1.5.9i
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: kleptog@svana.org
X-SA-Exim-Scanned: No (on svana.org); SAEximRunCond expanded to false
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200706/599
X-Sequence-Number: 115422


--1ccMZA6j1vT5UqiK
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Jun 08, 2007 at 09:30:21AM -0700, George Pavlov wrote:
> As to the full length of the entries that get
> interrupted they do seem to be all on the long side--I can't say with
> total certainty, but the couple of dozen that I looked at were all >
> 4096 when all the interruptions are taken out. So I think I can say that
> I see corruptions happen within "long" entries, but they can happen
> anywhere within that long entry, and one can have multiple interruptions
> within one entry.

That sounds about right. There's a requirement to be atomic for writes
less than PIPE_BUF, but for larger there are no guarentees at all. I
imagine the kernel noted the write was large and proceeded to fill the
last block of the buffer upto a round 4k. That would basically mean
they could get chopped into peices of any size.

Incidently, on systems I've worked with logging went through syslog and
it seems to split long lines at 80 characters or so:

[528-1] SELECT fo...
[528-2] ...
etc...

I always assumed it was postgres doing the splitting but perhaps it was
syslog all along.

Have a nice day,
--=20
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to l=
itigate.

--1ccMZA6j1vT5UqiK
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFGaZAhIB7bNG8LQkwRAn2GAJ0VvLmavBFSUapVZI/FC4Kc+JKTegCdHhpV
x77whwpSOl6QdsU0Mw9MNnA=
=chVn
-----END PGP SIGNATURE-----

--1ccMZA6j1vT5UqiK--