From: "achill@matrix" <achill@matrix.gatewaynet.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: syslog performance when logging big statements
Date: Wed, 9 Jul 2008 15:31:05 +0300
User-Agent: KMail/1.9.7
Cc: pgsql-performance@postgresql.org
References: <200807081524.47366.achill@matrix.gatewaynet.com>
	<200807081821.57935.achill@matrix.gatewaynet.com>
	<12596.1215542041@sss.pgh.pa.us>
In-Reply-To: <12596.1215542041@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Disposition: inline
Message-Id: <200807091531.05372.achill@matrix.gatewaynet.com>
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: quoted-printable

=CE=A3=CF=84=CE=B9=CF=82 Tuesday 08 July 2008 21:34:01 =CE=BF/=CE=B7 Tom La=
ne =CE=AD=CE=B3=CF=81=CE=B1=CF=88=CE=B5:
> Achilleas Mantzios <achill@matrix.gatewaynet.com> writes:
> > =CE=A3=CF=84=CE=B9=CF=82 Tuesday 08 July 2008 17:35:16 =CE=BF/=CE=B7 To=
m Lane =CE=AD=CE=B3=CF=81=CE=B1=CF=88=CE=B5:
> >> Hmm.  There's a function in elog.c that breaks log messages into chunks
> >> for syslog.  I don't think anyone's ever looked hard at its performance
> >> --- maybe there's an O(N^2) behavior?
> >
> > Thanx,
> > i changed PG_SYSLOG_LIMIT in elog.c:1269 from 128 to 1048576
> > and i got super fast stderr performance. :)
>
> Doesn't seem like a very good solution given its impact on the stack
> depth right there.
>
> Looking at the code, the only bit that looks like repeated work are the
> repeated calls to strchr(), which would not be an issue in the "typical"
> case where the very long message contains reasonably frequent newlines.
> Am I right in guessing that your problematic statement contained
> megabytes worth of text with nary a newline?

Yes it was the input source of a bytea field containing tiff or pdf data,
of size of some 1-3 megabytes, and it did not contain (many) newlines.

>
> If so, we can certainly fix it by arranging to remember the last
> strchr() result across loop iterations, but I'd like to confirm the
> theory before doing that work.
>
> 			regards, tom lane