pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Mark Harrison <mh@pixar.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Greg Spiegelberg <gspiegelberg@cranel.com>
Cc: PgSQL Performance ML <pgsql-performance@postgresql.org>
Cc: Postgres Admin List <pgsql-admin@postgresql.org>
Subject: Re: syslog slowing the database?
Date: Wed, 10 Mar 2004 11:08:50 -0800
Message-ID: <404F67C2.70003@pixar.com> (raw)
References: <404E2937.4060604@cranel.com>
	<370.1078877805@sss.pgh.pa.us>
	<404F3996.2070105@cranel.com>
	<7688.1078937631@sss.pgh.pa.us>

Tom Lane wrote:
> Greg Spiegelberg <gspiegelberg@cranel.com> writes:
> 
>>If the log and database were on the same disk I'd be okay with the
>>current workaround.  If the ``-'' gave me near the same performance as
>>turning syslog off I'd be okay with that too.  However, neither of these
>>are the case so there has to be something else blocking between the two
>>processes.
> 
> 
> You could also consider not using syslog at all: let the postmaster
> output to its stderr, and pipe that into a log-rotation program.
> I believe some people use Apache's log rotator for this with good
> results.

I do this... here's the relevant lines from my startup script:

ROTATE="/inst/apache/bin/rotatelogs $PGLOGS/postgresql 86400"
$PGBIN/pg_ctl start -s -D $PGDATA | $ROTATE &

Following is a patch to rotatelogs that does two things:

- makes a symbolic link 'foo.current' that points to the
   current output file.

- gzips the rotated logfile

If you have gnu tools installed, you can
     tail --retry --follow=name foo.current
and it will automatically track the most recent
log file.

HTH,
Mark

-- 
Mark Harrison
Pixar Animation Studios


inline patch 1: rotatelogs.c (+16/-0 lines) download inline-1.patch*** rotatelogs.c-orig 2004-03-10 10:24:02.000000000 -0800 --- rotatelogs.c 2004-03-10 11:01:55.000000000 -0800 *************** *** 25,30 **** --- 25,32 ---- int main (int argc, char **argv) { char buf[BUFSIZE], buf2[MAX_PATH], errbuf[ERRMSGSZ]; + char linkbuf[MAX_PATH]; + char oldbuf2[MAX_PATH]; time_t tLogEnd = 0, tRotation; int nLogFD = -1, nLogFDprev = -1, nMessCount = 0, nRead, nWrite; int utc_offset = 0; *************** *** 75,80 **** --- 77,84 ---- setmode(0, O_BINARY); #endif + sprintf(linkbuf, "%s.current", szLogRoot); + sprintf(oldbuf2, ""); use_strftime = (strstr(szLogRoot, "%") != NULL); for (;;) { nRead = read(0, buf, sizeof buf); *************** *** 99,104 **** --- 103,111 ---- sprintf(buf2, "%s.%010d", szLogRoot, (int) tLogStart); } tLogEnd = tLogStart + tRotation; + printf("oldbuf2=%s\n",oldbuf2); + printf("buf2=%s\n",buf2); + printf("linkbuf=%s\n",linkbuf); nLogFD = open(buf2, O_WRONLY | O_CREAT | O_APPEND, 0666); if (nLogFD < 0) { /* Uh-oh. Failed to open the new log file. Try to clear *************** *** 125,130 **** --- 132,146 ---- } else { close(nLogFDprev); + /* use: tail --follow=name foo.current */ + unlink(linkbuf); + symlink(buf2,linkbuf); + if (strlen(oldbuf2) > 0) { + char cmd[MAX_PATH+100]; + sprintf(cmd, "gzip %s &", oldbuf2); + system(cmd); + } + strcpy(oldbuf2, buf2); } nMessCount = 0; }

view thread (19+ messages)  latest in thread

Message-ID: <404F67C2.70003@pixar.com>
Permalink:  ../404F67C2.70003@pixar.com/
Also on:    postgresql.org/message-id/404F67C2.70003@pixar.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: mh@pixar.com, tgl@sss.pgh.pa.us, gspiegelberg@cranel.com, pgsql-admin@postgresql.org
  Subject: Re: syslog slowing the database?
  In-Reply-To: <404F67C2.70003@pixar.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox