pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedsyslog slowing the database?
19+ messages / 15 participants
[nested] [flat]
* syslog slowing the database?
@ 2004-03-09 20:29 Greg Spiegelberg <gspiegelberg@cranel.com>
0 siblings, 3 replies; 19+ messages in thread
From: Greg Spiegelberg @ 2004-03-09 20:29 UTC (permalink / raw)
To: pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
I've been waiting all day for a pg_restore to finish on a test system
identically configured as our production in hardware and software
with the exception prod is 7.3.5 and test is 7.4.1.
The file it's restoring from is about 8GB uncompressed from a
"pg_dump -b -F t" and after 2 hours the directory the database is in
contains only 1GB. iostat reported ~2000 blocks written every 2
seconds to the DB file system.
I turned syslog off to see if it was blocking anything and in the
past couple minutes 1GB has been restored and iostat reports ~35,000
blocks written every 2 seconds to the DB file system.
The system is completely idle except for this restore process. Could
syslog the culprit?
I turned syslog back on and the restore slowed down again. Turned
it off and it sped right back up.
Can anyone confirm this for me?
Greg
--
Greg Spiegelberg
Sr. Product Development Engineer
Cranel, Incorporated.
Phone: 614.318.4314
Fax: 614.431.8388
Email: gspiegelberg@Cranel.com
Cranel. Technology. Integrity. Focus.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-09 21:34 Richard Huxton <dev@archonet.com>
parent: Greg Spiegelberg <gspiegelberg@cranel.com>
2 siblings, 0 replies; 19+ messages in thread
From: Richard Huxton @ 2004-03-09 21:34 UTC (permalink / raw)
To: Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
On Tuesday 09 March 2004 20:29, Greg Spiegelberg wrote:
> iostat reported ~2000 blocks written every 2
> seconds to the DB file system.
>
> I turned syslog off to see if it was blocking anything and in the
> past couple minutes 1GB has been restored and iostat reports ~35,000
> blocks written every 2 seconds to the DB file system.
> Can anyone confirm this for me?
If syslog is set to sync after every line and you're logging too much then it
could slow things down as the disk heads shift back and fore between two
areas of disk. How many disks do you have and in what configuration?
Also - was PG logging a lot of info, or is some other application the culprit?
Tip: put a minus "-" in front of the file-path in your syslog.conf and it
won't sync to disk after every entry.
--
Richard Huxton
Archonet Ltd
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: syslog slowing the database?
@ 2004-03-10 00:16 Tom Lane <tgl@sss.pgh.pa.us>
parent: Greg Spiegelberg <gspiegelberg@cranel.com>
2 siblings, 3 replies; 19+ messages in thread
From: Tom Lane @ 2004-03-10 00:16 UTC (permalink / raw)
To: Greg Spiegelberg <gspiegelberg@cranel.com>; +Cc: pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Greg Spiegelberg <gspiegelberg@cranel.com> writes:
> I turned syslog back on and the restore slowed down again. Turned
> it off and it sped right back up.
We have heard reports before of syslog being quite slow. What platform
are you on exactly? Does Richard's suggestion of turning off syslog's
fsync help?
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-10 01:06 Magnus Naeslund(t) <mag@fbab.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 0 replies; 19+ messages in thread
From: Magnus Naeslund(t) @ 2004-03-10 01:06 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Tom Lane wrote:
> Greg Spiegelberg <gspiegelberg@cranel.com> writes:
>
>>I turned syslog back on and the restore slowed down again. Turned
>>it off and it sped right back up.
>
>
> We have heard reports before of syslog being quite slow. What platform
> are you on exactly? Does Richard's suggestion of turning off syslog's
> fsync help?
>
Another tip is to use a better (well atleast more optimized) syslog
implementation, like metalog. It optimizes log writes to a blocksize
that is better for disk throughput.
You can also use "per line" mode with those if you want, i think.
I use another logger that is called multilog (see at http://cr.yp.to),
that's a pipe logger thing, like one per postmaster.
It also gives very exact timestamps to every line, has built in log
rotation and works nice with all programs i use it for.
One thing is for sure, if you log much, standard syslog (atleast on
linux) sucks big time.
I gained back approx 30% CPU on a mailserver over here by changing to
another logger.
Cheers
Magnus
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-10 01:09 Gavin M. Roy <gmr@ehpg.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 19+ messages in thread
From: Gavin M. Roy @ 2004-03-10 01:09 UTC (permalink / raw)
To: ; +Cc: pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Might want to look at metalog, it does delayed writes, though ultimately
your issue is io bound and there's not much you can do to reduce io if
you want to keep syslog logging your pgsql queries and such.
Tom Lane wrote:
>Greg Spiegelberg <gspiegelberg@cranel.com> writes:
>
>
>>I turned syslog back on and the restore slowed down again. Turned
>>it off and it sped right back up.
>>
>>
>
>We have heard reports before of syslog being quite slow. What platform
>are you on exactly? Does Richard's suggestion of turning off syslog's
>fsync help?
>
> regards, tom lane
>
>---------------------------(end of broadcast)---------------------------
>TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-10 01:32 Stephen Robert Norris <srn@commsecure.com.au>
parent: Gavin M. Roy <gmr@ehpg.net>
0 siblings, 0 replies; 19+ messages in thread
From: Stephen Robert Norris @ 2004-03-10 01:32 UTC (permalink / raw)
To: Gavin M. Roy <gmr@ehpg.net>; +Cc: pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
On Wed, 2004-03-10 at 12:09, Gavin M. Roy wrote:
> Might want to look at metalog, it does delayed writes, though ultimately
> your issue is io bound and there's not much you can do to reduce io if
> you want to keep syslog logging your pgsql queries and such.
Yeah, but syslog with fsync() after each line is much, much worse than
syslog without it, assuming anything else is on the same disk (array).
It just guarantees to screw up your drive head movements...
--
Stephen Norris srn@fn.com.au
Farrow Norris Pty Ltd +61 417 243 239
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: syslog slowing the database?
@ 2004-03-10 15:51 Greg Spiegelberg <gspiegelberg@cranel.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 19+ messages in thread
From: Greg Spiegelberg @ 2004-03-10 15:51 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Tom Lane wrote:
> Greg Spiegelberg <gspiegelberg@cranel.com> writes:
>
>>I turned syslog back on and the restore slowed down again. Turned
>>it off and it sped right back up.
>
> We have heard reports before of syslog being quite slow. What platform
> are you on exactly? Does Richard's suggestion of turning off syslog's
> fsync help?
RedHat 7.3 w/ 2.4.24 kernel on a dual Intel PIII 1.3Ghz, 2GB memory,
U160 internal on integrated controller, 1Gbps SAN for database.
Database file being restored and the actual database are on different
disk and controllers than syslog files.
With the ``-'' in front of the syslog file postgres logs too gives
me roughly 75% of the I/O the performance as reported by iostat. So,
it helps though turning syslog off gives the optimum performance.
If the log and database were on the same disk I'd be okay with the
current workaround. If the ``-'' gave me near the same performance as
turning syslog off I'd be okay with that too. However, neither of these
are the case so there has to be something else blocking between the two
processes.
<2 hours and multiple test later>
I've found that hardware interrupts are the culprit. Given my system
config both SCSI and fibre controllers were throttling the system with
the interrupts required to write the data (syslog & database) and read
the data from the restore. I'm okay with that.
In the order of worst to best.
* There were, on average about 450 interrupts/sec with the default
config of syslog on one disk, database on the SAN and syslog using
fsync.
* Turning fsync off in syslog puts interrupts around 105/sec and.
* Having syslog fsync turned off in syslog AND moving the syslog file
to a filesystem serviced by the same fibre controller put interrupts
at around 92/sec. I decided to do this after watching the I/O on
the SAN with syslog turned off and found that it had bandwidth to
spare. FYI, the system when idle generated about 50 interrupts/sec.
I'm going with the later for now on the test system and after running
it through it's paces with all our processes I'll make the change in
production. I'll post if I run into anything else.
Greg
BTW, I like what metalog has to offer but I prefer using as many of the
default tools as possible and replacing them only when absolutely
necessary. What I've learned with syslog here is that it is still
viable but likely requires a minor tweak. If this tweak fails in
testing I'll look at metalog then.
--
Greg Spiegelberg
Sr. Product Development Engineer
Cranel, Incorporated.
Phone: 614.318.4314
Fax: 614.431.8388
Email: gspiegelberg@Cranel.com
Cranel. Technology. Integrity. Focus.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: syslog slowing the database?
@ 2004-03-10 16:53 Tom Lane <tgl@sss.pgh.pa.us>
parent: Greg Spiegelberg <gspiegelberg@cranel.com>
0 siblings, 2 replies; 19+ messages in thread
From: Tom Lane @ 2004-03-10 16:53 UTC (permalink / raw)
To: Greg Spiegelberg <gspiegelberg@cranel.com>; +Cc: pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Greg Spiegelberg <gspiegelberg@cranel.com> writes:
> If the log and database were on the same disk I'd be okay with the
> current workaround. If the ``-'' gave me near the same performance as
> turning syslog off I'd be okay with that too. However, neither of these
> are the case so there has to be something else blocking between the two
> processes.
You could also consider not using syslog at all: let the postmaster
output to its stderr, and pipe that into a log-rotation program.
I believe some people use Apache's log rotator for this with good
results.
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: syslog slowing the database?
@ 2004-03-10 19:08 Mark Harrison <mh@pixar.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 19+ messages in thread
From: Mark Harrison @ 2004-03-10 19:08 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Tom Lane wrote:
> Greg Spiegelberg <gspiegelberg@cranel.com> writes:
>
>>If the log and database were on the same disk I'd be okay with the
>>current workaround. If the ``-'' gave me near the same performance as
>>turning syslog off I'd be okay with that too. However, neither of these
>>are the case so there has to be something else blocking between the two
>>processes.
>
>
> You could also consider not using syslog at all: let the postmaster
> output to its stderr, and pipe that into a log-rotation program.
> I believe some people use Apache's log rotator for this with good
> results.
I do this... here's the relevant lines from my startup script:
ROTATE="/inst/apache/bin/rotatelogs $PGLOGS/postgresql 86400"
$PGBIN/pg_ctl start -s -D $PGDATA | $ROTATE &
Following is a patch to rotatelogs that does two things:
- makes a symbolic link 'foo.current' that points to the
current output file.
- gzips the rotated logfile
If you have gnu tools installed, you can
tail --retry --follow=name foo.current
and it will automatically track the most recent
log file.
HTH,
Mark
--
Mark Harrison
Pixar Animation Studios
inline patch 1: rotatelogs.c (+16/-0 lines) download inline-1.patch
*** rotatelogs.c-orig 2004-03-10 10:24:02.000000000 -0800
--- rotatelogs.c 2004-03-10 11:01:55.000000000 -0800
***************
*** 25,30 ****
--- 25,32 ----
int main (int argc, char **argv)
{
char buf[BUFSIZE], buf2[MAX_PATH], errbuf[ERRMSGSZ];
+ char linkbuf[MAX_PATH];
+ char oldbuf2[MAX_PATH];
time_t tLogEnd = 0, tRotation;
int nLogFD = -1, nLogFDprev = -1, nMessCount = 0, nRead, nWrite;
int utc_offset = 0;
***************
*** 75,80 ****
--- 77,84 ----
setmode(0, O_BINARY);
#endif
+ sprintf(linkbuf, "%s.current", szLogRoot);
+ sprintf(oldbuf2, "");
use_strftime = (strstr(szLogRoot, "%") != NULL);
for (;;) {
nRead = read(0, buf, sizeof buf);
***************
*** 99,104 ****
--- 103,111 ----
sprintf(buf2, "%s.%010d", szLogRoot, (int) tLogStart);
}
tLogEnd = tLogStart + tRotation;
+ printf("oldbuf2=%s\n",oldbuf2);
+ printf("buf2=%s\n",buf2);
+ printf("linkbuf=%s\n",linkbuf);
nLogFD = open(buf2, O_WRONLY | O_CREAT | O_APPEND, 0666);
if (nLogFD < 0) {
/* Uh-oh. Failed to open the new log file. Try to clear
***************
*** 125,130 ****
--- 132,146 ----
}
else {
close(nLogFDprev);
+ /* use: tail --follow=name foo.current */
+ unlink(linkbuf);
+ symlink(buf2,linkbuf);
+ if (strlen(oldbuf2) > 0) {
+ char cmd[MAX_PATH+100];
+ sprintf(cmd, "gzip %s &", oldbuf2);
+ system(cmd);
+ }
+ strcpy(oldbuf2, buf2);
}
nMessCount = 0;
}
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: syslog slowing the database?
@ 2004-03-10 21:00 Vivek Khera <khera@kcilink.com>
parent: Greg Spiegelberg <gspiegelberg@cranel.com>
2 siblings, 0 replies; 19+ messages in thread
From: Vivek Khera @ 2004-03-10 21:00 UTC (permalink / raw)
To: pgsql-performance
>>>>> "GS" == Greg Spiegelberg <gspiegelberg@cranel.com> writes:
GS> I've been waiting all day for a pg_restore to finish on a test system
GS> identically configured as our production in hardware and software
GS> with the exception prod is 7.3.5 and test is 7.4.1.
GS> The file it's restoring from is about 8GB uncompressed from a
GS> "pg_dump -b -F t" and after 2 hours the directory the database is in
GS> contains only 1GB. iostat reported ~2000 blocks written every 2
GS> seconds to the DB file system.
Have you considered increasing the value of checkpoint_segments to
something like 50 or 100 during your restore? It made a *dramatic*
improvement on my system when I did the same migration.
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: khera@kciLink.com Rockville, MD +1-301-869-4449 x806
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-11 01:34 Christopher Kings-Lynne <chriskl@familyhealth.com.au>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 5 replies; 19+ messages in thread
From: Christopher Kings-Lynne @ 2004-03-11 01:34 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
> You could also consider not using syslog at all: let the postmaster
> output to its stderr, and pipe that into a log-rotation program.
> I believe some people use Apache's log rotator for this with good
> results.
Not an option I'm afraid. PostgreSQL just jams and stops logging after
the first rotation...
I've read in the docs that syslog logging is the only "production"
solution...
Chris
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-11 01:37 Chris Smith <chris@interspire.com>
parent: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
4 siblings, 0 replies; 19+ messages in thread
From: Chris Smith @ 2004-03-11 01:37 UTC (permalink / raw)
To: 'Christopher Kings-Lynne' <chriskl@familyhealth.com.au>; +Cc: pgsql-performance; 'Postgres Admin List' <pgsql-admin@postgresql.org>
It might depend on how you're rotating it.
Try the copy/truncate method instead of moving the log file. If you move
the log file to another filename you usually have to restart the app
doing the logging before it starts logging again.
Chris.
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Christopher
Kings-Lynne
Sent: Thursday, March 11, 2004 12:35 PM
To: Tom Lane
Cc: Greg Spiegelberg; PgSQL Performance ML; Postgres Admin List
Subject: Re: [PERFORM] [ADMIN] syslog slowing the database?
> You could also consider not using syslog at all: let the postmaster
> output to its stderr, and pipe that into a log-rotation program. I
> believe some people use Apache's log rotator for this with good
> results.
Not an option I'm afraid. PostgreSQL just jams and stops logging after
the first rotation...
I've read in the docs that syslog logging is the only "production"
solution...
Chris
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-11 02:03 Bruno Wolff III <bruno@wolff.to>
parent: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
4 siblings, 0 replies; 19+ messages in thread
From: Bruno Wolff III @ 2004-03-11 02:03 UTC (permalink / raw)
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
On Thu, Mar 11, 2004 at 09:34:54 +0800,
Christopher Kings-Lynne <chriskl@familyhealth.com.au> wrote:
> >You could also consider not using syslog at all: let the postmaster
> >output to its stderr, and pipe that into a log-rotation program.
> >I believe some people use Apache's log rotator for this with good
> >results.
>
> Not an option I'm afraid. PostgreSQL just jams and stops logging after
> the first rotation...
>
> I've read in the docs that syslog logging is the only "production"
> solution...
I use multilog to log postgres' output and it works fine.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-11 04:09 Tom Lane <tgl@sss.pgh.pa.us>
parent: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
4 siblings, 1 reply; 19+ messages in thread
From: Tom Lane @ 2004-03-11 04:09 UTC (permalink / raw)
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>; +Cc: Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
>> You could also consider not using syslog at all: let the postmaster
>> output to its stderr, and pipe that into a log-rotation program.
>> I believe some people use Apache's log rotator for this with good
>> results.
> Not an option I'm afraid. PostgreSQL just jams and stops logging after
> the first rotation...
I know some people use this in production. Dunno what went wrong in
your test, but it can be made to work.
regards, tom lane
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-11 14:55 Joshua D. Drake <jd@commandprompt.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 19+ messages in thread
From: Joshua D. Drake @ 2004-03-11 14:55 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>; Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
>>Not an option I'm afraid. PostgreSQL just jams and stops logging after
>>the first rotation...
Are you using a copy truncate method to rotate the logs? In RedHat add
the keyword COPYTRUCATE to your /etc/logrotate.d/syslog file.
Sincerely,
Joshua D. Drake
>
>
> I know some people use this in production. Dunno what went wrong in
> your test, but it can be made to work.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
Attachments:
jd.vcf (text/x-vcard, 638B) download
begin:vcard
fn:Joshua D. Drake
n:Drake;Joshua D.
org:Command Prompt, Inc.
adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA
email;internet:jd@commandprompt.com
title:Consultant
tel;work:503-667-4564
tel;fax:503-210-0034
note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl.
x-mozilla-html:FALSE
url:http://www.commandprompt.com/
version:2.1
end:vcard
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [ADMIN] syslog slowing the database?
@ 2004-03-14 16:00 Andrew Sullivan <ajs@crankycanuck.ca>
parent: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
4 siblings, 0 replies; 19+ messages in thread
From: Andrew Sullivan @ 2004-03-14 16:00 UTC (permalink / raw)
To: pgsql-performance
On Thu, Mar 11, 2004 at 09:34:54AM +0800, Christopher Kings-Lynne wrote:
> >You could also consider not using syslog at all: let the postmaster
> >output to its stderr, and pipe that into a log-rotation program.
>
> Not an option I'm afraid. PostgreSQL just jams and stops logging after
> the first rotation...
Actually, this is what we do. Last year we offered an (admittedly
expensive) bespoke log rotator written in Perl for just this purpose.
It was rejected on the grounds that it didn't do anything Apache's
rotator didn't do, so I didn't pursue it. I'm willing to put it up
on gborg, though, if anyone thinks it'll be worth having around.
FWIW, we use ours in production.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The plural of anecdote is not data.
--Roger Brinner
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-15 15:17 scott.marlowe <scott.marlowe@ihs.com>
parent: Christopher Kings-Lynne <chriskl@familyhealth.com.au>
4 siblings, 1 reply; 19+ messages in thread
From: scott.marlowe @ 2004-03-15 15:17 UTC (permalink / raw)
To: Christopher Kings-Lynne <chriskl@familyhealth.com.au>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
On Thu, 11 Mar 2004, Christopher Kings-Lynne wrote:
> > You could also consider not using syslog at all: let the postmaster
> > output to its stderr, and pipe that into a log-rotation program.
> > I believe some people use Apache's log rotator for this with good
> > results.
>
> Not an option I'm afraid. PostgreSQL just jams and stops logging after
> the first rotation...
>
> I've read in the docs that syslog logging is the only "production"
> solution...
Can you use the apache log rotator? It's known to work in my environment
(redhat 7.2, postgresql 7.2 and 7.4) with this command to start it in my
rc.local file:
su - postgres -c 'pg_ctl start | rotatelogs $PGDATA/pglog 86400 2>1&'
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-15 15:38 Bruce Momjian <pgman@candle.pha.pa.us>
parent: scott.marlowe <scott.marlowe@ihs.com>
0 siblings, 1 reply; 19+ messages in thread
From: Bruce Momjian @ 2004-03-15 15:38 UTC (permalink / raw)
To: scott.marlowe <scott.marlowe@ihs.com>; +Cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>; Tom Lane <tgl@sss.pgh.pa.us>; Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
scott.marlowe wrote:
> On Thu, 11 Mar 2004, Christopher Kings-Lynne wrote:
>
> > > You could also consider not using syslog at all: let the postmaster
> > > output to its stderr, and pipe that into a log-rotation program.
> > > I believe some people use Apache's log rotator for this with good
> > > results.
> >
> > Not an option I'm afraid. PostgreSQL just jams and stops logging after
> > the first rotation...
> >
> > I've read in the docs that syslog logging is the only "production"
> > solution...
>
> Can you use the apache log rotator? It's known to work in my environment
> (redhat 7.2, postgresql 7.2 and 7.4) with this command to start it in my
> rc.local file:
>
> su - postgres -c 'pg_ctl start | rotatelogs $PGDATA/pglog 86400 2>1&'
Sure, our documentation specifically mentions using rotatelogs.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [PERFORM] syslog slowing the database?
@ 2004-03-15 15:57 scott.marlowe <scott.marlowe@ihs.com>
parent: Bruce Momjian <pgman@candle.pha.pa.us>
0 siblings, 0 replies; 19+ messages in thread
From: scott.marlowe @ 2004-03-15 15:57 UTC (permalink / raw)
To: Bruce Momjian <pgman@candle.pha.pa.us>; +Cc: Christopher Kings-Lynne <chriskl@familyhealth.com.au>; Tom Lane <tgl@sss.pgh.pa.us>; Greg Spiegelberg <gspiegelberg@cranel.com>; pgsql-performance; Postgres Admin List <pgsql-admin@postgresql.org>
On Mon, 15 Mar 2004, Bruce Momjian wrote:
> scott.marlowe wrote:
> > On Thu, 11 Mar 2004, Christopher Kings-Lynne wrote:
> >
> > > > You could also consider not using syslog at all: let the postmaster
> > > > output to its stderr, and pipe that into a log-rotation program.
> > > > I believe some people use Apache's log rotator for this with good
> > > > results.
> > >
> > > Not an option I'm afraid. PostgreSQL just jams and stops logging after
> > > the first rotation...
> > >
> > > I've read in the docs that syslog logging is the only "production"
> > > solution...
> >
> > Can you use the apache log rotator? It's known to work in my environment
> > (redhat 7.2, postgresql 7.2 and 7.4) with this command to start it in my
> > rc.local file:
> >
> > su - postgres -c 'pg_ctl start | rotatelogs $PGDATA/pglog 86400 2>1&'
>
> Sure, our documentation specifically mentions using rotatelogs.
hehe. What I meant was can Christopher use it, or does he have a
limitation in his environment where he can't get ahold of the apache log
rotater... :-)
^ permalink raw reply [nested|flat] 19+ messages in thread
end of thread, other threads:[~2004-03-15 15:57 UTC | newest]
Thread overview: 19+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-03-09 20:29 syslog slowing the database? Greg Spiegelberg <gspiegelberg@cranel.com>
2004-03-09 21:34 ` Richard Huxton <dev@archonet.com>
2004-03-10 00:16 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-03-10 01:06 ` Magnus Naeslund(t) <mag@fbab.net>
2004-03-10 01:09 ` Gavin M. Roy <gmr@ehpg.net>
2004-03-10 01:32 ` Stephen Robert Norris <srn@commsecure.com.au>
2004-03-10 15:51 ` Greg Spiegelberg <gspiegelberg@cranel.com>
2004-03-10 16:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-03-10 19:08 ` Mark Harrison <mh@pixar.com>
2004-03-11 01:34 ` Christopher Kings-Lynne <chriskl@familyhealth.com.au>
2004-03-11 01:37 ` Chris Smith <chris@interspire.com>
2004-03-11 02:03 ` Bruno Wolff III <bruno@wolff.to>
2004-03-11 04:09 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-03-11 14:55 ` Joshua D. Drake <jd@commandprompt.com>
2004-03-14 16:00 ` Andrew Sullivan <ajs@crankycanuck.ca>
2004-03-15 15:17 ` scott.marlowe <scott.marlowe@ihs.com>
2004-03-15 15:38 ` Bruce Momjian <pgman@candle.pha.pa.us>
2004-03-15 15:57 ` scott.marlowe <scott.marlowe@ihs.com>
2004-03-10 21:00 ` Vivek Khera <khera@kcilink.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox