From: Frank Joerdens <frank@joerdens.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org, Nic Ferrier <nic@woome.com>
Cc: Avleen Vig <avleen@woome.com>
Cc: Mike Rogers <mike.rogers@woome.com>
Subject: Re: Full statement logging problematic on larger machines?
Date: Wed, 11 Mar 2009 22:42:29 +0000
Message-ID: <7d10d2df0903111542o53abb3dbvfc405a597e4c9fa8@mail.gmail.com> (raw)
In-Reply-To: <3401.1236804370@sss.pgh.pa.us>
References: <7d10d2df0903111227g2e40e742g5992cba5158dd4d2@mail.gmail.com>
<3401.1236804370@sss.pgh.pa.us>
On Wed, Mar 11, 2009 at 8:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Frank Joerdens <frank@joerdens.de> writes:
>> Greetings. We're having trouble with full logging since we moved from
>> an 8-core server with 16 GB memory to a machine with double that
>> spec and I am wondering if this *should* be working or if there is a
>> point on larger machines where logging and scheduling seeks of
>> background writes - or something along those lines; it might be a
>> theory - doesn't work together any more?
>
> You didn't tell us anything interesting about *how* you are logging,
> so it's hard to comment on this. Are you just writing to stderr?
> syslog? using PG's built-in log capture process? It would be good
> to show all of your log-related postgresql.conf settings.
Here's the complete postgresql.conf (I've whittled it down as much as
I could so it's quite compact):
frank@db04:~$ cat /etc/postgresql/8.2/main/postgresql.conf
data_directory = '/var/lib/postgresql/8.2/main'
hba_file = '/etc/postgresql/8.2/main/pg_hba.conf'
ident_file = '/etc/postgresql/8.2/main/pg_ident.conf'
listen_addresses = 'localhost,172.16.222.62'
port = 5432
max_connections = 1000
shared_buffers = 16GB
work_mem = 200MB
maintenance_work_mem = 1GB
max_fsm_pages = 50000
wal_buffers = 8MB
checkpoint_segments = 16
autovacuum = on
stats_start_collector = on
stats_row_level = on
effective_cache_size = 4GB
default_statistics_target = 10
constraint_exclusion = off
checkpoint_warning = 1h
escape_string_warning = off
log_duration = off
log_min_duration_statement = 1000
log_statement = 'ddl'
log_line_prefix = '%m %p %h %u '
archive_command = '/usr/bin/walmgr.py
/var/lib/postgresql/walshipping/master.ini xarchive %p %f'
redirect_stderr = on
log_directory = '/dev/shm/'
log_rotation_age = 0
log_rotation_size = 0
The above is what we're doing right now, only logging queries that run
for over a second, and that is no problem; so the answer to Scott's
question in his reply to my posting is: Yes, logging only the slower
queries does work.
Yesterday I changed log_duration = on and log_statement = 'all' at
off-peak time and left it on for 4 hours while traffic was picking up.
Eventually I had to stop it because the server got bogged down.
Regards,
Frank
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: frank@joerdens.de, tgl@sss.pgh.pa.us, nic@woome.com, avleen@woome.com, mike.rogers@woome.com
Subject: Re: Full statement logging problematic on larger machines?
In-Reply-To: <7d10d2df0903111542o53abb3dbvfc405a597e4c9fa8@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox