pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: dx k9 <bitsandbytes88@hotmail.com>
To: posgres support <pgsql-admin@postgresql.org>
Subject: Zen and art if WAL and dump file mainteance
Date: Mon, 28 Jan 2008 15:37:58 -0500
Message-ID: <BAY116-W326F5FEA10C498D47792CDD1340@phx.gbl> (raw)


Hi,

We are trying to confirm our understanding of WAL.  Thank you in advance for any comments and help.  We have version 8.24.

We noticed that when our nightly maintenance of the WAL folders kicked off    *** - see below.  Production processing was "KILLED".  We were in the process of rekeying  MANY existing tables.
We were doing heavy processing(which we normally don't do at the time of maintenance -  adding fields, table creation and filling tables going on.  During this time, archiving was enabled.

## Our data is set up like this
A /sprj/sqldumps_Linux.pgsql_serverX is our storage for the WAL
B /sqldata_Linux.pgsql_serverX is another computer/folder where the actual data is (pgdata).

We are trying to figure out why maintenance on A (zipping up & removing files) would have an affect on production processing (killed it) on pgdata - B.

We are thinking we should have turned off the nightly maintenance at this point, but we are wondering why and trying to improve our process logic.  We were thinking that our zipping and deleting
activity on A would not have an affect on B.  And it never has before, although we never had heavy processing at that time before.
Since, the archives don't happen too often, until the 36 logfile segments locally gets filled up, then one gets pushed out.  However, our clean up process is running from 11:30 - 12:32 in this case.
In our case, the first script wasn't finished zipping causing the second remove script to fail.

***
#This is what happened on A - nightly maintenance of dumps folder ***  Maybe there is a better way to do this and we are missing something here
30 19,23 * * * find /maintenance/NFSmntdd.dump001/sqldumps_Linux.pgsql_* -type f -mmin +10 ! -name \*.gz -size +0c -exec gzip {} \; 
30 0 * * * find /maintenance/NFSmntdd.dump001/sqldumps_Linux.pgsql_* -type f -mmin +20 -exec rm {} \


# This is what was happening on B
#---------------------------------------------------------------------------
# WRITE AHEAD LOG
#---------------------------------------------------------------------------

# - Settings -

#fsync = on                             # turns forced synchronization on or off
#wal_sync_method = fsync                # the default is the first option
                                        # supported by the operating system:
                                        #   open_datasync
                                        #   fdatasync
                                        #   fsync
                                        #   fsync_writethrough
                                        #   open_sync
#full_page_writes = on                  # recover from partial page writes
#wal_buffers = 64kB                     # min 32kB
wal_buffers = 128kB                     # min 32kB
                                        # (change requires restart)
#commit_delay = 0                       # range 0-100000, in microseconds
#commit_siblings = 5                    # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3                # in logfile segments, min 1, 16MB each
checkpoint_segments = 36                # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min              # range 30s-1h
#checkpoint_warning = 30s               # 0 is off

# - Archiving -

#archive_command = ''           # command to use to archive a logfile segment
archive_command = 'cp "%p" /sprj/sqldumps_Linux.pgsql_serverx/"%f"'         # command to use to archive a logfile segment
archive_timeout = 0             # force a logfile segment switch after this
                                # many seconds; 0 is off

_________________________________________________________________
Shed those extra pounds with MSN and The Biggest Loser!
http://biggestloser.msn.com/=

view thread (6+ messages)  latest in thread

Message-ID: <BAY116-W326F5FEA10C498D47792CDD1340@phx.gbl>
Permalink:  ../BAY116-W326F5FEA10C498D47792CDD1340@phx.gbl/
Also on:    postgresql.org/message-id/BAY116-W326F5FEA10C498D47792CDD1340@phx.gbl

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-admin@postgresql.org
  Cc: bitsandbytes88@hotmail.com
  Subject: Re: Zen and art if WAL and dump file mainteance
  In-Reply-To: <BAY116-W326F5FEA10C498D47792CDD1340@phx.gbl>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox