pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Kevin Grittner <Kevin.Grittner@wicourts.gov>
To: Brian Fehrle <brianf@consistentstate.com>
To: pgsql-admin@postgresql.org
Subject: Re: WAL files containing old and unmodified data?
Date: Wed, 22 Feb 2012 14:53:23 -0600
Message-ID: <4F4501630200002500045A52@gw.wicourts.gov> (raw)
In-Reply-To: <4F45517F.4050508@consistentstate.com>
References: <4F45517F.4050508@consistentstate.com>

Brian Fehrle <brianf@consistentstate.com> wrote:
 
> I have a very active database that generates between 200 and 400
> WAL files that are archived for backups. Yesterday, the average
> number of WAL files being generated skyrocketed.
> 
> We took a look at some WAL files using the strings command, and
> notice quite a bit of data from a specific table with timestamps
> of last year.  This table does some basic tracking, where a row is
> inserted, then it's updated, and after that it's never modified
> again. So we're having data appear in our WAL files from rows that
> haven't (or should not have been) modified at all in over a year.
> 
> We figured that the rows were in fact being updated, so we turned 
> log_min_duration_statement to 0 for a period of time and monitored
> all queries going through, however we never found a single query
> that could possibly be updating these pieces of data.
 
> Can server processes such as autovacuum processes cause this
> behavior?
 
Yes, to prevent transaction ID wrap-around it must FREEZE each tuple
at some point, which is a WAL-logged operation.  If many rows were
loaded at about the same time, and they were not subsequently
updated or deleted, you can see quite a burst of activity when
things hit this point.  If you're using rsync for your base backups,
expect a lot more data to need to be written, too, because tuples
which have been stable for the last year have now changed.
 
-Kevin



view thread (3+ messages)  latest in thread

Message-ID: <4F4501630200002500045A52@gw.wicourts.gov>
Permalink:  ../4F4501630200002500045A52@gw.wicourts.gov/
Also on:    postgresql.org/message-id/4F4501630200002500045A52@gw.wicourts.gov

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: Kevin.Grittner@wicourts.gov, brianf@consistentstate.com
  Subject: Re: WAL files containing old and unmodified data?
  In-Reply-To: <4F4501630200002500045A52@gw.wicourts.gov>
  References: <4F45517F.4050508@consistentstate.com> <4F4501630200002500045A52@gw.wicourts.gov>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox