From: Kevin Grittner <Kevin.Grittner@wicourts.gov>
To: Brian Fehrle <brianf@consistentstate.com>
To: pgsql-admin@postgresql.org
Subject: Re: WAL files containing old and unmodified data?
Date: Wed, 22 Feb 2012 14:53:23 -0600
Message-ID: <4F4501630200002500045A52@gw.wicourts.gov> (raw)
In-Reply-To: <4F45517F.4050508@consistentstate.com>
References: <4F45517F.4050508@consistentstate.com>
Brian Fehrle <brianf@consistentstate.com> wrote:
> I have a very active database that generates between 200 and 400
> WAL files that are archived for backups. Yesterday, the average
> number of WAL files being generated skyrocketed.
>
> We took a look at some WAL files using the strings command, and
> notice quite a bit of data from a specific table with timestamps
> of last year. This table does some basic tracking, where a row is
> inserted, then it's updated, and after that it's never modified
> again. So we're having data appear in our WAL files from rows that
> haven't (or should not have been) modified at all in over a year.
>
> We figured that the rows were in fact being updated, so we turned
> log_min_duration_statement to 0 for a period of time and monitored
> all queries going through, however we never found a single query
> that could possibly be updating these pieces of data.
> Can server processes such as autovacuum processes cause this
> behavior?
Yes, to prevent transaction ID wrap-around it must FREEZE each tuple
at some point, which is a WAL-logged operation. If many rows were
loaded at about the same time, and they were not subsequently
updated or deleted, you can see quite a burst of activity when
things hit this point. If you're using rsync for your base backups,
expect a lot more data to need to be written, too, because tuples
which have been stable for the last year have now changed.
-Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: Kevin.Grittner@wicourts.gov, brianf@consistentstate.com
Subject: Re: WAL files containing old and unmodified data?
In-Reply-To: <4F4501630200002500045A52@gw.wicourts.gov>
References: <4F45517F.4050508@consistentstate.com> <4F4501630200002500045A52@gw.wicourts.gov>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox