From: Steven Flatt <steven.flatt@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org
Subject: Re: Insertion to temp table deteriorating over time
Date: Fri, 15 Dec 2006 12:55:46 -0500
Message-ID: <357fa7590612150955g1b9c733duaf000c23d28aa8af@mail.gmail.com> (raw)
In-Reply-To: <11651.1166202586@sss.pgh.pa.us>
References: <357fa7590612130844q58ef3ca5q31e164f9b006e0fc@mail.gmail.com>
<9315.1166030958@sss.pgh.pa.us>
<357fa7590612131517q375fc0by50cb4257bcce7a94@mail.gmail.com>
<13032.1166052447@sss.pgh.pa.us>
<357fa7590612141240t55e70695q770a05a68a67536c@mail.gmail.com>
<25209.1166131396@sss.pgh.pa.us>
<357fa7590612150821i3eca3f66ge2ced1ee20fa2dac@mail.gmail.com>
<11651.1166202586@sss.pgh.pa.us>
Our application is such that there is a great deal of activity at the
beginning of the hour and minimal activity near the end of the hour. Those
3 vacuums were done at (approximately) 30 minutes past, 40 minutes past, and
50 minutes past the hour, during low activity. Vacuums of pg_class look
like they're being done on the hour. So it's not surprising that the first
vacuum found a lot of dead rows while the latter two found very few.
In fact, I just did another vacuum (about 30 minutes past the hour again)
and got:
INFO: "pg_class": found 5490 removable, 3263 nonremovable row versions in
171 pages
DETAIL: 0 dead row versions cannot be removed yet.
... and clearly a vacuum was done under an hour ago.
The truncate and re-fill process is done once per hour, at the end of the
high-load cycle, so I doubt that's even a big contributor to the number of
removable rows in pg_class.
For this particular setup, we expect high load for 10-15 minutes at the
beginning of the hour, which is the case when a new connection is
initialized. After a day or so (as is happening right now), the high-load
period spills into the second half of the hour. Within 3-4 days, we start
spilling into the next hour and, as you can imagine, everything gets behind
and we spiral down from there. For now, our workaround is to manually kill
the connection every few days, but I would like a better solution than
setting up a cron job to do this!
Thanks again,
Steve
On 12/15/06, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Hm, look at the numbers of rows removed:
>
> > INFO: "pg_class": found 5680 removable, 3263 nonremovable row versions
> in
> > 625 pages
> > DETAIL: 0 dead row versions cannot be removed yet.
>
> > INFO: "pg_class": found 24 removable, 3263 nonremovable row versions in
> 625
> > pages
> > DETAIL: 0 dead row versions cannot be removed yet.
>
> > INFO: "pg_class": found 0 removable, 3263 nonremovable row versions in
> 625
> > pages
> > DETAIL: 0 dead row versions cannot be removed yet.
>
> The lack of unremovable dead rows is good, but why were there so many
> dead rows the first time? You didn't say what the cycle time is on your
> truncate-and-refill process, but the last two suggest that the average
> rate of accumulation of dead pg_class rows is only a couple per minute,
> in which case it's been a lot longer than an hour since the previous
> VACUUM of pg_class. I'm back to suspecting that you don't vacuum
> pg_class regularly. You mentioned having an hourly cron job to fire off
> vacuums ... are you sure it's run as a database superuser?
>
> regards, tom lane
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: steven.flatt@gmail.com, tgl@sss.pgh.pa.us
Subject: Re: Insertion to temp table deteriorating over time
In-Reply-To: <357fa7590612150955g1b9c733duaf000c23d28aa8af@mail.gmail.com>
References: <357fa7590612130844q58ef3ca5q31e164f9b006e0fc@mail.gmail.com> <9315.1166030958@sss.pgh.pa.us> <357fa7590612131517q375fc0by50cb4257bcce7a94@mail.gmail.com> <13032.1166052447@sss.pgh.pa.us> <357fa7590612141240t55e70695q770a05a68a67536c@mail.gmail.com> <25209.1166131396@sss.pgh.pa.us> <357fa7590612150821i3eca3f66ge2ced1ee20fa2dac@mail.gmail.com> <11651.1166202586@sss.pgh.pa.us> <357fa7590612150955g1b9c733duaf000c23d28aa8af@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox