pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedAutoanalyze settings with zero scale factor
6+ messages / 3 participants
[nested] [flat]
* Autoanalyze settings with zero scale factor
@ 2007-01-18 18:37 Jeremy Haile <jhaile@fastmail.fm>
0 siblings, 1 reply; 6+ messages in thread
From: Jeremy Haile @ 2007-01-18 18:37 UTC (permalink / raw)
To: pgsql-performance
Some of my very large tables (10 million rows) need to be analyzed by
autovacuum on a frequent basis. Rather than specifying this as a
percentage of table size + base threshold, I wanted to specify it as an
explicit number of rows.
I changed the table-specific settings so that the ANALYZE base threshold
was 5000 and the ANALYZE scale factor is 0. According to the documented
formula: analyze threshold = analyze base threshold + analyze scale
factor * number of tuples, I assumed that this would cause the table to
be analyzed everytime 5000 tuples were inserted/updated/deleted.
However, the tables have been updated with tens of thousands of inserts
and the table has still not been analyzed (according to
pg_stat_user_tables). Does a scale factor of 0 cause the table to never
be analyzed? What am I doing wrong?
I'm using PG 8.2.1.
Thanks,
Jeremy Haile
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autoanalyze settings with zero scale factor
@ 2007-01-18 19:20 Matthew T. O'Connor <matthew@zeut.net>
parent: Jeremy Haile <jhaile@fastmail.fm>
0 siblings, 1 reply; 6+ messages in thread
From: Matthew T. O'Connor @ 2007-01-18 19:20 UTC (permalink / raw)
To: Jeremy Haile <jhaile@fastmail.fm>; +Cc: pgsql-performance
Jeremy Haile wrote:
> I changed the table-specific settings so that the ANALYZE base threshold
> was 5000 and the ANALYZE scale factor is 0. According to the documented
> formula: analyze threshold = analyze base threshold + analyze scale
> factor * number of tuples, I assumed that this would cause the table to
> be analyzed everytime 5000 tuples were inserted/updated/deleted.
That is right, and exactly how the scaling factor / base value are
supposed to work, so this should be fine.
> However, the tables have been updated with tens of thousands of inserts
> and the table has still not been analyzed (according to
> pg_stat_user_tables). Does a scale factor of 0 cause the table to never
> be analyzed? What am I doing wrong? I'm using PG 8.2.1.
No a scaling factor of 0 shouldn't stop the table from being analyzed.
Unless it's just a bug, my only guess is that autovacuum may be getting
busy at times (vacuuming large tables for example) and hasn't had a
chance to even look at that table for a while, and by the time it gets
to it, there have been tens of thousands of inserts. Does that sounds
plausible?
Also, are other auto-vacuums and auto-analyzes showing up in the
pg_stats table? Maybe it's a stats system issue.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autoanalyze settings with zero scale factor
@ 2007-01-18 20:21 Jeremy Haile <jhaile@fastmail.fm>
parent: Matthew T. O'Connor <matthew@zeut.net>
0 siblings, 2 replies; 6+ messages in thread
From: Jeremy Haile @ 2007-01-18 20:21 UTC (permalink / raw)
To: Matthew T. O'Connor <matthew@zeut.net>; +Cc: pgsql-performance
> Unless it's just a bug, my only guess is that autovacuum may be getting
> busy at times (vacuuming large tables for example) and hasn't had a
> chance to even look at that table for a while, and by the time it gets
> to it, there have been tens of thousands of inserts. Does that sounds
> plausible?
Possible, but I think your next suggestion is more likely.
> Also, are other auto-vacuums and auto-analyzes showing up in the
> pg_stats table? Maybe it's a stats system issue.
No tables have been vacuumed or analyzed today. I had thought that this
problem was due to my pg_autovacuum changes, but perhaps not. I
restarted PostgreSQL (in production - yikes) About a minute after being
restarted, the autovac process fired up.
What could get PG in a state where autovac isn't running? Is there
anything I should watch to debug or monitor for this problem in the
future? I wish I'd noticed whether or not the stats collector process
was running before I restarted.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autoanalyze settings with zero scale factor
@ 2007-01-18 20:40 Matthew T. O'Connor <matthew@zeut.net>
parent: Jeremy Haile <jhaile@fastmail.fm>
1 sibling, 0 replies; 6+ messages in thread
From: Matthew T. O'Connor @ 2007-01-18 20:40 UTC (permalink / raw)
To: Jeremy Haile <jhaile@fastmail.fm>; +Cc: pgsql-performance
Jeremy Haile wrote:
>> Also, are other auto-vacuums and auto-analyzes showing up in the
>> pg_stats table? Maybe it's a stats system issue.
>>
>
> No tables have been vacuumed or analyzed today. I had thought that this
> problem was due to my pg_autovacuum changes, but perhaps not. I
> restarted PostgreSQL (in production - yikes) About a minute after being
> restarted, the autovac process fired up.
>
> What could get PG in a state where autovac isn't running? Is there
> anything I should watch to debug or monitor for this problem in the
> future? I wish I'd noticed whether or not the stats collector process
> was running before I restarted.
First off you shouldn't need to restart PG. When it wasn't working did
you ever check the autovacuum_enabled setting? For example within psql:
"show autovacuum;".
I would venture to guess that autovacuum was disabled for some reason.
Perhaps last time you started the server the stats settings weren't enabled?
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autoanalyze settings with zero scale factor
@ 2007-01-18 21:30 Tom Lane <tgl@sss.pgh.pa.us>
parent: Jeremy Haile <jhaile@fastmail.fm>
1 sibling, 1 reply; 6+ messages in thread
From: Tom Lane @ 2007-01-18 21:30 UTC (permalink / raw)
To: Jeremy Haile <jhaile@fastmail.fm>; +Cc: Matthew T. O'Connor <matthew@zeut.net>; pgsql-performance
"Jeremy Haile" <jhaile@fastmail.fm> writes:
> No tables have been vacuumed or analyzed today. I had thought that this
> problem was due to my pg_autovacuum changes, but perhaps not. I
> restarted PostgreSQL (in production - yikes) About a minute after being
> restarted, the autovac process fired up.
> What could get PG in a state where autovac isn't running?
Um, are you sure it wasn't? The autovac process is not an always-there
thing, it quits after each pass and then the postmaster starts a new one
awhile later.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autoanalyze settings with zero scale factor
@ 2007-01-18 21:53 Jeremy Haile <jhaile@fastmail.fm>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 6+ messages in thread
From: Jeremy Haile @ 2007-01-18 21:53 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Matthew T. O'Connor <matthew@zeut.net>; pgsql-performance
Well - it hadn't run on any table in over 24 hours (according to
pg_stat_user_tables). My tables are constantly being inserted into and
deleted from, and the autovacuum settings are pretty aggressive. I also
had not seen the autovac process running in the past 24 hours. (although
I wasn't watching it *all* the time)
So - as far as I could tell it wasn't running.
On Thu, 18 Jan 2007 16:30:17 -0500, "Tom Lane" <tgl@sss.pgh.pa.us> said:
> "Jeremy Haile" <jhaile@fastmail.fm> writes:
> > No tables have been vacuumed or analyzed today. I had thought that this
> > problem was due to my pg_autovacuum changes, but perhaps not. I
> > restarted PostgreSQL (in production - yikes) About a minute after being
> > restarted, the autovac process fired up.
>
> > What could get PG in a state where autovac isn't running?
>
> Um, are you sure it wasn't? The autovac process is not an always-there
> thing, it quits after each pass and then the postmaster starts a new one
> awhile later.
>
> regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2007-01-18 21:53 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-01-18 18:37 Autoanalyze settings with zero scale factor Jeremy Haile <jhaile@fastmail.fm>
2007-01-18 19:20 ` Matthew T. O'Connor <matthew@zeut.net>
2007-01-18 20:21 ` Jeremy Haile <jhaile@fastmail.fm>
2007-01-18 20:40 ` Matthew T. O'Connor <matthew@zeut.net>
2007-01-18 21:30 ` Tom Lane <tgl@sss.pgh.pa.us>
2007-01-18 21:53 ` Jeremy Haile <jhaile@fastmail.fm>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox