pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #2784: Performance serious degrades over a period of a month
5+ messages / 5 participants
[nested] [flat]
* BUG #2784: Performance serious degrades over a period of a month
@ 2006-11-26 16:35 Michael Simms <michael@tuxgames.com>
2006-11-28 02:59 ` Re: BUG #2784: Performance serious degrades over a period of a month Tom Lane <tgl@sss.pgh.pa.us>
2006-11-28 04:26 ` Re: BUG #2784: Performance serious degrades over a period of a month Bruno Wolff III <bruno@wolff.to>
2006-11-28 11:08 ` Re: BUG #2784: Performance serious degrades over a period of a month Heikki Linnakangas <heikki@enterprisedb.com>
0 siblings, 3 replies; 5+ messages in thread
From: Michael Simms @ 2006-11-26 16:35 UTC (permalink / raw)
To: pgsql-bugs@postgresql.org
The following bug has been logged online:
Bug reference: 2784
Logged by: Michael Simms
Email address: michael@tuxgames.com
PostgreSQL version: 8.1.4
Operating system: Linux kernel 2.6.12
Description: Performance serious degrades over a period of a month
Details:
OK, we have a database that runs perfectly well after a dump and restore,
but over a period of a month or two, it just degrades to the point of
uselessness.
vacuumdb -a is run every 24 hours. We have also run for months at a time
using -a -z but the effect doesnt change.
The database is for a counter, not the most critical part of the system, but
a part of the system nonetheless. Other tables we have also degrade over
time, but the counter is the most pronounced. There seems to be no common
feature of the tables that degrade. All I know is that a series of queries
that are run on the database every 24 hours, after a dump/restore takes 2
hours. Now, 2 months after, it is taking over 12. We are seriously
considering switching to mysql to avoid this issue.
But I wanted to let you guys have a chance to resolve the issue, we dont
have the manpower or expertise to fix it ourselves. I am willing to let
someone from the postgres development team have access to our server for a
period of time to have a look at the issue. This would need to be someone
extremely trustworthy as the database contains confidential client
information.
I am willing to wait 2 days for a response and for someone to take a look at
the problem. The performance degridation isnt something we can leave as it
is for long, and in 2 days time I will have to dump and restore the
database, which will reset it to a good state, and will mean I will have to
resort to the mysql switch instead.
Sorry this sounds a bit rushed, but it cant be helped, this is causing
*problems* and we need a solution, either a fix or a switch to another
database. Id rather a fix cos I like postgres, but Im willing to bite the
mysql bullet if I have to...
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #2784: Performance serious degrades over a period of a month
2006-11-26 16:35 BUG #2784: Performance serious degrades over a period of a month Michael Simms <michael@tuxgames.com>
@ 2006-11-28 02:59 ` Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 0 replies; 5+ messages in thread
From: Tom Lane @ 2006-11-28 02:59 UTC (permalink / raw)
To: Michael Simms <michael@tuxgames.com>; +Cc: pgsql-bugs@postgresql.org
"Michael Simms" <michael@tuxgames.com> writes:
> OK, we have a database that runs perfectly well after a dump and restore,
> but over a period of a month or two, it just degrades to the point of
> uselessness.
> vacuumdb -a is run every 24 hours. We have also run for months at a time
> using -a -z but the effect doesnt change.
You probably need significantly-more-frequent vacuuming. Have you
considered autovacuum?
This is not a bug --- you'd get better help on the pgsql-performance
mailing list.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #2784: Performance serious degrades over a period of a month
2006-11-26 16:35 BUG #2784: Performance serious degrades over a period of a month Michael Simms <michael@tuxgames.com>
@ 2006-11-28 04:26 ` Bruno Wolff III <bruno@wolff.to>
2006-11-28 11:34 ` Re: BUG #2784: Performance serious degrades over a period Bill Moran <wmoran@collaborativefusion.com>
2 siblings, 1 reply; 5+ messages in thread
From: Bruno Wolff III @ 2006-11-28 04:26 UTC (permalink / raw)
To: Michael Simms <michael@tuxgames.com>; +Cc: pgsql-bugs@postgresql.org; pgsql-performance
This really should have been asked on pgsql-performance and would probably
get a better response there..
On Sun, Nov 26, 2006 at 16:35:52 +0000,
Michael Simms <michael@tuxgames.com> wrote:
> PostgreSQL version: 8.1.4
> Operating system: Linux kernel 2.6.12
> Description: Performance serious degrades over a period of a month
> Details:
>
> OK, we have a database that runs perfectly well after a dump and restore,
> but over a period of a month or two, it just degrades to the point of
> uselessness.
> vacuumdb -a is run every 24 hours. We have also run for months at a time
> using -a -z but the effect doesnt change.
>
This sounds like you either need to increase your FSM setting or vacuum
more often. I think vacuumdb -v will give you enough information to tell
if FSM is too low at the frequency you are vacuuming.
> The database is for a counter, not the most critical part of the system, but
> a part of the system nonetheless. Other tables we have also degrade over
> time, but the counter is the most pronounced. There seems to be no common
> feature of the tables that degrade. All I know is that a series of queries
> that are run on the database every 24 hours, after a dump/restore takes 2
> hours. Now, 2 months after, it is taking over 12. We are seriously
> considering switching to mysql to avoid this issue.
You probably will want to vacuum the counter table more often than the other
tables in the database. Depending on how often the counter(s) are being
updated and how many separate counters are in the table you might want to
vacuum that table as often as once a minute.
Depending on your requirements you might also want to consider using a sequence
instead of a table row for the counter.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #2784: Performance serious degrades over a period
2006-11-26 16:35 BUG #2784: Performance serious degrades over a period of a month Michael Simms <michael@tuxgames.com>
2006-11-28 04:26 ` Re: BUG #2784: Performance serious degrades over a period of a month Bruno Wolff III <bruno@wolff.to>
@ 2006-11-28 11:34 ` Bill Moran <wmoran@collaborativefusion.com>
0 siblings, 0 replies; 5+ messages in thread
From: Bill Moran @ 2006-11-28 11:34 UTC (permalink / raw)
To: Bruno Wolff III <bruno@wolff.to>; +Cc: Michael Simms <michael@tuxgames.com>; pgsql-performance
Bruno Wolff III <bruno@wolff.to> wrote:
>
> This really should have been asked on pgsql-performance and would probably
> get a better response there..
>
> On Sun, Nov 26, 2006 at 16:35:52 +0000,
> Michael Simms <michael@tuxgames.com> wrote:
> > PostgreSQL version: 8.1.4
> > Operating system: Linux kernel 2.6.12
> > Description: Performance serious degrades over a period of a month
> > Details:
> >
> > OK, we have a database that runs perfectly well after a dump and restore,
> > but over a period of a month or two, it just degrades to the point of
> > uselessness.
> > vacuumdb -a is run every 24 hours. We have also run for months at a time
> > using -a -z but the effect doesnt change.
> >
>
> This sounds like you either need to increase your FSM setting or vacuum
> more often. I think vacuumdb -v will give you enough information to tell
> if FSM is too low at the frequency you are vacuuming.
>
> > The database is for a counter, not the most critical part of the system, but
> > a part of the system nonetheless. Other tables we have also degrade over
> > time, but the counter is the most pronounced. There seems to be no common
> > feature of the tables that degrade. All I know is that a series of queries
> > that are run on the database every 24 hours, after a dump/restore takes 2
> > hours. Now, 2 months after, it is taking over 12. We are seriously
> > considering switching to mysql to avoid this issue.
>
> You probably will want to vacuum the counter table more often than the other
> tables in the database. Depending on how often the counter(s) are being
> updated and how many separate counters are in the table you might want to
> vacuum that table as often as once a minute.
>
> Depending on your requirements you might also want to consider using a sequence
> instead of a table row for the counter.
Just to throw it in to the mix: you might also be in a usage pattern that would
benefit from a scheduled reindex every so often.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #2784: Performance serious degrades over a period of a month
2006-11-26 16:35 BUG #2784: Performance serious degrades over a period of a month Michael Simms <michael@tuxgames.com>
@ 2006-11-28 11:08 ` Heikki Linnakangas <heikki@enterprisedb.com>
2 siblings, 0 replies; 5+ messages in thread
From: Heikki Linnakangas @ 2006-11-28 11:08 UTC (permalink / raw)
To: Michael Simms <michael@tuxgames.com>; +Cc: pgsql-bugs@postgresql.org
Michael Simms wrote:
> The following bug has been logged online:
>
> Bug reference: 2784
> Logged by: Michael Simms
> Email address: michael@tuxgames.com
> PostgreSQL version: 8.1.4
> Operating system: Linux kernel 2.6.12
> Description: Performance serious degrades over a period of a month
> Details:
>
> OK, we have a database that runs perfectly well after a dump and restore,
> but over a period of a month or two, it just degrades to the point of
> uselessness.
> vacuumdb -a is run every 24 hours. We have also run for months at a time
> using -a -z but the effect doesnt change.
You might have a hung transaction that never finishes open, which
prevents vacuum from removing old tuple versions. Or you might have too
low FSM settings as others suggested.
I'd try running VACUUM VERBOSE by hand, and taking a good look at the
output. If there's nothing obviously wrong with it, please send the
output back to the list (or pgsql-performance, as Tom suggested), and
maybe we can help.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2006-11-28 11:34 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-11-26 16:35 BUG #2784: Performance serious degrades over a period of a month Michael Simms <michael@tuxgames.com>
2006-11-28 02:59 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-11-28 04:26 ` Bruno Wolff III <bruno@wolff.to>
2006-11-28 11:34 ` Re: BUG #2784: Performance serious degrades over a period Bill Moran <wmoran@collaborativefusion.com>
2006-11-28 11:08 ` Heikki Linnakangas <heikki@enterprisedb.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox