pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2
6+ messages / 3 participants
[nested] [flat]

* Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2
@ 2003-04-04 10:49 Robert John Shepherd <robert@reviewer.co.uk>
  2003-04-04 14:50 ` Re: [BUGS] Rapid deteriation of performance (might be caused by Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-04-04 14:53 ` Re: Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 2 replies; 6+ messages in thread

From: Robert John Shepherd @ 2003-04-04 10:49 UTC (permalink / raw)
  To: pgsql-bugs@postgresql.org

Up until a few days ago I have been running Postgresl 7.2.3 with Tsearch
from the contrib dir, but at various times the performance of the
database would suddenly and rapidly deteriate so that queries which
previously took 500ms then took 8 or 9 seconds.

The only cure is a backup and restore of the database, vacuuming and
analysing does nothing. I even tried rebuilding all indexes once which
didn't seem to help.

This was an annoying but intermittent thing, which happened the last
time this Wednesday. Since I was doing a backup and restore anyway, I
decided to upgrade to 7.3.2 in the hope this might fix the annoying
problem, however it has made it WAY worse.

Rather than going a few weeks (and sometimes months) in between having
to use this fix, I am now having to do it almost every single day. I'm
now lucky if it lasts 24 hours before it brings my website to a total
crawl.

There is nothing special about my database other than the fact that I
use the Tsearch addon. Now if I go and do a bit update to the Tsearch
indexes on a table, with for example:

	UPDATE tblmessages SET strmessageidx=txt2txtidx(strheading || '
' || strmessage);

Then that instantly brings the whole database to a crawl, which no
amount of index rebuilding, vacuuming and analysing helps.

Help! (And sorry if this is the wrong list)



Yours Unwhettedly,
Robert John Shepherd.

Editor
DVD REVIEWER
The UK's BIGGEST Online DVD Magazine
http://www.dvd.reviewer.co.uk

For a copy of my Public PGP key, email: pgp@robertsworld.org.uk  




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [BUGS] Rapid deteriation of performance (might be caused by
  2003-04-04 10:49 Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
@ 2003-04-04 14:50 ` Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-04-04 15:19   ` Re: [BUGS] Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
  1 sibling, 1 reply; 6+ messages in thread

From: Stephan Szabo @ 2003-04-04 14:50 UTC (permalink / raw)
  To: Robert John Shepherd <robert@reviewer.co.uk>; +Cc: pgsql-performance


On Fri, 4 Apr 2003, Robert John Shepherd wrote:

> Up until a few days ago I have been running Postgresl 7.2.3 with Tsearch
> from the contrib dir, but at various times the performance of the
> database would suddenly and rapidly deteriate so that queries which
> previously took 500ms then took 8 or 9 seconds.

Hmm, what are the before and after explain analyze results?  Also, what
are your conf settings for shared buffers, sort memory and the fsm
parameters?

> The only cure is a backup and restore of the database, vacuuming and
> analysing does nothing. I even tried rebuilding all indexes once which
> didn't seem to help.

Did you do a regular vacuum or vacuum full?  If only the former, it's
possible that you need to either vacuum more frequently and/or raise the
free space map settings in your configuration file.

What does vacuum full verbose <table>; give you for the tables involved?

> Help! (And sorry if this is the wrong list)
pgsql-performance is a better list, so I've replied to there.  You'll
probably need to join in order to reply to list.




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [BUGS] Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2
  2003-04-04 10:49 Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
  2003-04-04 14:50 ` Re: [BUGS] Rapid deteriation of performance (might be caused by Stephan Szabo <sszabo@megazone23.bigpanda.com>
@ 2003-04-04 15:19   ` Robert John Shepherd <robert@reviewer.co.uk>
  2003-04-04 15:29     ` Re: [BUGS] Rapid deteriation of performance (might be Stephan Szabo <sszabo@megazone23.bigpanda.com>
  0 siblings, 1 reply; 6+ messages in thread

From: Robert John Shepherd @ 2003-04-04 15:19 UTC (permalink / raw)
  To: pgsql-performance

> > Up until a few days ago I have been running Postgresl 7.2.3 with
Tsearch
> > from the contrib dir, but at various times the performance of the
> > database would suddenly and rapidly deteriate so that queries which
> > previously took 500ms then took 8 or 9 seconds.

> Hmm, what are the before and after explain analyze results?  Also,
what
> are your conf settings for shared buffers, sort memory and the fsm
> parameters?

shared_buffers = 40960
sort_mem = 20480
#max_fsm_relations = 1000
#max_fsm_pages = 10000

As you can see I've not uncommented or touched the fsm parameters, I
have no idea what they do. Optimisation wise I have only played with
shared_buffers, sort_mem and max_connections.


> > The only cure is a backup and restore of the database

> Did you do a regular vacuum or vacuum full?  If only the former, it's
> possible that you need to either vacuum more frequently and/or raise
the
> free space map settings in your configuration file.

I've been running this daily:

	vacuumdb -h localhost -a -z

Should I be using the full switch then?

I'll get back to you on the other questions if you think they are still
needed.


> pgsql-performance is a better list, so I've replied to there.  You'll
> probably need to join in order to reply to list.

Thanks, especially for not shouting at me heh, this is stressful enough
as it is.


Yours Unwhettedly,
Robert John Shepherd.

Editor
DVD REVIEWER
The UK's BIGGEST Online DVD Magazine
http://www.dvd.reviewer.co.uk

For a copy of my Public PGP key, email: pgp@robertsworld.org.uk  




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [BUGS] Rapid deteriation of performance (might be
  2003-04-04 10:49 Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
  2003-04-04 14:50 ` Re: [BUGS] Rapid deteriation of performance (might be caused by Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-04-04 15:19   ` Re: [BUGS] Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
@ 2003-04-04 15:29     ` Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-04-07 14:25       ` Re: [BUGS] Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
  0 siblings, 1 reply; 6+ messages in thread

From: Stephan Szabo @ 2003-04-04 15:29 UTC (permalink / raw)
  To: Robert John Shepherd <robert@reviewer.co.uk>; +Cc: pgsql-performance


On Fri, 4 Apr 2003, Robert John Shepherd wrote:

> > > Up until a few days ago I have been running Postgresl 7.2.3 with
> Tsearch
> > > from the contrib dir, but at various times the performance of the
> > > database would suddenly and rapidly deteriate so that queries which
> > > previously took 500ms then took 8 or 9 seconds.
>
> > Hmm, what are the before and after explain analyze results?  Also,
> what
> > are your conf settings for shared buffers, sort memory and the fsm
> > parameters?
>
> shared_buffers = 40960
> sort_mem = 20480
> #max_fsm_relations = 1000
> #max_fsm_pages = 10000
>
> As you can see I've not uncommented or touched the fsm parameters, I
> have no idea what they do. Optimisation wise I have only played with
> shared_buffers, sort_mem and max_connections.
>
>
> > > The only cure is a backup and restore of the database
>
> > Did you do a regular vacuum or vacuum full?  If only the former, it's
> > possible that you need to either vacuum more frequently and/or raise
> the
> > free space map settings in your configuration file.
>
> I've been running this daily:
>
> 	vacuumdb -h localhost -a -z
>
> Should I be using the full switch then?

Well, you generally shouldn't need to if the fsm settings are high enough.
If you're doing really big updates like update each row of a 1 billion
row table, you may end up having to do one immediately following that.
Of course, if you're doing that, performance is probably not your biggest
concern. ;)

Explain analyze'll tell us if the system is changing plans (presumably to
a worse one) - for example, deciding to move to a sequence scan because it
thinks that the index scan is now to expensive, or conversely moving to an
index scan because it thinks that there'll be too many reads, while those
page reads actually are fairly localized. The vacuum full verbose should
get some idea of how much empty space is there.

> > pgsql-performance is a better list, so I've replied to there.  You'll
> > probably need to join in order to reply to list.
>
> Thanks, especially for not shouting at me heh, this is stressful enough
> as it is.

:)




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: [BUGS] Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2
  2003-04-04 10:49 Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
  2003-04-04 14:50 ` Re: [BUGS] Rapid deteriation of performance (might be caused by Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-04-04 15:19   ` Re: [BUGS] Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
  2003-04-04 15:29     ` Re: [BUGS] Rapid deteriation of performance (might be Stephan Szabo <sszabo@megazone23.bigpanda.com>
@ 2003-04-07 14:25       ` Robert John Shepherd <robert@reviewer.co.uk>
  0 siblings, 0 replies; 6+ messages in thread

From: Robert John Shepherd @ 2003-04-07 14:25 UTC (permalink / raw)
  To: 'Stephan Szabo' <sszabo@megazone23.bigpanda.com>; +Cc: pgsql-performance

> > I've been running this daily:
> > 	vacuumdb -h localhost -a -z
> > Should I be using the full switch then?
> 
> Well, you generally shouldn't need to if the fsm settings are high
enough.
> If you're doing really big updates like update each row of a 1 billion
> row table, you may end up having to do one immediately following that.
> Of course, if you're doing that, performance is probably not your
biggest
> concern. ;)

Not doing that, no. ;)


> Explain analyze'll tell us if the system is changing plans (presumably
to
> a worse one)

It wasn't, oddly enough.

I've added a new table that cuts down 85% of the work this query has to
do, and it seems to have helped an awful lot at the moment. Of course
only time will tell. :)

Thanks for the suggestions.


Yours Unwhettedly,
Robert John Shepherd.

Editor
DVD REVIEWER
The UK's BIGGEST Online DVD Magazine
http://www.dvd.reviewer.co.uk

For a copy of my Public PGP key, email: pgp@robertsworld.org.uk  




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2
  2003-04-04 10:49 Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
@ 2003-04-04 14:53 ` Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 0 replies; 6+ messages in thread

From: Tom Lane @ 2003-04-04 14:53 UTC (permalink / raw)
  To: Robert John Shepherd <robert@reviewer.co.uk>; +Cc: pgsql-bugs@postgresql.org

"Robert John Shepherd" <robert@reviewer.co.uk> writes:
> Help! (And sorry if this is the wrong list)

Yes, it's the wrong list.  pgsql-performance would be the place to
discuss this.  We can't help you anyway without more details: show us
the EXPLAIN ANALYZE results for some of the slow queries.  (Ideally
I'd like to see EXPLAIN ANALYZE for the same queries in both fast
and slow states ...)

			regards, tom lane




^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2003-04-07 14:25 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-04-04 10:49 Rapid deteriation of performance (might be caused by tsearch?) in 7.3.2 Robert John Shepherd <robert@reviewer.co.uk>
2003-04-04 14:50 ` Re: [BUGS] Rapid deteriation of performance (might be caused by Stephan Szabo <sszabo@megazone23.bigpanda.com>
2003-04-04 15:19   ` Robert John Shepherd <robert@reviewer.co.uk>
2003-04-04 15:29     ` Re: [BUGS] Rapid deteriation of performance (might be Stephan Szabo <sszabo@megazone23.bigpanda.com>
2003-04-07 14:25       ` Robert John Shepherd <robert@reviewer.co.uk>
2003-04-04 14:53 ` Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox