From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Alison Winters <alison@mirrabooka.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: lots of updates on small table
Date: Thu, 14 Jul 2005 20:28:24 -0400
Message-ID: <20050715002824.GA23728@alvh.no-ip.org> (raw)
In-Reply-To: <42D6F854.nail3K81VIICH@pluto.mirrabooka.com>
References: <42D5F34E.nail9QE1195IE@pluto.mirrabooka.com>
<20050714162114.GA19778@alvh.no-ip.org>
<42D6F854.nail3K81VIICH@pluto.mirrabooka.com>
On Fri, Jul 15, 2005 at 09:42:12AM +1000, Alison Winters wrote:
> > > Our application requires a number of processes to select and update rows
> > > from a very small (<10 rows) Postgres table on a regular and frequent
> > > basis. These processes often run for weeks at a time, but over the
> > > space of a few days we find that updates start getting painfully slow.
> > > We are running a full vacuum/analyze and reindex on the table every day,
> > Full vacuum, eh? I wonder if what you really need is very frequent
> > non-full vacuum. Say, once in 15 minutes (exact rate depending on dead
> > tuple rate.)
> >
> Is there a difference between vacuum and vacuum full?
Yes. Vacuum full is more aggresive in compacting the table. Though it
really works the same in the presence of long-running transactions:
tuples just can't be removed.
> The most recent output was this:
>
> INFO: --Relation public.plc_fldio--
> INFO: Pages 1221: Changed 3, reaped 256, Empty 0, New 0; Tup 108137: Vac 4176, Keep/VTL 108133/108133, UnUsed 19, MinLen 84, MaxLen 84; Re-using: Free/Avail. Space 445176/371836; EndEmpty/Avail. Pages 0/256.
> CPU 0.04s/0.14u sec elapsed 0.18 sec.
> INFO: Index plcpage_idx: Pages 315; Tuples 108137: Deleted 4176.
> CPU 0.03s/0.04u sec elapsed 0.14 sec.
> INFO: Rel plc_fldio: Pages: 1221 --> 1221; Tuple(s) moved: 0.
> CPU 0.03s/0.04u sec elapsed 0.36 sec.
> INFO: Analyzing public.plc_fldio
Hmm, so it seems your hourly vacuum is enough. I think the bloat theory
can be trashed. Unless I'm reading this output wrong; I don't remember
the details of this vacuum output.
> We'll up it to every 15 minutes, but i don't know if that'll help
> because even with the current vacuuming the updates are still getting
> slower and slower over the course of several days. What really puzzles
> me is why restarting the processes fixes it.
I wonder if the problem may be plan caching. I didn't pay full
attention to the description of your problem, so I don't remember if it
could be an issue, but it's something to consider.
> Does PostgreSQL keep some kind of backlog of transactions all for one
> database connection?
No. There could be a problem if you had very long transactions, but
apparently this isn't your problem.
> Isn't it normal to have processes that keep a single database
> connection open for days at a time?
I guess it depends on exactly what you do with it. I know of at least
one case where an app keeps a connection open for months, without a
problem. (It's been running for four or five years, and monthly
"uptime" for that particular daemon is not unheard of.)
--
Alvaro Herrera (<alvherre[a]alvh.no-ip.org>)
"Everybody understands Mickey Mouse. Few understand Hermann Hesse.
Hardly anybody understands Einstein. And nobody understands Emperor Norton."
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: alvherre@alvh.no-ip.org, alison@mirrabooka.com
Subject: Re: lots of updates on small table
In-Reply-To: <20050715002824.GA23728@alvh.no-ip.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox