Cc: pgsql-performance@postgresql.org
Message-Id: <7A2BA520-ED19-418F-9339-95C7F7158664@socket.ch>
From: Thomas Spreng <spreng@socket.ch>
To: Christopher Browne <cbbrowne@acm.org>
In-Reply-To: <87mynp3h3q.fsf@wolfe.cbbrowne.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v919.2)
Subject: Re: Oddly slow queries
Date: Tue, 22 Apr 2008 15:42:25 +0200
References: <3D7728ED-3D36-47E6-9266-351073ED4E11@socket.ch>
	<op.t9odybcgcigqcu@apollo13.peufeu.com>
	<AE4F81FF-C8BE-44BC-A79D-48EC5BF20162@socket.ch>
	<60y77du7qs.fsf@dba2.int.libertyrms.com>
	<20B852C4-2ACA-445F-9289-558B6C86BAB4@socket.ch>
	<87mynp3h3q.fsf@wolfe.cbbrowne.com>

On 19.04.2008, at 19:11, Christopher Browne wrote:
> Martha Stewart called it a Good Thing when spreng@socket.ch (Thomas  
> Spreng) wrote:
>> On 16.04.2008, at 17:42, Chris Browne wrote:
>> What I meant is if there are no INSERT's or UPDATE's going on it
>> shouldn't affect SELECT queries, or am I wrong?
>
> Yes, that's right.  (Caveat: VACUUM would be a form of update, in this
> context...)

thanks for pointing that out, at the moment we don't run autovacuum but
VACUUM ANALYZE VERBOSE twice a day.

>>> 2.  On the other hand, if you're on 8.1 or so, you may be able to
>>> configure the Background Writer to incrementally flush checkpoint  
>>> data
>>> earlier, and avoid the condition of 1.
>>>
>>> Mind you, you'd have to set BgWr to be pretty aggressive, based on  
>>> the
>>> "10s periodicity" that you describe; that may not be a nice
>>> configuration to have all the time :-(.
>>
>> I've just seen that the daily vacuum tasks didn't run,
>> apparently. The DB has almost doubled it's size since some days
>> ago. I guess I'll have to VACUUM FULL (dump/restore might be faster,
>> though) and check if that helps anything.
>
> If you're locking out users, then it's probably a better idea to use
> CLUSTER to reorganize the tables, as that simultaneously eliminates
> empty space on tables *and indices.*
>
> In contrast, after running VACUUM FULL, you may discover you need to
> reindex tables, because the reorganization of the *table* leads to
> bloating of the indexes.

I don't VACUUM FULL but thanks for the hint.

> Pre-8.3 (I *think*), there's a transactional issue with CLUSTER where
> it doesn't fully follow MVCC, so that "dead, but still accessible, to
> certain transactions" tuples go away.  That can cause surprises
> (e.g. - queries missing data) if applications are accessing the
> database concurrently with the CLUSTER.  It's safe as long as the DBA
> can take over the database and block out applications.  And at some
> point, the MVCC bug got fixed.

I think I'll upgrade PostgreSQL to the latest 8.3 version in the next
few days anyway, along with a memory upgrade (from 1.5GB to 4GB) and a
new 2x RAID-1 (instead of RAID-5) disk configuration. I hope that this
has already a noticeable impact on the performance.

> Note that you should check the output of a VACUUM VERBOSE run, and/or
> use the contrib function pgsstattuples() to check how sparse the
> storage usage is.  There may only be a few tables that are behaving
> badly, and cleaning up a few tables will be a lot less intrusive than
> cleaning up the whole database.

That surely is the case because about 90% of all data is stored in one
big table and most of the rows are deleted and newly INSERT'ed every
night.

cheers,

tom