pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Marinos J. Yannikos <mjy@geizhals.at>
To: josh@agliodbs.com
Cc: pgsql-performance@postgresql.org
Subject: Re: optimization ideas for frequent, large(ish) updates
Date: Sun, 15 Feb 2004 03:02:48 +0100
Message-ID: <402ED348.7000800@geizhals.at> (raw)
In-Reply-To: <200402122228.41680.josh@agliodbs.com>
References: <402C213A.4030105@geizhals.at>
	<200402122228.41680.josh@agliodbs.com>

Josh Berkus wrote:

> 800MB for sort mem?   Are you sure you typed that correctly?   You must be 
> counting on not having a lot of concurrent queries.  It sure will speed up 
> index updating, though!

800MB is correct, yes... There are usually only 10-30 postgres processes 
  active (imagine 5-10 people working on the web front-end while cron 
jobs access the db occasionally). Very few queries can use such large 
amounts of memory for sorting, but they do exist.

> I think you might do well to experiment with using the checkpoint_delay and 
> checkpoint_sibilings settings in order to get more efficient batch processing 
> of updates while selects are going on.
[commit_*?]

I thought that could improve only concurrent transactions...

> What have you set max_fsm_relations and max_fsm_pages to?  The latter should 
> be very high for you, like 10,000,000

good guess ;-) the former is set to 10,000 (I'm not sure how useful this 
is for those temporary tables)

> For that matter, what *version* of PostgreSQL are you running?

7.4.1

> Also, make sure that your tables get vaccuumed regularly.  

There is a noticeable difference between a properly vacuumed db (nightly 
"vacuum full") and a non-vacuumed one and people will start complaining 
immediately if something goes wrong there...

> Well, a battery-backed RAID controller with a fast cache would certainly help.

http://www.lsilogic.com/products/ultra320_scsi_megaraid_storage_adapters/320x4128t.html
(RAID-5 with 9 15k rpm drives; at a hindsight, perhaps we should have 
tried a 0+1)

> You'll also be glad to know that a *lot* of the improvements in the upcoming 
> PostgreSQL 7.5 are aimed at giving better peformance on large, high-activity 
> databases like yours.

That's good to hear...

Regards,
  Marinos



view thread (10+ messages)  latest in thread

Message-ID: <402ED348.7000800@geizhals.at>
Permalink:  ../402ED348.7000800@geizhals.at/
Also on:    postgresql.org/message-id/402ED348.7000800@geizhals.at

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: mjy@geizhals.at, josh@agliodbs.com
  Subject: Re: optimization ideas for frequent, large(ish) updates
  In-Reply-To: <402ED348.7000800@geizhals.at>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox