To: "Josh Berkus" <josh@agliodbs.com>
Cc: "Simon Riggs" <simon@2ndquadrant.com>, <pgsql-performance@postgresql.org>
Subject: Re: 2GB or not 2GB
In-Reply-To: <200805311153.13186.josh@agliodbs.com> (Josh Berkus's message of
	"Sat\, 31 May 2008 11\:53\:13 -0700")
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (gnu/linux)
References: <200805281659.26945.josh@agliodbs.com>
	<1212219897.4120.195.camel@ebony.site>
	<200805311153.13186.josh@agliodbs.com>
From: Gregory Stark <stark@enterprisedb.com>
Organization: EnterpriseDB
Date: Sat, 31 May 2008 20:41:14 +0100
Message-ID: <87prr2jm79.fsf@oxford.xeocode.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Josh Berkus" <josh@agliodbs.com> writes:

> Simon,
>
>> There is an optimum for each specific sort.
>
> Well, if the optimum is something other than "as much as we can get", then we 
> still have a pretty serious issue with work_mem, no?

With the sort algorithm. The problem is that the database can't predict the
future and doesn't know how many more records will be arriving and how out of
order they will be.

What appears to be happening is that if you give the tape sort a large amount
of memory it keeps a large heap filling that memory. If that large heap
doesn't actually save any passes and doesn't reduce the number of output tapes
then it's just wasted cpu time to maintain such a large heap. If you have any
clever ideas on how to auto-size the heap based on how many output tapes it
will create or avoid then by all means speak up.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!