From: Martijn van Oosterhout <kleptog@svana.org>
To: Pailloncy Jean-Gerard <jg@rilk.com>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: Releasing memory during External sorting?
Date: Fri, 23 Sep 2005 20:32:46 +0200
Message-ID: <20050923183241.GA12774@svana.org> (raw)
In-Reply-To: <AC85B14C-FFB0-455B-88E3-6997B46269BE@rilk.com>
References: <1127468233.4145.178.camel@localhost.localdomain>
<3465.1127484579@sss.pgh.pa.us>
<AC85B14C-FFB0-455B-88E3-6997B46269BE@rilk.com>
On Fri, Sep 23, 2005 at 06:39:35PM +0200, Pailloncy Jean-Gerard wrote:
> >On most platforms it's quite unlikely that any memory would actually
> >get released back to the OS before transaction end, because the
> >memory blocks belonging to the tuplesort context will be intermixed
> >with blocks belonging to other contexts. So I think this is pretty
> >pointless. (If you can't afford to have the sort using all of
> >sort_mem, you've set sort_mem too large, anyway.)
> On OpenBSD 3.8 malloc use mmap(2) and no more sbrk.
> So, as soon as the bloc is free, it returns to the OS.
> Access to the freed pointer crashs immediatly.
Interesting point. Glibc also uses mmap() but only for allocations
greater than a few K, otherwise it's a waste of space.
I guess you would have to look into the postgresql allocator to see if
it doesn't divide the mmap()ed space up between multiple contexts.
Large allocations certainly appear to be passed off to malloc() but I
don't think execSort allocates all it's space in one go, it just counts
the space allocated by palloc().
So, unless someone goes and adds changes the tuplesort code to allocate
big blocks and use them only for tuples, I think you're going to run
into issues with data interleaved, meaning not much to give back to the
OS...
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: kleptog@svana.org, jg@rilk.com, pgsql-hackers@postgresql.org, tgl@sss.pgh.pa.us
Subject: Re: Releasing memory during External sorting?
In-Reply-To: <20050923183241.GA12774@svana.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox