From: Andres Freund <andres@2ndquadrant.com>
To: Heikki Linnakangas <hlinnakangas@vmware.com>
Cc: Noah Misch <noah@leadboat.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: Something fishy happening on frogmouth
Date: Mon, 4 Nov 2013 12:19:12 +0100
Message-ID: <20131104111912.GD17233@awork2.anarazel.de> (raw)
In-Reply-To: <52778157.9050907@vmware.com>
References: <20131031095021.GA31628@alap2.anarazel.de>
<CA+Tgmoa6gJAPeMKCK77xzUbhqriO0C+24Ow7ezz4GP8wHCF-EA@mail.gmail.com>
<22875.1383229757@sss.pgh.pa.us>
<CA+Tgmobk7nL8MAQdOVPTHzn2pA7aP-kfYvjHTqUsZnbu3FsOxQ@mail.gmail.com>
<5272EC6A.2020004@vmware.com>
<CA+TgmoZhFZkdywhCwu-kB9CdrPz_gZ6scSjSjBpjE3W4FPAu=g@mail.gmail.com>
<20131101162230.GA649568@tornado.leadboat.com>
<52775A83.7090206@vmware.com>
<20131104095521.GI3567@awork2.anarazel.de>
<52778157.9050907@vmware.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 2013-11-04 13:13:27 +0200, Heikki Linnakangas wrote:
> On 04.11.2013 11:55, Andres Freund wrote:
> >On 2013-11-04 10:27:47 +0200, Heikki Linnakangas wrote:
> >>Postmaster creates the POSIX shared memory object at startup, by calling
> >>shm_open(), and immediately calls shm_unlink on it. That way, once all the
> >>processes have exited, the object will be removed automatically. Child
> >>processes inherit the file descriptor at fork(), and don't need to call
> >>shm_open, just mmap().
> >
> >Uh. Won't that completely and utterly remove the point of dsm which is
> >that you can create segments *after* startup? We surely don't want to
> >start overallocating enough shmem so we don't ever dynamically need to
> >allocate segments.
> You don't need to allocate the shared memory beforehand, only create the
> file descriptor. Note that the size of the segment is specified in the
> shm_open() call, but the mmap() that comes later.
>
> If we need a large amount of small segments, so that it's not feasible to
> shm_open() them all at postmaster startup, you could shm_open() just one
> segment, and carve out smaller segments from it by mmap()ing at different
> offsets.
That quickly will result in fragmentation which we don't have the tools
to handle.
> >Also, I don't think it's portable across platforms to access segments
> >that already have been unlinked.
>
> See
> http://pubs.opengroup.org/onlinepubs/009695299/functions/shm_unlink.html:
> "If one or more references to the shared memory object exist when the object
> is unlinked, the name shall be removed before shm_unlink() returns, but the
> removal of the memory object contents shall be postponed until all open and
> map references to the shared memory object have been removed."
We also support sysv shmem and have the same cleanup problem there.
> That doesn't explicitly say that a new shm_open() on the file descriptor
> must still work after shm_unlink, but I would be surprised if there is a
> platform where it doesn't.
Probably true.
> >I think this is looking for a solution without an actually relevant
> >problem disregarding the actual problem space.
To make that clearer: I think the discussions about making it impossible
to leak segments after rm -rf are the irrelevant problem.
> I agree. What are these dynamic shared memory segments supposed to be used
> for?
Parallel sort and stuff like that.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@2ndquadrant.com, hlinnakangas@vmware.com, noah@leadboat.com, robertmhaas@gmail.com, tgl@sss.pgh.pa.us, andrew@dunslane.net
Subject: Re: Something fishy happening on frogmouth
In-Reply-To: <20131104111912.GD17233@awork2.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox