Feedback-ID: id4a34324:Fastmail
Date: Thu, 8 Jun 2023 11:41:00 -0700
From: Andres Freund <andres@anarazel.de>
To: Hannu Krosing <hannuk@google.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Matthias van de Meent <boekewurm+postgres@gmail.com>,
	"Jonathan S. Katz" <jkatz@postgresql.org>,
	Heikki Linnakangas <hlinnaka@iki.fi>, Tom Lane <tgl@sss.pgh.pa.us>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Let's make PostgreSQL multi-threaded
Message-ID: <20230608184100.xbvis2dtwjrfu754@awork3.anarazel.de>
References: <31cc6df9-53fe-3cd9-af5b-ac0d801163f4@iki.fi>
 <4178104.1685978307@sss.pgh.pa.us>
 <4ce6c0f8-e8a4-1672-93fd-49d3fa975ee5@iki.fi>
 <cd27db0c-01ae-ba4d-f141-8712cbb4188c@postgresql.org>
 <20230607213721.al3etgcgtija3ytz@awork3.anarazel.de>
 <CAMT0RQTAFrx5xVJ9SXyjP71=SmU+U9+4dDm+XEZWWoy6pbCF5g@mail.gmail.com>
 <CAEze2Whojjh++1Ybw8ohxo46DPeDS4aD5dkPsOse4Q-8r56awQ@mail.gmail.com>
 <CAMT0RQSfoUCNskuweVBhmEiWh76q+eqDhX+5_bWpq8Qq-KuTTg@mail.gmail.com>
 <CA+TgmoZhySzy4QuupAMCi3Fs7ZDwwKkUE9VEgxF2nn71Bo-FXQ@mail.gmail.com>
 <CAMT0RQTCFec0J4hQDcpC0CZ4F-7VWFZXpREgaRU-96bVQ1pj2Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAMT0RQTCFec0J4hQDcpC0CZ4F-7VWFZXpREgaRU-96bVQ1pj2Q@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/20230608184100.xbvis2dtwjrfu754%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2023-06-08 17:02:08 +0200, Hannu Krosing wrote:
> On Thu, Jun 8, 2023 at 4:56 PM Robert Haas <robertmhaas@gmail.com> wrote:
> >
> > On Thu, Jun 8, 2023 at 8:44 AM Hannu Krosing <hannuk@google.com> wrote:
> > > > That sounds like a bad idea, dynamic shared memory is more expensive
> > > > to maintain than our static shared memory systems, not in the least
> > > > because DSM is not guaranteed to share the same addresses in each
> > > > process' address space.
> > >
> > > Then this too needs to be fixed
> >
> > Honestly, I'm struggling to respond to this non-sarcastically. I mean,
> > I was the one who implemented DSM. Do you think it works the way that
> > it works because I considered doing something smart and decided to do
> > something dumb instead?
>
> No, I meant that this needs to be fixed at OS level, by being able to
> use the same mapping.
>
> We should not shy away from asking the OS people for adding the useful
> features still missing.

There's a large part of this that is about hardware, not software. And
honestly, for most of the problems the answer is to just use threads. Adding
complexity to operating systems to make odd architectures like postgres'
better is a pretty dubious proposition.

I don't think we have even remotely enough influence on CPU design to make
e.g. *partial* TLB sharing across processes a thing.


> It was mentioned in the Unconference Kernel Hacker AMA talk and said
> kernel hacker works for Oracle, andf they also seemed to be needing
> this :)

The proposals around that don't really help us all that much. Sharing the page
table will be a bit more efficient, but it won't really change anything
dramatically.  From what I understand they are primarily interested in
changing properties of a memory mapping across multiple processes, e.g. making
some memory executable and have that reflected in all processes. I don't think
this will help us much.

Greetings,

Andres Freund