Feedback-ID: id4a34324:Fastmail
Date: Thu, 8 Jun 2023 09:00:02 -0700
From: Andres Freund <andres@anarazel.de>
To: Hannu Krosing <hannuk@google.com>
Cc: "Jonathan S. Katz" <jkatz@postgresql.org>,
	Heikki Linnakangas <hlinnaka@iki.fi>, Tom Lane <tgl@sss.pgh.pa.us>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Let's make PostgreSQL multi-threaded
Message-ID: <20230608160002.qeollvbvfcrbq7sh@awork3.anarazel.de>
References: <31cc6df9-53fe-3cd9-af5b-ac0d801163f4@iki.fi>
 <4178104.1685978307@sss.pgh.pa.us>
 <4ce6c0f8-e8a4-1672-93fd-49d3fa975ee5@iki.fi>
 <cd27db0c-01ae-ba4d-f141-8712cbb4188c@postgresql.org>
 <20230607213721.al3etgcgtija3ytz@awork3.anarazel.de>
 <CAMT0RQTAFrx5xVJ9SXyjP71=SmU+U9+4dDm+XEZWWoy6pbCF5g@mail.gmail.com>
 <CAMT0RQR1kAEhHmiy2SfhqbDvoq3mrezcENM=3vozL5cpkNDPOA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAMT0RQR1kAEhHmiy2SfhqbDvoq3mrezcENM=3vozL5cpkNDPOA@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/20230608160002.qeollvbvfcrbq7sh%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2023-06-08 12:15:58 +0200, Hannu Krosing wrote:
> On Thu, Jun 8, 2023 at 11:54 AM Hannu Krosing <hannuk@google.com> wrote:
> >
> > On Wed, Jun 7, 2023 at 11:37 PM Andres Freund <andres@anarazel.de> wrote:
> > >
> > > Hi,
> > >
> > > On 2023-06-05 13:40:13 -0400, Jonathan S. Katz wrote:
> > > > 2. While I wouldn't want to necessarily discourage a moonshot effort, I
> > > > would ask if developer time could be better spent on tackling some of the
> > > > other problems around vertical scalability? Per some PGCon discussions,
> > > > there's still room for improvement in how PostgreSQL can best utilize
> > > > resources available very large "commodity" machines (a 448-core / 24TB RAM
> > > > instance comes to mind).
> > >
> > > I think we're starting to hit quite a few limits related to the process model,
> > > particularly on bigger machines. The overhead of cross-process context
> > > switches is inherently higher than switching between threads in the same
> > > process - and my suspicion is that that overhead will continue to
> > > increase. Once you have a significant number of connections we end up spending
> > > a *lot* of time in TLB misses, and that's inherent to the process model,
> > > because you can't share the TLB across processes.
> >
> >
> > This part was touched in the "AMA with a Linux Kernale Hacker"
> > Unconference session where he mentioned that the had proposed a
> > 'mshare' syscall for this.

As-is that'd just lead to sharing page table, not the TLB. I don't think you
currently do sharing of the TLB for parts of your address space on x86
hardware. It's possible that something like that gets added to future
hardware, but ...


> Also, the *static* huge pages already let you solve this problem now
> by sharing the page tables

You don't share the page tables with huge pages on linux.


- Andres