MIME-Version: 1.0
References: <31cc6df9-53fe-3cd9-af5b-ac0d801163f4@iki.fi> <CAMT0RQR+NLJw6hmN88D7RPe=vUEr7DB52dKmS9tWPO_eX2Np0g@mail.gmail.com>
 <CAFiTN-vJqo4TSBpkQTJqhYz6CL0M=cPhQZUXnop1uDC47s2hBg@mail.gmail.com>
 <CALT9ZEH_ZT6Fv8KFmEPf2qM7g0Y0mdU-J4PQDVRN+vhn0CLSOQ@mail.gmail.com>
 <2c2665d2-c513-c12e-9097-9b1805bc2471@garret.ru> <CALDQ5NwfEA+s8O=G-frW6WfRA_yXj_AyhLf2Lb_ZgXWjH3nyEg@mail.gmail.com>
 <36f61a71-3bbb-b7b0-0d99-db5e69715af7@garret.ru> <CALDQ5NwotYMZtXA2z6EkBQ72jVBrfyrn-+a9xU8=w54VBQPOhg@mail.gmail.com>
In-Reply-To: <CALDQ5NwotYMZtXA2z6EkBQ72jVBrfyrn-+a9xU8=w54VBQPOhg@mail.gmail.com>
From: Hannu Krosing <hannuk@google.com>
Date: Thu, 15 Jun 2023 11:04:20 +0200
Message-ID: <CAMT0RQTRzDgVzj13zryKRCcwt5VvNwWcw_d+-dQQeh7GkF40Fg@mail.gmail.com>
Subject: Re: Let's make PostgreSQL multi-threaded
To: James Addison <jay@jp-hosting.net>
Cc: Konstantin Knizhnik <knizhnik@garret.ru>, Pavel Borisov <pashkin.elfe@gmail.com>, 
	Dilip Kumar <dilipbalaut@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, 
	pgsql-hackers <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAMT0RQTRzDgVzj13zryKRCcwt5VvNwWcw_d%2B-dQQeh7GkF40Fg%40mail.gmail.com>
Precedence: bulk

On Thu, Jun 15, 2023 at 10:41=E2=80=AFAM James Addison <jay@jp-hosting.net>=
 wrote:
>
> This is making me wonder about other performance/scalability areas
> that might not have been considered due to focus on the details of the
> existing codebase, but I'll save that for another thread and will try
> to learn more first.

A gradual move to more shared structures seems to be a way forward

It should get us all the benefits of threading minus the need for TLB
reloading and (in some cases) reduction of per-process virtual memory
mapping tables.

In any case we would need to implement all the locking and parallelism
management of these shared structures that are not there in the
current process architecture.

So a fair bit of work but also a clearly defined benefits of
1) reduced memory usage
2) no need to rebuild caches for each new connection
3) no need to track PREPARE statements inside connection poolers.

There can be extra complexity when different connections use the same
prepared statement name (say "PREP001") for different queries.
For this wel likely will need a good cooperation with connection
pooler where it passes some kind of client connection id along at the
transaction start