MIME-Version: 1.0
References: 
 <CA+HiwqFfAY_ZFqN8wcAEMw71T9hM_kA8UtyHaZZEZtuT3UyogA@mail.gmail.com>
 <aNaZ6-DH5eVbFd9Y@momjian.us>
In-Reply-To: <aNaZ6-DH5eVbFd9Y@momjian.us>
From: Amit Langote <amitlangote09@gmail.com>
Date: Tue, 30 Sep 2025 11:15:02 +0900
Message-ID: 
 <CA+HiwqE0rknZnif+03QqynaeO-wLgEZzGxEk+bt8mCpERdg00Q@mail.gmail.com>
Subject: Re: Batching in executor
To: Bruce Momjian <bruce@momjian.us>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2BHiwqE0rknZnif%2B03QqynaeO-wLgEZzGxEk%2Bbt8mCpERdg00Q%40mail.gmail.com>
Precedence: bulk

Hi Bruce,

On Fri, Sep 26, 2025 at 10:49=E2=80=AFPM Bruce Momjian <bruce@momjian.us> w=
rote:
> On Fri, Sep 26, 2025 at 10:28:33PM +0900, Amit Langote wrote:
> > At PGConf.dev this year we had an unconference session [1] on whether
> > the community can support an additional batch executor. The discussion
> > there led me to start hacking on $subject. I have also had off-list
> > discussions on this topic in recent months with Andres and David, who
> > have offered useful thoughts.
> >
> > This patch series is an early attempt to make executor nodes pass
> > around batches of tuples instead of tuple-at-a-time slots. The main
> > motivation is to enable expression evaluation in batch form, which can
> > substantially reduce per-tuple overhead (mainly from function calls)
> > and open the door to further optimizations such as SIMD usage in
> > aggregate transition functions. We could even change algorithms of
> > some plan nodes to operate on batches when, for example, a child node
> > can return batches.
>
> For background, people might want to watch these two videos from POSETTE
> 2025.  The first video explains how data warehouse query needs are
> different from OLTP needs:
>
>         Building a PostgreSQL data warehouse
>         https://www.youtube.com/watch?v=3Dtpq4nfEoioE
>
> and the second one explains the executor optimizations done in PG 18:
>
>         Hacking Postgres Executor For Performance
>         https://www.youtube.com/watch?v=3DD3Ye9UlcR5Y
>
> I learned from these two videos that to handle new workloads, I need to
> think of the query demands differently, and of course can this be
> accomplished without hampering OLTP workloads?

Thanks for pointing to those talks -- I gave the second one. :-)

Yes, the idea here is to introduce batching without adding much
overhead or new code into the OLTP path.

--=20
Thanks, Amit Langote