Subject: Re: Batching in executor

From: Daniil Davydov <[email-masked]>
Date: Mon, 26 Jan 2026 16:34:44 +0700

Hi,

On Mon, Dec 22, 2025 at 6:46 PM cca5507 <[email-masked]> wrote:
> Some comments for v4:

Agree with your (1)-(4) comments.

> 5) heapgettup_pagemode_batch()
> If the scan key filters out all tuples on a page, we may return 0 before reaching the end of scan, right?
>

Yes. I think that we should advance to the next page if "nout == 0"
at the end of walking through the rs_vistuples.

> 6) heap_begin_batch()
> ```
>         hb = palloc(sizeof(HeapBatch));
>         hb->tupdata = palloc(sizeof(HeapTupleData) * maxitems);
> ```
> Can we just use one palloc() for cache-friendly?
>

Actually, we are using memory context when calling the palloc function.
I.e. in the general case it will not cause memory allocation. But of course
there is no guarantee for it. I saw a lot of places in the code where we
are calling the palloc function several times in a row, so I guess that
this is OK.

If you will decide to leave these palloc calls, I suggest using the
palloc_object/palloc_array functions.

A few other comments on 0001 patch:

1)
+    void       *(*scan_begin_batch)(TableScanDesc sscan, int maxitems);
Is it syntactically correct?

2)
    /* Initialize static fields of HeapTupleData. Row bodies remain on page. */
    relid = RelationGetRelid(sscan->rs_rd);
    for (int i = 0; i < maxitems; i++)
        hb->tupdata[i].t_tableOid = relid;

Is it really necessary? I see that we are setting this field inside the
heapgettup_pagemode_batch function.

A few comment on 0002 patch:

1)
I guess that you should rebase your patches on the current master, because
the second patch doesn't apply.

2)
Maybe we can use tuplestore for tuples stored in TupleBatch? It is just a
proposal - I didn't check this idea carefully.

--
Best regards,
Daniil Davydov

---

From: Amit Langote <[email-masked]>
Date: Tue, 27 Jan 2026 12:00:13 +0900

Hi,

> at the end of walking through the rs_vistuples.

Next version (v5) does it like that.

> palloc_object/palloc_array functions.

I think combining those individual pallocs into one is a good idea, so
v5 does it like that.

> Is it syntactically correct?

Yes, it compiles fine. Though I'm considering changing the return type
to a struct with common fields (like nitems) so callers can access
them directly without callback indirection.  Maybe call it TAMBatch or
something.

> heapgettup_pagemode_batch function.

It's intentional -- by initializing t_tableOid once in
heap_begin_batch, we can avoid setting it repeatedly for every tuple
in heapgettup_pagemode_batch().  Though you are correct to point out
the redundant assignment in heapgettup_pagemode_batch(); I'll change
it to an Assert instead. The relid doesn't change during the scan.

> the second patch doesn't apply.

Yep, will do.

> proposal - I didn't check this idea carefully.

TupleBatch is designed to be lightweight -- it holds an array of
TupleTableSlot pointers, not the tuple data itself. The slots
reference tuples that remain in the AM's buffer (no copy). Using
tuplestore would require materializing tuples, adding overhead we're
trying to avoid.

-- 
Thanks, Amit Langote

---

From: Amit Langote <[email-masked]>
Date: Thu, 29 Jan 2026 08:35:13 +0100

Hi,

Here is v5 of the patch series.

Patches 0001-0003 add the core batching infrastructure. 0001 adds the
batch table AM API with heapam implementation, 0002 wires up SeqScan
to use it (still returning one slot at a time), and 0003 adds EXPLAIN
(BATCHES). I'd love to hear people's thoughts around TupleBatch
structure added in 0002. I thought about making it a separate patch so
that 0002 will still populate the single ScanState.ss_scanTupleSlot,
but that means we'd still have to call the TAM callback to populate
the tuple in the TAM's batch struct into the slot, defeating the whole
point. With TupleBatch, you have executor_batch_rows number of slots
which are filled in one TAM callback (materialize_all) call. So I
decided to keep the TupleBatch and related things in 0002.

For scans without quals, batching shows 20-30% improvement with no
visible regressions when batching is disabled (batch_rows=0):

SELECT * FROM t LIMIT n (no qual)

  Rows      Master    batch=0   %diff    batch=64  %diff
  ------   --------   -------   -----    --------  -----
  1M        12.42 ms  11.96 ms   3.7%     8.56 ms  31.0%
  3M        38.95 ms  38.92 ms   0.1%    28.59 ms  26.6%
  10M      153.64 ms 150.28 ms   2.2%   112.95 ms  26.5%

(%diff: positive = faster than master, negative = slower)

Patches 0004-0005 add batched qual evaluation and are more
experimental (see below on why 0005 exists). For quals referencing
early columns, the improvement is significant:

  SELECT * FROM t WHERE a = 0 ... OFFSET n (qual on 1st col)

  Rows      Master    batch=64  %diff
  ------   --------   --------  -----
  1M        30.19 ms  15.55 ms  48.5%
  3M        92.47 ms  50.01 ms  45.9%
  10M      325.58 ms 211.83 ms  34.9%

However, for quals on later columns (e.g., 15th), batching provides no
benefit - deformation dominates and batching doesn't help:

  SELECT * FROM t WHERE o = 0 ... OFFSET n (qual on 15th col)

  Rows      Master    batch=64  %diff
  ------   --------   --------  -----
  1M        44.14 ms  44.56 ms  -0.9%
  3M       133.89 ms 137.77 ms  -2.9%
  10M      503.33 ms 528.88 ms  -5.1%

I don't have a satisfactory explanation for why batching doesn't help
the deform-heavy case at all. One would expect at least some benefit
from reduced per-tuple overhead, but that's not materializing.

I've also been struggling to understand why 0004 affects the per-tuple
path even when batch_rows=0. For quals with 0% selectivity (all rows
fail the qual), perf shows ExecInterpExpr is noticeably hotter with
the patched code compared to master, even though batching is disabled:

  SELECT * FROM t WHERE a = 0 ... OFFSET n (0% selectivity)

  Rows      Master    batch=0   %diff    batch=64  %diff
  ------   --------   -------   -----    --------  -----
  1M        24.37 ms  28.67 ms -17.6%    12.46 ms  48.9%
  3M        73.95 ms  85.07 ms -15.0%    41.64 ms  43.7%
  10M      287.63 ms 316.81 ms -10.1%   188.01 ms  34.6%

Compare that to 100% selectivity (all rows pass), where there's no regression:

  SELECT * FROM t WHERE a > 0 ... OFFSET n (100% selectivity)

  Rows      Master    batch=0   %diff    batch=64  %diff
  ------   --------   -------   -----    --------  -----
  1M        29.44 ms  29.10 ms   1.2%    16.61 ms  43.6%
  3M        91.22 ms  90.28 ms   1.0%    54.10 ms  40.7%
  10M      360.77 ms 331.25 ms   8.2%   224.00 ms  37.9%

I tried moving batch opcodes to a separate interpreter (0005) thinking
it might be register pressure or jump table effects from adding cases
to ExecInterpExpr's switch. With 0005, the generated assembly for
ExecInterpExpr looks identical to master (same stack frame size, same
epilogue), yet the performance still differs. Specifically, the ldp
instruction in the function epilogue shows 53% hotness in patched vs
35% in master. We still need placeholder entries in the dispatch
table, so it's unclear if this fully isolates the per-tuple path. I'll
continue looking at perf, but I feel like at a bit of a loss here and
would appreciate any insights.

Other changes worth noting:

- I removed the BatchVector intermediate representation that copied
Datums into columnar arrays before qual evaluation (it used to be in
the batched qual patch 0004). Now quals access batch slots' tts_values
directly. This simplifies the code and the copy overhead wasn't paying
off. If we pursue serious vectorization later, this may need to be
revisited, but removing it doesn't degrade performance.

--
Thanks, Amit Langote

---

From: Amit Langote <[email-masked]>
Date: Thu, 29 Jan 2026 11:04:17 +0100

>   Rows      Master    batch=0   %diff    batch=64  %diff

Oops, I meant SELECT * FROM t LIMIT 1 OFFSET n (no qual).

-- 
Thanks, Amit Langote

---

From: Junwang Zhao <[email-masked]>
Date: Sun, 1 Feb 2026 22:49:31 +0800

Hi Amit,

>   Rows      Master    batch=0   %diff    batch=64  %diff

Here are some comments for v5:

0001:

+/*
+ * heap_scan_begin_batch
+ *
+ * Allocate a HeapBatch with space for 'maxitems' tuple headers. No pin is
+ * taken here. Memory is allocated under the scan's memory context.
+ */
+void *
+heap_begin_batch(TableScanDesc sscan, int maxitems)

+/*
+ * heap_scan_end_batch
+ *
+ * Release any outstanding pin and free the batch allocations. Caller will
+ * not use 'am_batch' after this point.
+ */
+void
+heap_end_batch(TableScanDesc sscan, void *am_batch)

These function names are not consistent with comments.

0002:

+/*
+ * heap_scan_materialize_all
+ *
+ * Bind all tuples of the current batch into 'slots'. We bind the
+ * HeapTupleData header that points into the pinned page. No per-row copy.
+ */
+void
+heap_materialize_batch_all(void *am_batch, TupleTableSlot **slots, int n)

ditto.

+const TupleBatchOps *
+table_batch_callbacks(Relation relation)
+{
+ if (relation->rd_tableam)
+ return relation->rd_tableam->batch_callbacks(relation);
+ elog(ERROR, "relation does not support TupleBatch operations");
+}

Is there any chance this batch_callbacks can be NULL? In that case it
can cause a segfault. I felt changing to
if (relation->rd_tableam && relation->rd_tableam->batch_callbacks)
should be more robust, but then I found table_slot_callbacks follow
the same pattern, so this shouldn't be a problem.

0003:

+++ b/src/include/executor/execBatch.h
@@ -13,6 +13,8 @@
#ifndef EXECBATCH_H
#define EXECBATCH_H
+#include <limits.h>

I guess the reason for including this header is because of the use
of INT_MAX, so maybe put that line into execBatch.c?

-- 
Regards
Junwang Zhao

---

From: "cca5507" <[email-masked]>
Date: Tue, 3 Feb 2026 21:30:15 +0800

Hi,

Some comments for v5:

0001
====

1) heap_begin_batch()

```
	/* Single allocation for HeapBatch header + tupdata array */
	alloc_size = sizeof(HeapBatch) + sizeof(HeapTupleData) * maxitems;
	hb = palloc(alloc_size);
	hb->tupdata = (HeapTupleData *) ((char *) hb + sizeof(HeapBatch));
```

Do we need a MAXALIGN() here to avoid unaligned access? Something like this:

```
	/* Single allocation for HeapBatch header + tupdata array */
	alloc_size = MAXALIGN(sizeof(HeapBatch)) + sizeof(HeapTupleData) * maxitems;
	hb = palloc(alloc_size);
	hb->tupdata = (HeapTupleData *) ((char *) hb + MAXALIGN(sizeof(HeapBatch)));
```

Or how about just using zero-length array:

```
typedef struct HeapBatch
{
	Buffer			buf;
	int				maxitems;
	int				nitems;
	HeapTupleData	tupdata[FLEXIBLE_ARRAY_MEMBER];
} HeapBatch;

// and
hb = palloc(offsetof(HeapBatch, tupdata) + sizeof(HeapTupleData) * maxitems);
```

2) pgstat_count_heap_getnext_batch()

```
#define pgstat_count_heap_getnext_batch(rel, n)						\
	do {															\
		if (pgstat_should_count_relation(rel))						\
			(rel)->pgstat_info->counts.tuples_returned += n;		\
	} while (0)
```

"+= n" -> "+= (n)", just like pgstat_count_index_tuples().

0002
====

1) TupleBatchCreate()

```
	/* Single allocation for TupleBatch + inslots + outslots arrays */
	alloc_size = sizeof(TupleBatch) + 2 * sizeof(TupleTableSlot *) * capacity;
	b = palloc(alloc_size);
	inslots = (TupleTableSlot **) ((char *) b + sizeof(TupleBatch));
	outslots = (TupleTableSlot **) ((char *) b + sizeof(TupleBatch) +
		sizeof(TupleTableSlot *) * capacity);
```

Do we need a MAXALIGN() here to avoid unaligned access?

2) TupleBatchReset()

```
	for (int i = 0; i < b->maxslots; i++)
	{
		ExecClearTuple(b->inslots[i]);
		if (drop_slots)
			ExecDropSingleTupleTableSlot(b->inslots[i]);
	}
```

ExecDropSingleTupleTableSlot() will call ExecClearTuple(), so ExecClearTuple() will be
called twice if drop_slots is true, I think we can avoid this.

3) ScanCanUseBatching()

In heap_beginscan(), we may disable page-at-a-time mode:

```
	/*
	 * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
	 */
	if (!(snapshot && IsMVCCSnapshot(snapshot)))
		scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
```

It seems that ScanCanUseBatching() didn't consider this.

4) struct TupleBatch

```
	struct TupleTableSlot **inslots; /* slots for tuples read "into" batch */
	struct TupleTableSlot **outslots; /* slots for tuples going "out of"
									   * batch */
	struct TupleTableSlot **activeslots;
```

I think we can remove the word "struct".

5) ExecScanExtendedBatchSlot()

```
		/* Get next input slot from current batch, or refill */
		if (!TupleBatchHasMore(b))
		{
			if (!accessBatchMtd(node))
				return NULL;
		}
```

I think we cannot just return NULL here, see comments in ExecScanExtended():

```
		/*
		 * if the slot returned by the accessMtd contains NULL, then it means
		 * there is nothing more to scan so we just return an empty slot,
		 * being careful to use the projection result slot so it has correct
		 * tupleDesc.
		 */
		if (TupIsNull(slot))
		{
			if (projInfo)
				return ExecClearTuple(projInfo->pi_state.resultslot);
			else
				return slot;
		}
```

And why not just write this function like ExecScanExtended() and ExecScanFetch()?

--
Regards,
ChangAo Chen

---

From: Junwang Zhao <[email-masked]>
Date: Tue, 3 Feb 2026 23:54:48 +0800

> Do we need a MAXALIGN() here to avoid unaligned access? Something like this:

TBH I don't think this single allocation helps too much, it's not on
the hot path,
but makes the code harder to read ;(

-- 
Regards
Junwang Zhao

---

From: Amit Langote <[email-masked]>
Date: Tue, 24 Mar 2026 09:59:33 +0900

Hi,

Here is a significantly revised version of the patch series. A lot has
changed since the January submission, so I want to summarize the
design changes before getting into the patches.  I think it does
address the points in the two reviews that landed since v5 but maybe a
bunch of points became moot after my rewrite of the relevant portions
(thanks Junwang and ChangAo for the review in any case).

At this point it might be better to think of this as targeting v20,
except that if there is review bandwidth in the remaining two weeks
before the v19 feature freeze, the rs_vistuples[] change described
below as a standalone improvement to the existing pagemode scan path
could be considered for v19, though that too is an optimistic
scenario.

It is also worth noting that Andres identified a number of
inefficiencies in the existing scan path in:

Re: unnecessary executor overheads around seqscans
https://postgr.es/m/xzflwwjtwxin3dxziyblrnygy3gfygo5dsuw6ltcoha73ecmnf%40nh6nonzta7kw

that are worth fixing independently of batching. Some of those fixes
may be better pursued first, both because they benefit all scan paths
and because they would make batching's gains more honest.

Separately, after looking at the previous version, Andres pointed out
offlist two fundamental issues with the patch's design:

* The heapam implementation (in a version of the patch I didn't post
to the thread) duplicated heap_prepare_pagescan() logic in a separate
batch-specific code path, which is not acceptable as changes should
benefit the existing slot interface too.  Code duplication is not good
either from a future maintainability aspect. The v5 version of that
code is not great in that respect either; it instead duplicated
heapggettup_pagemode() to slap batching on it.

* Allocating executor_batch_rows slots on the executor side to receive
rows from the AM adds significant overhead for slot initialization and
management, and for non-row-organized AMs that do not produce
individual rows at all, those slots would never be meaningfully
populated.

In any case, he just wasn't a fan of the slot-array approach the
moment I mentioned it. The previous version had two slot arrays,
inslots and outslots, of TTSOpsHeapTuple type (not
TTSOpsBufferHeapTuple because buffer pins were managed by the batch
code, which has its own modularity/correctness issues), populated via
a materialize_all callback. A batch qual evaluator would copy
qualifying tuples into outslots, with an activeslots pointer switching
between the two depending on whether batch qual evaluation was used.

The new design addresses both issues and differs from the previous
version in several other ways:

 * Single slot instead of slot arrays: there is a single
TupleTableSlot, reusing the scan node's ss_ScanTupleSlot whose type
was already determined by the AM via table_slot_callbacks().  The slot
is re-pointed to each HeapTuple in the current buffer page via a new
repoint_slot AM callback, with no materialization or copying.  Tuples
are returned one by one from the executor's perspective, but the AM
serves them in page-sized batches from pre-built HeapTupleData
descriptors in rs_vistuples[], avoiding repeated descent into heapam
per tuple.  This is heapam's implementation of the batch interface;
there is no intention to force other AMs into the same row-oriented
model.

 * Batch qual evaluator not included: with the single-slot model,
quals are evaluated per tuple via the existing ExecQual path after
each repoint_slot call.  A natural next step would be a new opcode
(EEOP) that calls repoint_slot() internally within expression
evaluation, allowing ExecQual to advance through multiple tuples from
the same batch without returning to the scan node each time, with qual
results accumulated in a bitmask in ExprState.  The details of that
will be worked out in a follow-on series.

* heapgettup_pagemode_batch() gone: patch 0001 (described below) makes
HeapScanDesc store full HeapTupleData entries in rs_vistuples[], which
allows heap_getnextbatch() to simply advance a slice pointer into that
array without any additional copying or re-entering heap code, making
a separate batch-specific scan function unnecessary.

 * TupleBatch renamed to RowBatch: "row batch" is more natural
terminology for this concept and also consistent with how similar
abstractions are named in columnar and OLAP systems.

 * AM callbacks now take RowBatch directly: previously
heap_getnextbatch() returned a void pointer that the executor would
store into RowBatch.am_payload, because only the executor knew the
internals of RowBatch.  Now the AM receives RowBatch directly as a
parameter and can populate it without the executor acting as an
intermediary.  This is also why RowBatch is introduced in its own
patch ahead of the AM API addition, so the struct definition is
available to both sides.

Patch 0001 changes rs_vistuples[] to store full HeapTupleData entries
instead of OffsetNumbers, as a standalone improvement to the existing
pagemode scan path. Measured on a pg_prewarm'd  (also vaccum freeze'd
in the all-visible case) table with 1M/5M/10M rows:

  query                           all-visible      not-all-visible
  count(*)                        -0.2% to +0.9%   -0.4% to +0.5%
  count(*) WHERE id % 10 = 0     -1.1% to +3.4%   +0.2% to +1.5%
  SELECT * LIMIT 1 OFFSET N      -2.2% to -0.6%   -0.9% to +6.6%
  SELECT * WHERE id%10=0 LIMIT   -0.8% to +3.9%   +0.9% to +9.6%

No significant regression on either page type. The structural
improvement is most visible on not-all-visible pages where
HeapTupleSatisfiesMVCCBatch() already reads every tuple header during
visibility checks, so persisting the result into rs_vistuples[]
eliminates the downstream re-read (in heapgettupe_pagemode()) with no
measurable overhead.  That said, these numbers are somewhat noisy on
my machine.  Results on other machines would be welcome.

Patches 0002-0005 add the RowBatch infrastructure, the batch AM API
and heapam implementation including seqscan variants that use the new
scan_getnextbatch() API, and EXPLAIN (ANALYZE, BATCHES) support,
respectively. With batching enabled (executor_batch_rows=300,
~MaxHeapTuplesPerPage):

  query                           all-visible    not-all-visible
  count(*)                        +11 to +15%    +9 to +13%
  count(*) WHERE id % 10 = 0     +6 to +11%     +10 to +14%
  SELECT * LIMIT 1 OFFSET N      +16 to +19%    +16 to +22%
  SELECT * WHERE id%10=0 LIMIT   +8 to +10%     +8 to +13%

With executor_batch_rows=0, results are within noise of master across
all query types and sizes, confirming no regression from the
infrastructure changes themselves.  The not-all-visible results tend
to show slightly higher gains than the all-visible case. This is
likely because the existing heapam code is more optimized for the
all-visible path, so the not-all-visible path, which goes through
HeapTupleSatisfiesMVCCBatch() for per-tuple visibility checks, has
more headroom that batching can exploit.

Setting aside the current series for a moment, there are some broader
design questions worth raising while we have attention on this area.
Some of these echo points Tomas raised in his first reply on this
thread, and I am reiterating them deliberately since I have not
managed to fully address them on my own or I simply didn't need to for
the TAM-to-scan-node batching and think they would benefit from wider
input rather than just my own iteration.

We should also start thinking about other ways the executor can
consume batch rows, not always assuming they are presented as
HeapTupleData. For instance, an AM could expose decoded column arrays
directly to operators that can consume them, bypassing slot-based
deform entirely, or a columnar AM could implement scan_getnextbatch by
decoding column strips directly into the batch without going through
per-tuple HeapTupleData at all. Feedback on whether the current
RowBatch design and the choices made in the scan_getnextbatch and
RowBatchOps API make that sort of thing harder than it needs to be
would be appreciated. For example, heapam's implementation of
scan_getnextbatch uses a single TTSOpsBufferHeapTuple slot re-pointed
to HeapTupleData entries one at a time via repoint_slot in
RowBatchHeapOps. That works for heapam but a columnar AM could
implement scan_getnextbatch to decode column strips directly into
arrays in the batch, with no per-row repoint step needed at all. Any
adjustments that would make RowBatch more AM-agnostic are worth
discussing now before the design hardens.

There are also broader open questions about how far the batch model
can extend beyond the scan node. Qual pushdown into the AM has been
discussed in nearby threads and would be one way to allow expression
evaluation to happen before data reaches the executor proper, though
that is a separate effort. For the purposes of this series, expression
evaluation still happens in the executor after scan_getnextbatch
returns. If the scan node does not project, the buffer heap slot is
passed directly to the parent node, which calls slot callbacks to
deform as needed. But once a node above projects, aggregates, or
joins, the notion of a page-sized batch from a single AM loses its
meaning and virtual slots take over. Whether RowBatch is usable or
meaningful beyond the scan/TAM boundary in any form, and whether the
core executor will ever have non-HeapTupleData batch consumption paths
or leave that entirely to extensions, are open questions worth
discussing.

For RowBatch to eventually play the role that TupleTableSlot plays for
row-at-a-time execution, something inside it would need to serve as
the common currency for batch data, analogous to TupleTableSlot's
datum/isnull arrays. Column arrays are the obvious direction, but even
that leaves open the question of representation. PostgreSQL's Datum is
a pointer-sized abstraction that boxes everything, whereas vectorized
systems use typed packed arrays of native types with validity
bitmasks, which is a significant part of why tight vectorized loops
are fast there. Whether column arrays of Datum would be good enough,
or whether going further toward typed packed arrays would be necessary
to get meaningful vectorization, is a deeper design question that this
series deliberately does not try to answer.

Even though the focus is on getting batching working at the scan/TAM
boundary first, thoughts on any of these points would be welcome.

--
Thanks, Amit Langote

---

From: Amit Langote <[email-masked]>
Date: Mon, 6 Apr 2026 21:02:35 +0900

> boundary first, thoughts on any of these points would be welcome.

Rebased.

-- 
Thanks, Amit Langote